Test method, device and equipment for surface mounting production and storage medium

Abstract

The invention provides a test method and device for surface mounting production, equipment and a storage medium. The method comprises the steps of collecting image information and an electric signal after surface mounting of a printed circuit board; pre-processing the image information to generate a pre-processed image; performing ellipse fitting on the preprocessed image to determine geometric features of the bonding pad; performing cross-modal feature fusion on the electrical signal feature vectors corresponding to the electrical signals and the geometric features to construct a multi-dimensional defect description matrix; inputting the multi-dimensional defect description matrix into a neural network model, and outputting a defect type classification label and a corresponding confidence score; and generating a dynamic test path planning scheme through an optimization algorithm according to the defect type classification label and the confidence score. By extracting the geometric features of the bonding pad, constructing the defect description matrix fusing the electric signals and the geometric information, and performing defect identification through the neural network, deep collaboration and information complementation between modals are realized, and the identification capability of complex defects is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of circuit board mounting technology, and in particular to a testing method, device, equipment and storage medium for surface mount production. Background Art

[0002] As electronic products continue to evolve toward higher density, miniaturization, and multifunctionality, surface mount technology (SMT) is increasingly being used in printed circuit board (PCB) manufacturing, becoming a core component of the modern electronics manufacturing industry. On surface mount production lines, the quality of component placement directly impacts overall device performance and product reliability. Therefore, rapid and accurate post-placement inspection and evaluation is a critical quality control task. Traditional inspection methods, such as automated optical inspection (AOI) and electrical performance testing (ETT), typically identify defects using either image or telecommunication signal dimensions, but lack effective linkage between the two. This is particularly true for unusually shaped components, irregular pads, and complex solder joint structures. Single-modal data often suffers from insufficient information dimensionality and localized judgment failures, resulting in the inability to accurately identify critical defects such as cold solder joints, bridging, and missed solder joints. For example, on a high-speed SMT production line, due to the inconsistent representation of minute pad offsets and weak electrical signal fluctuations across different modalities, existing inspection systems struggle to establish a unified judgment basis, leading to batch quality misjudgments. Summary of the Invention

[0003] The present application provides a testing method, apparatus, equipment and storage medium for surface mount production, which are used to solve the problems of insufficient information dimension and local judgment failure of single modal data in related technologies.

[0004] A first aspect of the present application provides a test method for surface mount production, the test method for surface mount production comprising: Collect image information and electrical signals after printed circuit board surface mounting; Generate a pre-processed image of the edge of the protruding pad by performing median filtering and binarization processing on the image data corresponding to the image information; Performing ellipse fitting on the preprocessed image to determine geometric features of the pad; Performing cross-modal feature fusion on the electrical signal feature vector corresponding to the electrical signal and the geometric feature to construct a multi-dimensional defect description matrix; Input the multi-dimensional defect description matrix into a pre-trained neural network model, and output defect type classification labels and corresponding confidence scores; A dynamic test path planning scheme is generated through an optimization algorithm according to the defect type classification label and the confidence score.

[0005] Optionally, in a first implementation of the first aspect of the present application, the step of generating a preprocessed image of the protruding pad edge by performing median filtering and binarization on the image data corresponding to the image information includes: Input the image data corresponding to the image information into a median filter, traverse the image pixel matrix through a preset sliding window, determine the median of the pixel grayscale value in each window, and generate a denoised image; Performing grayscale histogram statistics on the denoised image, determining a segmentation threshold between the pad area and the background based on a maximum inter-class variance algorithm, and generating a binary mask image; Performing connected domain analysis on the binary mask image to extract boundary pixel coordinates of the pad area; Dynamically adjusting the sliding window size according to the distribution density of the boundary pixel coordinates; Median filtering and binarization are re-performed according to the adjusted sliding window size to generate a pre-processed image that highlights the pad edge.

[0006] Optionally, in a second implementation of the first aspect of the present application, the step of performing ellipse fitting on the preprocessed image to determine the geometric features of the pad includes: Screening candidate edge points in a corresponding area of ​​the pad edge; Performing local surface modeling on the grayscale values ​​of the neighborhood pixels of the candidate edge points to generate a sub-pixel edge coordinate sequence; Generating initial pad ellipse equation parameters by performing outlier detection on the sub-pixel edge coordinate sequence; The geometric features of the pad are determined according to the parameters of the initial pad ellipse equation.

[0007] Optionally, in a third implementation of the first aspect of the present application, the step of performing cross-modal feature fusion of the electrical signal feature vector corresponding to the electrical signal and the geometric feature to construct a multi-dimensional defect description matrix includes: Extract the features of the electric signal to generate an electric signal feature vector Performing tensor splicing on the electrical signal feature vector and the geometric feature to generate an initial fusion matrix; Performing covariance analysis on the column vectors of the initial fusion matrix to dynamically assign weight coefficients of the electrical signal and the geometric features; Based on the weight coefficients, a dual-channel attention mechanism is used to perform feature interaction on the electrical signal channel and the geometric feature channel in the initial fusion matrix to generate a high-dimensional defect description tensor; Feature selection is performed on the high-dimensional defect description tensor to generate a multi-dimensional defect description matrix.

[0008] Optionally, in a fourth implementation of the first aspect of the present application, the step of inputting the multi-dimensional defect description matrix into a pre-trained neural network model and outputting a defect type classification label and a corresponding confidence score includes: Performing tensor decomposition on the multi-dimensional defect description matrix to generate a three-dimensional feature tensor with spatiotemporal correlation; Performing parallel feature extraction on the three-dimensional feature tensor through the multi-scale convolution kernel group of the neural network model to generate a high-frequency defect feature map and a low-frequency topology feature map; Performing a residual connection on the high-frequency defect feature map and the low-frequency topology feature map to construct a cross-scale fusion feature channel; Perform feature interaction and weight allocation on the cross-scale fusion feature channels based on a self-attention mechanism to generate a defect semantic encoding vector; The defect semantic encoding vector is input into the fully connected classification layer, and the defect type classification label and the corresponding confidence score are output.

[0009] Optionally, in a fifth implementation of the first aspect of the present application, the step of generating a dynamic test path planning solution through an optimization algorithm based on the defect type classification label and the confidence score includes: Constructing a defect test priority weight matrix based on the defect type classification label and the confidence score; Generate a pad space topology adjacency graph according to the pad center position, and map the priority weight matrix to the edge weights of the pad space topology adjacency graph; Generating an initial test path sequence by performing ant colony optimization on the pad space topology adjacency graph; During the execution of the initial test path, a local area of ​​the priority weight matrix is ​​updated through a sliding window mechanism to trigger a dynamic modification of the edge weights; The path sequence is locally reoptimized according to the corrected edge weights to generate the final dynamic test path planning solution.

[0010] Optionally, in a sixth implementation of the first aspect of the present application, the method further includes: Performing difference analysis on the defect type classification labels and the manually verified labels to generate a set of misjudged defect samples and a set of missed defect samples; Based on the confidence score distribution of the misjudged defect sample set and the missed defect sample set, weighting the corresponding samples in the multi-dimensional defect description matrix by feature dimensions to generate an incremental training data set; The incremental training data set is input into the neural network model to generate updated model parameters after online fine-tuning, and the updated model parameters are synchronized to the neural network model for updating.

[0011] A second aspect of the present application provides a testing device for surface mount production, the testing device for surface mount production comprising: An acquisition module is used to collect image information and electrical signals after the printed circuit board is surface mounted; a processing module, configured to generate a pre-processed image of a protruding pad edge by performing median filtering and binarization on image data corresponding to the image information; A determination module, configured to perform ellipse fitting on the pre-processed image to determine geometric features of the pad; A construction module is used to perform cross-modal feature fusion on the electrical signal feature vector corresponding to the electrical signal and the geometric feature to construct a multi-dimensional defect description matrix; An output module, configured to input the multi-dimensional defect description matrix into a pre-trained neural network model and output defect type classification labels and corresponding confidence scores; A generation module is used to generate a dynamic test path planning solution through an optimization algorithm according to the defect type classification label and the confidence score.

[0012] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the processor is used to execute a computer program stored on the memory. When the processor executes the computer program, it implements each step of the test method for surface mount production provided in the first aspect of the embodiment of the present application.

[0013] The fourth aspect of the embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the test method for surface mount production provided in the first aspect of the embodiment of the present application are implemented.

[0014] In summary, according to the test method, device, equipment and storage medium for surface mount production provided by the present application, the image information and electrical signals of the printed circuit board after surface mounting are collected; the image data corresponding to the image information is subjected to median filtering and binarization processing to generate a preprocessed image of the edge of the protruding pad; the geometric features of the pad are determined by ellipse fitting on the preprocessed image; the electrical signal feature vector corresponding to the electrical signal is cross-modally fused with the geometric features to construct a multi-dimensional defect description matrix; the multi-dimensional defect description matrix is ​​input into a pre-trained neural network model to output a defect type classification label and a corresponding confidence score; a dynamic test path planning scheme is generated through an optimization algorithm based on the defect type classification label and the confidence score. By collecting the image information and electrical signals of the circuit board after mounting, extracting the geometric features of the pad, constructing a defect description matrix that fuses the electrical signal and geometric information, and inputting it into the pre-trained neural network for defect recognition, deep collaboration and information complementarity between modalities are achieved, effectively improving the ability to recognize complex defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flow chart of a test method for surface mount production provided in an embodiment of the present application; Figure 2 A schematic diagram of a program module of a test device for surface mount production provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0017] In order to solve the problem of insufficient information dimension and local judgment failure of single modal data in related technologies, the embodiment of the present application provides a test method for surface mount production, such as Figure 1 The present invention provides a flow chart of a test method for surface mount production, which includes the following steps: Step 110: Collect image information and electrical signals of the printed circuit board after surface mounting.

[0018] Specifically, in surface mount production, industrial cameras and electrical performance testing modules are used to synchronously capture image information and electrical signal data after the printed circuit board is assembled. The image information includes high-definition images of the pads, solder joints, and surrounding areas, while the electrical signals include multi-dimensional test data such as voltage, resistance, and capacitance response. These images and signals are collected simultaneously and accompanied by timestamps to ensure accurate correspondence between the two. This provides a unified data foundation for subsequent analysis and constitutes the comprehensive performance of the circuit board during the test cycle.

[0019] Step 120 : Generate a pre-processed image of the edge of the prominent pad by performing median filtering and binarization processing on the image data corresponding to the image information.

[0020] Specifically, the collected image information is preprocessed to enhance the analyzability of the image. The process first denoises the image through median filtering, uses a sliding window to traverse the image pixel matrix, and replaces the median of the pixel grayscale values ​​in each local area, thereby effectively removing the noise caused by the shooting environment. Then, based on the distribution characteristics of the grayscale histogram, the maximum inter-class variance method is used to binarize the image to set the optimal threshold to segment the foreground and background, so that the edge of the pad area and the surrounding area have obvious contrast differences in the image. This processing process can ensure that the boundary extraction has higher positioning accuracy and stability.

[0021] In an optional implementation of the present embodiment, the step of generating a preprocessed image of the protruding pad edge by performing median filtering and binarization on the image data corresponding to the image information includes: inputting the image data corresponding to the image information into the median filter, traversing the image pixel matrix through a preset sliding window, determining the median of the pixel grayscale values ​​in each window, and generating a denoised image; performing grayscale histogram statistics on the denoised image, determining the segmentation threshold of the pad area and the background based on the maximum between-class variance algorithm, and generating a binary mask image; performing connected domain analysis on the binary mask image to extract the boundary pixel coordinates of the pad area; dynamically adjusting the sliding window size according to the distribution density of the boundary pixel coordinates; and re-performing median filtering and binarization according to the adjusted sliding window size to generate a preprocessed image of the protruding pad edge.

[0022] Specifically, to enhance the edge features of the pad area in the post-assembly printed circuit board image, the original image data must first be subjected to noise suppression and structure enhancement. The image data corresponding to the image information is input into a median filter. A fixed-size sliding window is set to traverse the entire image pixel matrix pixel by pixel. During each sliding process, the grayscale value set of all pixels within the window's coverage area is extracted and the median of this set is calculated. The median value is the new grayscale value of the center pixel in the region, thereby constructing a new image with more stable pixel values ​​and less interference from local extreme values. Median filtering is a nonlinear smoothing filter method that is more edge-preserving than mean filtering and can effectively remove noise. For example, when bright or dark spots are present near the edges of certain pads, median filtering can effectively eliminate isolated pixels that do not belong to the solder joint structure. After denoising, the generated image is statistically analyzed using a grayscale histogram. By counting the frequency of occurrence of the grayscale levels of all pixels in the entire image, a grayscale distribution image is constructed. Based on this, the maximum inter-class variance algorithm is used to determine the optimal threshold for distinguishing foreground from background. The maximum inter-class variance algorithm automatically selects a threshold for image binary segmentation based on the principle of maximizing inter-class variance. Its goal is to find, among all possible grayscale thresholds, a grayscale value that maximizes the variance between the foreground (pad area) and the background (printed circuit board surface), thereby achieving optimal segmentation. For example, if the grayscale of the pad area in an image is concentrated in a high-grayscale region while the PCB background is dark, the algorithm automatically selects the grayscale cutoff point that best distinguishes the two. Based on this threshold, the image is binarized, with all pixels with grayscale values ​​above the threshold set to foreground (white) and the rest set to background (black). This ultimately generates a binary mask image containing the key structure of the pad. Next, to further extract the geometric information of the pad edges, a connected domain analysis is performed on this mask image. This scan involves scanning the image for adjacent, connected pixels with the same pixel value. All connected white regions are labeled, and regions whose characteristics, such as area, aspect ratio, and position, match the pad's geometric characteristics are identified. This process uses 4-connectivity or 8-connectivity criteria to determine whether pixels belong to the same connected domain. For example, in an image of a standard 0805 resistor pad, there should be regular, symmetrical rectangular regions on its left and right sides. Connected domain analysis can accurately mark the outer boundaries of these two regions. After the connected domain is extracted, the size of the sliding window is dynamically adjusted based on the coordinate density and distribution range of the extracted boundary pixels. If the boundaries are densely distributed and vary dramatically, a smaller window size can be used to more precisely preserve the boundary changes. Conversely, if the boundary distribution is stable and the structure is clear, a slightly larger window size can be used to suppress residual noise and improve processing efficiency. For example, when the edge of the pad is concave or slightly defective, a small window is more conducive to preserving its true shape; on the other hand, in areas with large, pure backgrounds, a large window size is suitable for improving image stability.Finally, median filtering and binarization processing are re-performed according to the adjusted sliding window size, so that the filtering and segmentation process are more in line with the actual morphology of the pad, thereby generating a pre-processed image with clear boundaries and complete structure.

[0023] Step 130: Perform ellipse fitting on the pre-processed image to determine the geometric features of the pad.

[0024] Specifically, within the resulting binary image, boundary pixels are extracted and spatially distributed within the pad edge region. A set of edge pixels meeting the shape criteria is selected and fitted to their spatial coordinates using a standard elliptical curve model. This method determines the pad's center position, major axis, minor axis, tilt angle, and other geometric features. This elliptical curve fitting method exhibits strong geometric robustness and is suitable for cases where the pad edge is slightly defective or has irregular extensions. The fitting parameters are calculated using the least squares method, and a spatial feature vector is constructed for subsequent analysis.

[0025] In an optional implementation of this embodiment, the step of performing ellipse fitting on the preprocessed image to determine the geometric features of the pad includes: screening candidate edge points in the corresponding area of ​​the pad edge; performing local surface modeling on the grayscale values ​​of the neighborhood pixels of the candidate edge points to generate a sub-pixel edge coordinate sequence; generating initial pad ellipse equation parameters by performing outlier detection on the sub-pixel edge coordinate sequence; and determining the geometric features of the pad based on the initial pad ellipse equation parameters.

[0026] Specifically, within the region corresponding to the pad edge, the approximate position of the pad outline is first located using a binary mask and connected domain analysis results. Pixel-level edge points are then extracted from the binary edge image within this region, thereby screening out a set of candidate edge pixels that may constitute the pad outline. Next, for each pixel of the candidate edge point, a small neighborhood window is captured centered on that point, and the grayscale values ​​in the window are treated as discrete samples from a continuous surface. A quadratic surface model is fitted using the least squares method, and the sub-pixel edge coordinates corresponding to the window center are calculated, forming a coordinate sequence accurate to two decimal places. This sequence reflects the continuous changes in the pad's true outline across the pixel level. For example, a smoother outline shape can be obtained on a 0402 resistor pad that is only a few pixels wide. A distance- and angle-based outlier detection algorithm is then used to remove abnormal points in the sub-pixel coordinate sequence that deviate from the overall trend. After removal, the remaining coordinates better represent the pad's main outline, stabilizing subsequent geometric fitting. Finally, a least-squares ellipse fitting is performed on the cleaned sub-pixel coordinate set to solve geometric features such as the pad center coordinates, major and minor axis lengths, and tilt angle. This feature can be directly used for positioning, absorption, and centering compensation. For example, it can accurately obtain the center position in a pad array with slight deformation or wear to guide subsequent component placement and alignment.

[0027] Step 140: Perform cross-modal feature fusion on the electrical signal feature vector corresponding to the electrical signal and the geometric feature to construct a multi-dimensional defect description matrix.

[0028] Specifically, after the spatial features are constructed, the aforementioned pad geometry features are fused with the feature vectors generated by electrical signal testing. A unified data representation structure is established through feature hierarchy unification and tensor concatenation. To address the scale differences and information representation structures between the two modalities, a covariance analysis mechanism is introduced to adjust the weights of the fused tensor channels. Furthermore, a dual-channel attention mechanism is introduced to enable deep semantic interaction between electrical signal features and geometric morphology information, thereby forming a multi-dimensional defect description matrix that comprehensively describes the state of a single solder joint.

[0029] In an optional implementation of this embodiment, the electrical signal feature vector corresponding to the electrical signal is cross-modally fused with the geometric feature to construct a multi-dimensional defect description matrix, including: extracting features from the electrical signal to generate an electrical signal feature vector, performing tensor splicing of the electrical signal feature vector and the geometric feature to generate an initial fusion matrix; performing covariance analysis on the column vectors of the initial fusion matrix to dynamically assign weight coefficients of the electrical signal and the geometric feature; based on the weight coefficient, using a dual-channel attention mechanism to perform feature interaction between the electrical signal channel and the geometric feature channel in the initial fusion matrix to generate a high-dimensional defect description tensor; and performing feature selection on the high-dimensional defect description tensor to generate a multi-dimensional defect description matrix.

[0030] Specifically, the processing of electrical signals should extract a variety of defect-characterizing information from the original test waveform or pulse response. For example, by calculating parameters such as the peak-to-peak value of the open-circuit voltage curve, the slope of the impedance change with frequency, and the half-maximum width of the time domain response, the parameters are then combined into a one-dimensional feature vector in a predefined order. This vector can intuitively reflect the connectivity of the electrical path on the solder joint and the abnormal electrical performance. Subsequently, the obtained electrical signal feature vector is fused with the previously obtained pad geometric features (including center coordinates, major axis, minor axis, and tilt angle) in a tensor splicing manner to generate an initial fusion matrix with two channels, where the first channel aggregates electrical features and the second channel carries spatial geometric information, thereby containing time domain and spatial bimodal data in parallel in the same data structure. Next, a covariance analysis is performed on each column vector of the fusion matrix. By calculating the covariance matrix of the electrical channel and the geometric channel on the sample set, the strength of the correlation between the two features is identified. Weight coefficients are dynamically assigned based on the feature correlation to ensure that electrical and geometric features with the same sequence number can proportionally highlight their shared or complementary information when they are subsequently complemented. For example, when detecting bridging defects, resistance mutations and elliptical major axis deviations with high covariance should be given higher weights. After the weight coefficients are determined, a dual-channel attention mechanism is introduced, that is, self-attention submodules are constructed for the electrical channel and the geometric channel respectively. Each channel automatically learns the key information of the other channel based on its own characteristics, and the input matrix is ​​fused and enhanced based on the weight coefficients. The output is a three-dimensional high-dimensional defect description tensor, where the newly added dimensions can correspond to different attention heads or feature subspaces to fully capture cross-modal interaction features. Finally, a feature selection operation is performed on the high-dimensional tensor, usually using a variance threshold or a sparse constraint method based on a regularization model to automatically screen out the dimensions that contribute most to defect type discrimination, thereby refining it into a multi-dimensional defect description matrix. This matrix not only maintains the key complementarity between electrical signals and geometric features, but also eliminates redundant information, providing a compact and semantically rich feature representation for subsequent classification network input.

[0031] Step 150: Input the multi-dimensional defect description matrix into the pre-trained neural network model, and output the defect type classification label and the corresponding confidence score.

[0032] Specifically, the defect description matrix is ​​input into a pre-trained neural network model for classification and recognition. This neural network, consisting of a multi-scale convolutional structure and an attention mechanism module, first performs tensor decomposition on the defect description matrix to extract its implicit patterns in space, morphology, and electrical signals. It then uses kernel groups with different receptive fields in the convolutional network to extract features in parallel, generating two types of feature maps: high-frequency structure and low-frequency trend. Residual connections and a self-attention mechanism are then used to perform cross-channel fusion, resulting in a semantic encoding vector for classification. Finally, a fully connected layer outputs the classification label and confidence score of the defect type, providing a basis for test judgment.

[0033] In an optional implementation of this embodiment, the multi-dimensional defect description matrix is ​​input into a pre-trained neural network model, and the step of outputting a defect type classification label and a corresponding confidence score includes: performing tensor decomposition on the multi-dimensional defect description matrix to generate a three-dimensional feature tensor with spatiotemporal correlation; performing parallel feature extraction on the three-dimensional feature tensor through a multi-scale convolution kernel group of the neural network model to generate a high-frequency defect feature map and a low-frequency topological feature map; performing residual connection on the high-frequency defect feature map and the low-frequency topological feature map to construct a cross-scale fusion feature channel; performing feature interaction and weight allocation on the cross-scale fusion feature channel based on the self-attention mechanism to generate a defect semantic encoding vector; inputting the defect semantic encoding vector into a fully connected classification layer to output a defect type classification label and a corresponding confidence score.

[0034] Specifically, after completing multimodal fusion, tensor decomposition of the resulting description matrix can expand it from a two-dimensional plane to a spatiotemporal perspective, thereby revealing the intrinsic coupling between electrical signals and geometric features in both time series and spatial layout. First, a tensor decomposition method is used to decompose the original matrix into multiple factor tensors, representing the distribution patterns of different modalities in the channel, spatial, and sample dimensions. This operation ensures that the joint feature not only contains individual channel information but also preserves cross-channel and sequence interactions. For example, when detecting the phenomenon of gradually increasing solder joint resistance and geometric eccentricity due to thermal cycling, its evolution can be intuitively reflected by the decomposed time series factors. The reconstructed three-dimensional feature tensor is then fed into a neural network with a multi-scale convolutional kernel group for parallel processing. Small convolution kernels (e.g., 3×3) are used to focus on capturing local high-frequency anomalies such as solder cracks or bridging, while large convolution kernels (e.g., 7×7) are used to extract low-frequency topological features that indicate overall pad deformation or solder joint defects. High-frequency defect feature maps and low-frequency topological feature maps are generated, respectively, to achieve a balanced perception of both microscopic details and macroscopic morphology. To integrate information from two different scales, the high-frequency and low-frequency maps are directly added together in the channel dimension to form a residual fusion feature channel. This residual connection not only preserves high-frequency details and low-frequency structure but also helps mitigate gradient vanishing during deep network training, enabling the model to establish a stable association between small cracks in pads caused by mechanical vibration and overall shape shifts due to thermal stress. Next, a self-attention mechanism is implemented on this fused feature channel. The fused features are first used to generate a query, key, and value matrix to calculate similarity attention weights between different spatial locations and channels. Important feature locations and channel responses are then amplified based on the weighted distribution. For example, in BGA pad array inspection, self-attention can highlight the characteristic responses of a few damaged solder balls and suppress interference from healthy solder joints. Ultimately, the vector is aggregated into a defect semantic encoding vector, which performs macroscopic semantic compression of the fused information in the channel dimension. Finally, this semantic encoding vector is input into a fully connected classification layer, which, through linear mapping and softmax activation, outputs labels for predefined defect categories and corresponding confidence scores. For example, in multi-class defect detection scenarios, it can distinguish between bridging, cold solder joints, and shifts, providing a quantitative basis for subsequent test ranking or rework decisions. Throughout the entire process, tensor decomposition, parallel multi-scale convolution, residual fusion, self-attention, and fully connected classification form a closed-loop architecture from spatiotemporal coupling to semantic discrimination, enabling complex defect patterns to be accurately captured and classified.

[0035] Step 160: Generate a dynamic test path planning solution through an optimization algorithm based on the defect type classification label and the confidence score.

[0036] Specifically, based on the defect type labels and confidence scores output by the model, a priority ranking system for pad inspection is established. Combined with the spatial distribution of pads, a topological adjacency graph based on pad centers is generated, in which defect confidence scores are embedded as edge weights. Optimization algorithms, such as ant colony optimization, are then used to search for the optimal inspection path sequence. The system dynamically schedules different testing methods based on priority (such as whether to combine electrical testing with image review) and utilizes a sliding window to locally update test weights during testing, enabling dynamic adjustment of path planning schemes. This improves the flexibility and coverage of the entire post-assembly inspection process.

[0037] In an optional implementation of this embodiment, the steps of generating a dynamic test path planning scheme through an optimization algorithm based on defect type classification labels and confidence scores include: constructing a defect test priority weight matrix based on the defect type classification labels and confidence scores; generating a pad space topology adjacency graph based on the pad center position, and mapping the priority weight matrix to the edge weights of the pad space topology adjacency graph; generating an initial test path sequence by performing ant colony optimization on the pad space topology adjacency graph; during the execution of the initial test path, updating the local area of ​​the priority weight matrix through a sliding window mechanism to trigger dynamic correction of the edge weights; and locally re-optimizing the path sequence based on the corrected edge weights to generate a final dynamic test path planning scheme.

[0038] Specifically, after defect identification is completed, a defect testing priority weight matrix is ​​first constructed based on each pad's classification label and its corresponding confidence score. This matrix has pad indices as rows and columns, and the matrix elements represent the detection priority values ​​of the corresponding pads. The priority value can be defined as the product of the classification confidence and the category criticality. "Bridged" pads with high confidence receive the highest weight. For example, a bridge state with a confidence of 0.9 is assigned a value of 0.9 × the criticality coefficient of 1.0. Next, by reading the pad center coordinates extracted in the previous stage, a pad spatial topological adjacency graph is generated on a two-dimensional plane. This graph consists of nodes and edges. Nodes correspond to pad centers, and edges connect pairs of nodes whose spatial distance is less than a set threshold, thus reflecting the spatial topological relationship between adjacent pads in the physical layout. Subsequently, the row vectors of the priority weight matrix are mapped to the edge weights of the topological graph. This can be assigned by taking the arithmetic average or weighted average of the weights of the nodes at both ends of each edge, so that the edge weights between high-risk pads are lower (so that the ant colony prioritizes high-risk areas when searching for optimal solutions). An ant colony optimization algorithm (ACO) is applied to this weighted graph. This algorithm simulates ant pheromone release and iteratively updates the path probability distribution based on a heuristic function (which can be set to the inverse of the edge weight) to generate an initial test path sequence that covers all pads and prioritizes high-risk nodes. For example, in a compact package array, the ant colony can automatically find the shortest path to connect multiple bridged pads. During test execution, a sliding window mechanism monitors the inspection results of the current execution segment and updates local regions of the priority weight matrix in real time. For example, if a pad experiences a sudden change in confidence during inspection, the node weights within the window are reweighted, triggering a dynamic correction of the adjacent edge weights. Finally, the original path sequence is locally reoptimized on the corrected edge weighted graph, using local search or simplified ant colony iteration to generate the final dynamic test path plan. This allows the inspection sequence to adaptively respond to emerging high-risk information at runtime, significantly reducing test time while ensuring complete coverage and focusing resources on the pad areas of greatest concern.

[0039] In an optional implementation of this embodiment, a difference analysis is performed on the defect type classification labels and the manually verified labels to generate a set of misjudged defect samples and a set of missed defect samples; based on the confidence score distribution of the set of misjudged defect samples and the set of missed defect samples, the corresponding samples in the multi-dimensional defect description matrix are weighted by feature dimensions to generate an incremental training data set; the incremental training data set is input into the neural network model to generate updated model parameters after online fine-tuning, and the updated model parameters are synchronized to the neural network model for updating.

[0040] Specifically, in order to achieve continuous updating of the model and build adaptive capabilities, it is first necessary to perform a difference analysis between the defect type classification labels output by the model and the manually verified labels. Specifically, by comparing the predicted category of each test sample with the manual labeling result, it is determined whether there is classification bias. Samples with incorrect predictions but high confidence are classified into the misjudged defect sample set, while actual defect samples that the model fails to detect are classified into the missed defect sample set. For example, for a solder joint that actually has a bridging defect but the model classifies it as normal with low confidence, this sample is identified as a missed sample. By statistically analyzing the confidence score distribution of misjudged and missed samples, the fuzzy area of ​​the model's current classification boundary can be analyzed, and the feature dimensions of the corresponding samples in the multi-dimensional defect description matrix can be weighted and adjusted accordingly. The weighting strategy can assign weights based on the degree of influence of the features on the classification bias, that is, higher gradient-sensitive weights are given to channels that appear more frequently in the misclassified area, so that they have stronger parameter update capabilities during training, thereby generating an incremental training dataset with discriminant enhanced labels. This incremental training dataset is then fed into the original neural network structure, where it undergoes a limited number of rounds of online fine-tuning while maintaining the network topology unchanged. Weight parameters are updated using optimization algorithms such as mini-batch gradient descent or adaptive learning rate scheduling, ensuring the model's rapid response to recent changes in sample distribution. For example, when process adjustments cause changes in the appearance of a particular type of weld defect, the response mode of the feature extraction channel can be promptly corrected. After fine-tuning is complete, the updated model parameters are synchronized with the main model for direct application in the next batch of tests. This creates an online learning closed-loop system driven by actual errors, ensuring the long-term availability and structural self-evolution of the detection model in a dynamic production environment.

[0041] According to a test method for surface mount production provided by the present application, image information and electrical signals of the printed circuit board after surface mounting are collected; a pre-processed image of the prominent pad edge is generated by median filtering and binarization of the image data corresponding to the image information; an ellipse fitting is performed on the pre-processed image to determine the geometric features of the pad; the electrical signal feature vector corresponding to the electrical signal is cross-modally fused with the geometric features to construct a multi-dimensional defect description matrix; the multi-dimensional defect description matrix is ​​input into a pre-trained neural network model to output defect type classification labels and corresponding confidence scores; a dynamic test path planning scheme is generated through an optimization algorithm based on the defect type classification labels and confidence scores. By collecting the image information and electrical signals of the circuit board after mounting, extracting the geometric features of the pad, constructing a defect description matrix that fuses the electrical signal and geometric information, and inputting it into the pre-trained neural network for defect recognition, deep collaboration and information complementarity between modalities are achieved, effectively improving the ability to recognize complex defects.

[0042] Figure 2The present invention provides a test device for surface mount production, which can be used to implement the test method for surface mount production in the above embodiment. Figure 2 As shown in the figure, the test device for surface mount production mainly includes: The acquisition module 10 is used to collect image information and electrical signals after the printed circuit board is surface mounted; The processing module 20 is used to generate a pre-processed image of the edge of the protruding pad by performing median filtering and binarization processing on the image data corresponding to the image information; A determination module 30 is used to perform ellipse fitting on the pre-processed image to determine the geometric features of the pad; A construction module 40 is used to perform cross-modal feature fusion on the electrical signal feature vector corresponding to the electrical signal and the geometric feature to construct a multi-dimensional defect description matrix; Output module 50, used to input the multi-dimensional defect description matrix into the pre-trained neural network model and output the defect type classification label and the corresponding confidence score; The generation module 60 is used to generate a dynamic test path planning solution through an optimization algorithm according to the defect type classification label and the confidence score.

[0043] In an optional implementation of this embodiment, the processing module is specifically used to: input image data corresponding to the image information into a median filter, traverse the image pixel matrix through a preset sliding window, determine the median of the pixel grayscale values ​​in each window, and generate a denoised image; perform grayscale histogram statistics on the denoised image, determine the segmentation threshold of the pad area and the background based on the maximum inter-class variance algorithm, and generate a binary mask image; perform connected domain analysis on the binary mask image to extract the boundary pixel coordinates of the pad area; dynamically adjust the sliding window size according to the distribution density of the boundary pixel coordinates; re-execute median filtering and binarization processing according to the adjusted sliding window size to generate a preprocessed image that highlights the pad edge.

[0044] In an optional implementation of this embodiment, the determination module is specifically used to: screen candidate edge points in the corresponding area of ​​the pad edge; perform local surface modeling on the grayscale values ​​of the neighborhood pixels of the candidate edge points to generate a sub-pixel edge coordinate sequence; generate initial pad ellipse equation parameters by performing outlier detection on the sub-pixel edge coordinate sequence; and determine the geometric features of the pad based on the initial pad ellipse equation parameters.

[0045] In an optional implementation of this embodiment, the construction module is specifically used to: perform feature extraction on the electrical signal to generate an electrical signal feature vector, perform tensor splicing of the electrical signal feature vector and the geometric feature to generate an initial fusion matrix; perform covariance analysis on the column vectors of the initial fusion matrix, and dynamically assign weight coefficients of the electrical signal and the geometric feature; based on the weight coefficient, use a dual-channel attention mechanism to perform feature interaction between the electrical signal channel and the geometric feature channel in the initial fusion matrix to generate a high-dimensional defect description tensor; perform feature selection on the high-dimensional defect description tensor to generate a multi-dimensional defect description matrix.

[0046] In an optional implementation of this embodiment, the output module is specifically used to: perform tensor decomposition on the multi-dimensional defect description matrix to generate a three-dimensional feature tensor with spatiotemporal correlation; perform parallel feature extraction on the three-dimensional feature tensor through the multi-scale convolution kernel group of the neural network model to generate a high-frequency defect feature map and a low-frequency topological feature map; perform residual connection on the high-frequency defect feature map and the low-frequency topological feature map to construct a cross-scale fusion feature channel; perform feature interaction and weight allocation on the cross-scale fusion feature channel based on the self-attention mechanism to generate a defect semantic encoding vector; input the defect semantic encoding vector into the fully connected classification layer, and output the defect type classification label and the corresponding confidence score.

[0047] In an optional implementation of this embodiment, the generation module is specifically used to: construct a defect test priority weight matrix based on defect type classification labels and confidence scores; generate a pad space topology adjacency graph based on the center position of the pad, and map the priority weight matrix to the edge weights of the pad space topology adjacency graph; generate an initial test path sequence by performing ant colony optimization on the pad space topology adjacency graph; during the execution of the initial test path, update the local area of ​​the priority weight matrix through a sliding window mechanism to trigger dynamic correction of the edge weights; perform local re-optimization on the path sequence according to the corrected edge weights to generate a final dynamic test path planning scheme.

[0048] In an optional implementation of this embodiment, the surface mount production test device further includes an update module. The update module is configured to: perform differential analysis between defect type classification labels and manually verified labels to generate a set of falsely identified defect samples and a set of missed defect samples; perform feature dimension weighting on corresponding samples in a multi-dimensional defect description matrix based on the confidence score distribution of the falsely identified defect samples and the missed defect samples to generate an incremental training dataset; input the incremental training dataset into a neural network model to generate updated model parameters after online fine-tuning; and synchronize the updated model parameters to the neural network model for updating.

[0049] According to the test device for surface mount production provided by the present application, the image information and electrical signals of the printed circuit board after surface mounting are collected; a pre-processed image of the edge of the protruding pad is generated by median filtering and binarization of the image data corresponding to the image information; the geometric features of the pad are determined by ellipse fitting of the pre-processed image; the electrical signal feature vector corresponding to the electrical signal is cross-modally fused with the geometric features to construct a multi-dimensional defect description matrix; the multi-dimensional defect description matrix is ​​input into a pre-trained neural network model to output defect type classification labels and corresponding confidence scores; a dynamic test path planning scheme is generated through an optimization algorithm based on the defect type classification labels and confidence scores. By collecting the image information and electrical signals of the circuit board after mounting, extracting the geometric features of the pad, constructing a defect description matrix that fuses the electrical signal and geometric information, and inputting it into the pre-trained neural network for defect recognition, deep collaboration and information complementarity between modalities are achieved, effectively improving the recognition capability of complex defects.

[0050] According to the application plan Figure 3 An electronic device provided in an embodiment of the present application can be used to implement the test method for surface mount production in the aforementioned embodiment, mainly comprising: Memory 301, processor 302, and computer program 303 stored in memory 301 and executable on processor 302. Memory 301 and processor 302 are connected via communication. When processor 302 executes computer program 303, the test method for surface mount device production described in the aforementioned embodiment is implemented. The number of processors can be one or more.

[0051] The memory 301 can be a high-speed random access memory (RAM) memory or a non-volatile memory such as a disk drive. The memory 301 is used to store executable program code. The processor 302 is coupled to the memory 301 .

[0052] Furthermore, the embodiment of the present application also provides a computer-readable storage medium, which can be provided in the electronic device in the above embodiments. The computer-readable storage medium can be the above Figure 3 Memory in the illustrated embodiment.

[0053] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the surface mount manufacturing test method described in the aforementioned embodiments. Furthermore, the computer-readable storage medium may be a USB flash drive, a removable hard drive, a read-only memory (ROM), RAM, a magnetic disk, or an optical disk, among other media capable of storing program code.

[0054] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0055] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0056] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A test method for surface mount production, characterized in that, include: Collect image information and electrical signals after printed circuit board surface mounting; Generate a pre-processed image of the edge of the protruding pad by performing median filtering and binarization processing on the image data corresponding to the image information; Performing ellipse fitting on the preprocessed image to determine geometric features of the pad; Performing cross-modal feature fusion on the electrical signal feature vector corresponding to the electrical signal and the geometric feature to construct a multi-dimensional defect description matrix; Input the multi-dimensional defect description matrix into a pre-trained neural network model, and output defect type classification labels and corresponding confidence scores; A dynamic test path planning scheme is generated through an optimization algorithm according to the defect type classification label and the confidence score.

2. The test method for surface mount production according to claim 1, characterized in that: The step of generating a pre-processed image of the edge of the protruding pad by performing median filtering and binarization processing on the image data corresponding to the image information includes: Input the image data corresponding to the image information into a median filter, traverse the image pixel matrix through a preset sliding window, determine the median of the pixel grayscale value in each window, and generate a denoised image; Performing grayscale histogram statistics on the denoised image, determining a segmentation threshold between the pad area and the background based on a maximum inter-class variance algorithm, and generating a binary mask image; Performing connected domain analysis on the binary mask image to extract boundary pixel coordinates of the pad area; Dynamically adjusting the sliding window size according to the distribution density of the boundary pixel coordinates; Median filtering and binarization are re-performed according to the adjusted sliding window size to generate a pre-processed image that highlights the pad edge.

3. The test method for surface mount production according to claim 2, characterized in that: The step of performing ellipse fitting on the pre-processed image to determine the geometric features of the pad comprises: Screening candidate edge points in a corresponding area of ​​the pad edge; Performing local surface modeling on the grayscale values ​​of the neighborhood pixels of the candidate edge points to generate a sub-pixel edge coordinate sequence; Generating initial pad ellipse equation parameters by performing outlier detection on the sub-pixel edge coordinate sequence; The geometric features of the pad are determined according to the parameters of the initial pad ellipse equation.

4. The test method for surface mount production according to claim 1, characterized in that: The step of performing cross-modal feature fusion of the electrical signal feature vector corresponding to the electrical signal and the geometric feature to construct a multi-dimensional defect description matrix includes: Extract the features of the electric signal to generate an electric signal feature vector Performing tensor splicing on the electrical signal feature vector and the geometric feature to generate an initial fusion matrix; Performing covariance analysis on the column vectors of the initial fusion matrix to dynamically assign weight coefficients of the electrical signal and the geometric features; Based on the weight coefficients, a dual-channel attention mechanism is used to perform feature interaction on the electrical signal channel and the geometric feature channel in the initial fusion matrix to generate a high-dimensional defect description tensor; Feature selection is performed on the high-dimensional defect description tensor to generate a multi-dimensional defect description matrix.

5. The test method for surface mount production according to claim 1, characterized in that: The step of inputting the multi-dimensional defect description matrix into a pre-trained neural network model and outputting a defect type classification label and a corresponding confidence score includes: Performing tensor decomposition on the multi-dimensional defect description matrix to generate a three-dimensional feature tensor with spatiotemporal correlation; Performing parallel feature extraction on the three-dimensional feature tensor through the multi-scale convolution kernel group of the neural network model to generate a high-frequency defect feature map and a low-frequency topology feature map; Performing a residual connection on the high-frequency defect feature map and the low-frequency topology feature map to construct a cross-scale fusion feature channel; Perform feature interaction and weight allocation on the cross-scale fusion feature channels based on a self-attention mechanism to generate a defect semantic encoding vector; The defect semantic encoding vector is input into the fully connected classification layer, and the defect type classification label and the corresponding confidence score are output.

6. The test method for surface mount production according to claim 5, characterized in that: The step of generating a dynamic test path planning solution through an optimization algorithm according to the defect type classification label and the confidence score includes: Constructing a defect test priority weight matrix based on the defect type classification label and the confidence score; Generate a pad space topology adjacency graph according to the pad center position, and map the priority weight matrix to the edge weights of the pad space topology adjacency graph; Generating an initial test path sequence by performing ant colony optimization on the pad space topology adjacency graph; During the execution of the initial test path, a local area of ​​the priority weight matrix is ​​updated through a sliding window mechanism to trigger a dynamic modification of the edge weights; The path sequence is locally reoptimized according to the corrected edge weights to generate the final dynamic test path planning solution.

7. The test method for surface mount production according to claim 5, characterized in that: The method further comprises: Performing difference analysis on the defect type classification labels and the manually verified labels to generate a set of misjudged defect samples and a set of missed defect samples; Based on the confidence score distribution of the misjudged defect sample set and the missed defect sample set, weighting the corresponding samples in the multi-dimensional defect description matrix by feature dimensions to generate an incremental training data set; The incremental training data set is input into the neural network model to generate updated model parameters after online fine-tuning, and the updated model parameters are synchronized to the neural network model for updating.

8. A test device for surface mount production, characterized in that: The test device for surface mount production includes: An acquisition module is used to collect image information and electrical signals after the printed circuit board is surface mounted; a processing module, configured to generate a pre-processed image of a protruding pad edge by performing median filtering and binarization on image data corresponding to the image information; A determination module, configured to perform ellipse fitting on the pre-processed image to determine geometric features of the pad; A construction module is used to perform cross-modal feature fusion on the electrical signal feature vector corresponding to the electrical signal and the geometric feature to construct a multi-dimensional defect description matrix; An output module, configured to input the multi-dimensional defect description matrix into a pre-trained neural network model and output defect type classification labels and corresponding confidence scores; A generation module is used to generate a dynamic test path planning solution through an optimization algorithm according to the defect type classification label and the confidence score.

9. An electronic device, characterized in that: Comprising a memory and a processor, wherein: The processor is configured to execute a computer program stored in the memory; When the processor executes the computer program, the steps of the test method for surface mount production according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the test method for surface mount production according to any one of claims 1 to 7 are implemented.

Images