Deep learning-based production detection anomaly recognition method and system, and storage medium

Abstract

The application provides a kind of production detection abnormality identification method, system and storage medium based on deep learning, it is first to obtain the continuous acquisition image sequence of electronic component production line online detection station, including the image of the welded component body and surrounding solder joint area, then the solder joint shape anomaly directional capture is executed to the welded component body image, and the solder joint shape distortion distribution map is generated, the wet spreading behavior quantification is executed to the image of the welded component around solder joint area, and the wet spreading contour abnormal fluctuation is generated, then the space coupling correlation model is constructed and the correlation mapping is output by fusing the solder joint shape distortion distribution map and the wet spreading contour abnormal fluctuation map, finally, the defect category attribution determination is executed to the space coupling defect correlation mapping by calling the welding defect type decision tree, and the component welding defect identification result is generated.The application comprehensively and accurately identifies welding defects, improves production detection efficiency and quality.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and more specifically, to a method, system, and storage medium for identifying production anomalies based on deep learning. Background Technology

[0002] The soldering process of electronic components involves many complex factors, such as soldering temperature, soldering time, and solder composition. Even slight deviations in these factors can lead to soldering defects, which in turn affect the performance and lifespan of the entire electronic product.

[0003] Currently, traditional methods for identifying anomalies in production inspection mainly rely on manual visual inspection or image processing techniques based on simple rules. Manual visual inspection is not only inefficient but also easily influenced by the subjective factors of the inspectors, making it difficult to guarantee the consistency and accuracy of the inspection results. While image processing techniques based on simple rules can achieve a certain degree of automated inspection, these rules are often based on fixed thresholds or simple feature extraction, lacking sufficient adaptability and flexibility for complex and varied welding defects. For example, when dealing with abnormal weld joint morphology and wetting spread behavior, traditional methods struggle to accurately capture subtle differences in weld joint position offsets, size variations, and wetting spread contours, and are unable to deeply analyze the intrinsic relationships between these anomalies, thus failing to comprehensively and accurately identify welding defects. Summary of the Invention

[0004] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a deep learning-based method for identifying production anomalies, the method comprising: The continuous acquisition image sequence output from the online inspection station of the electronic component production line is obtained. The continuous acquisition image sequence includes images of the soldered component body and images of the solder joint area around the soldered component. An abnormal solder joint shape is captured by performing a solder joint shape orientation capture operation on the image of the soldered component body, and a solder joint shape distortion distribution map of the soldered component body is generated. The solder joint shape distortion distribution map is marked with the solder joint position offset trajectory and the solder joint size change gradient. A wetting and spreading behavior quantification operation is performed on the image of the solder joint area around the soldered components to generate an abnormal fluctuation map of the wetting and spreading contour. The abnormal fluctuation map of the wetting and spreading contour includes a marker of the location where the spreading contour diffusion is blocked and a marker of the direction of asymmetric extension of the spreading contour. By integrating the solder joint morphology distortion distribution map and the wetting spread profile abnormal fluctuation map, a spatial coupling correlation model of component welding defects is constructed, and a spatial coupling defect correlation mapping map is output. The spatial coupling defect correlation mapping map records the coordinate set of the spatially overlapping area between solder joint morphology distortion and wetting spread profile abnormality. The welding defect type decision tree is invoked to perform defect category determination on the spatially coupled defect association mapping, generating component welding defect identification results containing defect category labels and defect region boundary coordinates.

[0005] Furthermore, embodiments of the present invention also provide a deep learning-based production detection anomaly identification system, comprising: A processor; a machine-readable storage medium for storing machine-executable instructions of the processor; wherein the processor is configured to execute the aforementioned deep learning-based production detection anomaly identification method by executing the machine-executable instructions.

[0006] In another aspect, embodiments of the present invention also provide a machine-readable storage medium, characterized in that it is used to store machine-executable instructions of a processor; wherein the processor is configured to execute the above-described deep learning-based production detection anomaly identification method by executing the machine-executable instructions.

[0007] Based on the above, by acquiring a continuous sequence of images from the online inspection station of an electronic component production line, an anomaly detection operation is performed on the image of the soldered component body to generate a solder joint morphology distortion distribution map. This map can mark the solder joint position offset trajectory and the solder joint size change gradient. Furthermore, a wetting and spreading behavior quantification operation is performed on the image of the solder joint area surrounding the soldered component to generate a wetting and spreading contour anomaly fluctuation map. This map can mark the location of obstructed spreading contour diffusion and the direction of asymmetric extension of the spreading contour. Finally, the solder joint morphology distortion distribution map and the wetting and spreading contour anomaly fluctuation map are fused to construct a spatial coupling correlation model, outputting a spatial... The coupled defect association mapping map can record the coordinate set of the spatially overlapping area between solder joint morphology distortion and wetting spread contour anomaly. It reveals the intrinsic relationship between the two anomalies from a spatial dimension. Finally, the welding defect type decision tree is called to perform defect category determination on the spatial coupled defect association mapping map, generating component welding defect identification results containing defect category labels and defect area boundary coordinates. This achieves accurate identification and location of welding defects, and can comprehensively and accurately identify various anomalies in the welding process of electronic components, effectively improving the efficiency and accuracy of production inspection and ensuring the production quality of electronic components. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the execution flow of the deep learning-based production detection anomaly identification method provided in an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram of exemplary hardware and software components of a deep learning-based production detection anomaly identification system provided in an embodiment of the present invention. Detailed Implementation

[0010] Figure 1 This is a flowchart illustrating a deep learning-based production detection anomaly identification method according to an embodiment of the present invention, which will be described in detail below.

[0011] Step S110: Obtain a continuous sequence of images output from the online inspection station of the electronic component production line. The continuous sequence of images includes images of the soldered component body and images of the solder joint area around the soldered component.

[0012] In this embodiment, a scenario of soldering quality inspection of a printed circuit board (PCB) on a surface mount technology (SMT) production line is used as an example. The PCB has 16-pin SMT components soldered on it using a reflow soldering process. An online inspection station uses a high-resolution industrial camera to continuously image the soldered PCB at a rate of 15 frames per second. The camera lens optical axis is perpendicular to the PCB surface, and the illumination system uses a ring-shaped coaxial light source. The online inspection station outputs a continuously acquired image sequence, where each frame is a 16-bit grayscale image with a resolution of 24 megapixels. Each frame contains two spatially registered image blocks: an image of the soldered component itself, cropped to a 600*600 pixel rectangle based on the component's outline; and an image of the solder joint area surrounding the component, cropped to an 800*800 pixel rectangle based on the convex envelope formed by the 16 solder joints surrounding the component. The continuously acquired image sequence is stored in chronological order of acquisition time. Each acquisition time corresponds to a set of images of the welded component body and the area of ​​the solder joints around the welded component. The two sets of images are linked and indexed by timestamps.

[0013] Step S120: Perform solder joint morphology anomaly directional capture operation on the image of the soldered component body to generate a solder joint morphology distortion distribution map of the soldered component body. The solder joint morphology distortion distribution map marks the solder joint position offset trajectory and solder joint size change gradient.

[0014] Step S121: Segment the local image blocks where each solder joint is located from the image of the soldered component body to obtain a set of local image blocks of solder joints. Each local image block of solder joints in the set of local image blocks of solder joints contains the complete contour information of a single solder joint.

[0015] An anomaly detection operation for solder joint morphology is performed on the image of the soldered component body. Sixteen local image blocks for each solder joint are segmented from the component body image. Binarization is performed using a grayscale threshold of 128, and pixels with grayscale values ​​below 128 are marked as candidate solder joint regions. Morphological opening operations are performed on the candidate solder joint regions using circular structuring elements with a radius of 3 pixels, involving erosion followed by dilation to remove isolated noise and restore connectivity. This results in 16 connected components, each corresponding to a solder joint. The minimum bounding rectangle of each connected component is calculated, and the rectangle is extended outward by 10 pixels before cropping, resulting in local image blocks for solder joints ranging from 80*80 to 120*120 pixels. The 16 local image blocks are organized into a set of local image blocks for solder joints according to pin numbering, with each element containing a solder joint index and local image data.

[0016] Step S122: Perform solder joint contour edge tracking processing on each solder joint local image block in the set of solder joint local image blocks to obtain a solder joint contour edge coordinate sequence. The solder joint contour edge coordinate sequence is arranged in the order of edge point connection to form a closed contour curve.

[0017] For each local image block in the set of local image blocks of solder joints, solder joint contour edge tracking processing is performed. The Canny edge detection operator is used to extract edge pixels. The Canny operator has a low threshold of 50, a high threshold of 150, and a Gaussian filter standard deviation of 1.2, resulting in a binary edge image. In the binary edge image, an eight-neighbor connected component tracing algorithm is used to extract closed contour curves: starting from the first row and first column, pixel-by-pixel scanning is performed. An unvisited edge pixel with a value of 1 is used as the starting point. Adjacent pixels are checked in eight directions: right, upper right, upper, upper left, left, lower left, lower, and lower right. The first pixel with a value of 1 is taken as the next contour point and moved. Each time a contour point is tracked, its pixel coordinates are recorded in the solder joint contour edge coordinate sequence. When tracing back to the starting point, a closed contour curve is formed. If multiple connected contours exist, the above process is repeated. Finally, a two-dimensional coordinate point list is obtained, arranged in the order of edge point connections. The length of the list is equal to the perimeter of the contour pixels, and each coordinate point contains row and column coordinates.

[0018] Step S123: Calculate the coordinates of the geometric center point of the weld point contour edge coordinate sequence, and compare the coordinates of the geometric center point with the preset standard weld point position coordinates by spatial offset vector to obtain a set of weld point position offset vectors. The set of weld point position offset vectors includes the offset direction angle and the offset distance length.

[0019] Calculate the geometric center point coordinates for each solder joint's outline edge coordinate sequence. Assume the solder joint outline edge coordinate sequence contains M coordinate points. The row coordinates C_row of the geometric center point are equal to the sum of all row coordinates in the sequence divided by M, i.e., C_row = (1 / M) × (P_1_row + P_2_row + ... + P_M_row); the column coordinates C_col of the geometric center point are equal to the sum of all column coordinates in the sequence divided by M, i.e., C_col = (1 / M) × (P_1_col + P_2_col + ... + P_M_col). The geometric center point coordinates (C_row, C_col) are thus obtained. The standard solder joint position coordinates (S_row, S_col) corresponding to the solder joint index are read from the standard solder joint position coordinate mapping table. Calculate the offset vector row component D_row = C_row - S_row, and the column component D_col = C_col - S_col. The offset distance length L_offset = sqrt(D_row^2 + D_col^2). The offset direction angle θ_offset = arctan2(D_col, D_row). By determining the signs of D_row and D_col, θ_offset is converted to the range of 0 to 2π. The above calculation is performed on each of the 16 solder joints to obtain a set of solder joint position offset vectors containing 16 offset vectors. Each offset vector records the θ_offset_i and L_offset_i of the corresponding solder joint index.

[0020] Step S124: Extract the maximum inscribed circle radius and minimum circumscribed circle radius of the solder joint contour from each solder joint local image block in the set of solder joint local image blocks, calculate the radial difference between the maximum inscribed circle radius and the minimum circumscribed circle radius, and generate solder joint size deformation parameters.

[0021] Extract the maximum inscribed circle radius and minimum circumscribed circle radius of the solder joint contour from the local image block of each solder joint. The center of the maximum inscribed circle is located at the geometric center point (C_row, C_col), and the maximum inscribed circle radius R_in is equal to the minimum distance from all points on the contour to the geometric center point, i.e., R_in=min{sqrt((P_i_row-C_row)^2+(P_i_col-C_col)^2)}, where i ranges from 1 to M. The center of the minimum circumscribed circle is also located at the geometric center point, and the minimum circumscribed circle radius R_out is equal to the maximum distance from all points on the contour to the geometric center point, i.e., R_out=max{sqrt((P_i_row-C_row)^2+(P_i_col-C_col)^2)}, where i ranges from 1 to M. The solder joint size deformation parameter ΔR=R_out-R_in. Perform the above calculations on all 16 solder joints to generate 16 ΔR_i, each corresponding to the solder joint index i.

[0022] Based on the set of solder joint position offset vectors and the solder joint size deformation parameters, a trajectory line and gradient sequence are constructed and superimposed with the identified distortion candidate regions to generate a solder joint morphology distortion distribution map of the soldered component body. Specifically, this may include: Step S125: Connect the offset direction angles in the set of solder joint position offset vectors in the order of solder joint index to form a solder joint position offset trajectory line. The solder joint position offset trajectory line records the continuous change trend of the offset direction angle between adjacent solder joints.

[0023] The 16 offset direction angles in the solder joint position offset vector set are connected sequentially according to the solder joint index. The solder joint indexes are numbered 1 to 16 according to the row priority order of the printed circuit board solder joint layout. In a two-dimensional coordinate system, the coordinate point (i, θ_offset_i) is plotted with the solder joint index as the horizontal axis and θ_offset_i as the vertical axis. The coordinate points (i, θ_offset_i) and (i+1, θ_offset_i+1) corresponding to adjacent indices are connected by straight line segments, i=1 to 15, forming a polyline composed of 15 line segments connected end to end. This polyline is the solder joint position offset trajectory line, recording the continuous change trend of the offset direction angle between adjacent solder joints.

[0024] Step S126: Arrange the solder joint size deformation parameters into a size change sequence according to the solder joint index order, calculate the difference between adjacent elements in the size change sequence, and obtain the solder joint size change gradient sequence.

[0025] Arrange the 16 ΔR_i values ​​in solder joint index order to form a dimensional change sequence {ΔR_1, ΔR_2, ..., ΔR_16}. Calculate the difference between adjacent elements, G_i = ΔR_{i+1} - ΔR_i, where i = 1 to 15, resulting in 15 solder joint dimensional change gradient values ​​forming a solder joint dimensional change gradient sequence {G_1, G_2, ..., G_15}. Each gradient value reflects the rate of change of dimensional deformation between adjacent solder joints.

[0026] Step S127: Based on the change in the offset direction angle in the solder joint position offset trajectory line, identify the inflection point where the offset direction changes abruptly and the region where the offset direction is continuously abnormal, and mark the inflection point and the region where the offset direction is continuously abnormal as candidate positions for morphological distortion; based on the solder joint index corresponding to the gradient peak value in the solder joint size change gradient sequence that exceeds the preset change threshold, determine the solder joint index interval where the solder joint size changes drastically, and mark the solder joint region corresponding to the solder joint index interval as a candidate region for size distortion.

[0027] Identify abrupt inflection points in the offset direction based on the solder joint offset trajectory. Set Δθ_threshold = π / 6. For vertex (i, θ_offset_i), calculate the change in direction of the line segment before and after it. If |(θ_offset_{i+1}-θ_offset_i)-(θ_offset_i-θ_offset_{i-1})|>π / 6, then mark solder joint index i as the inflection point position. Identify regions with persistent abnormal offset direction: Set a normal range [-π / 6, π / 6]. When the θ_offset_i of three or more consecutive solder joints is not within the normal range, mark the region corresponding to the above consecutive solder joint indices as a region with persistent abnormal offset direction. Mark the inflection point position and the persistent abnormal region as candidate positions for morphological distortion.

[0028] Identify regions of drastic size changes based on the gradient sequence of solder joint size variations. Set a preset change threshold G_th = 0.05 * avg(ΔR_i). Scan {G_i}; when |G_i| > G_th, mark solder joint indices i and i+1 as solder joints with drastic size changes. Merge consecutively marked solder joint indices into intervals, such as [2, 4] indicating that solder joints 2, 3, and 4 are marked. Label the solder joint regions corresponding to the above intervals as candidate regions for size distortion.

[0029] Step S128: The candidate positions of morphological distortion and the candidate regions of dimensional distortion are spatially superimposed to generate a spatial overlap determination result of morphological distortion and dimensional distortion. Based on the spatial overlap determination result, a solder joint morphological distortion distribution map containing the solder joint position offset trajectory line, the peak annotation point of the solder joint size change gradient sequence, and the spatial overlap region of morphological distortion and dimensional distortion is drawn on the image of the soldered component body. The spatial overlap determination result records the boundary coordinates of the overlap region and the independent distribution coordinates of the non-overlapping region.

[0030] The candidate locations for morphological distortion and the candidate regions for dimensional distortion are spatially superimposed. For each solder joint index i, it is determined whether it appears simultaneously in both the candidate locations for morphological distortion and the candidate regions for dimensional distortion: if it appears simultaneously, the spatial region corresponding to the solder joint is marked as a spatially overlapping region of morphological and dimensional distortion, and the solder joint contour edge coordinate sequence is recorded as the boundary coordinates of the overlapping region; if it only appears in the candidate locations for morphological distortion, it is marked as an independent region of morphological distortion, and its independently distributed coordinates are recorded; if it only appears in the candidate regions for dimensional distortion, it is marked as an independent region of dimensional distortion, and its independently distributed coordinates are recorded. The above records constitute the spatial overlap determination result.

[0031] Draw a distribution map of solder joint morphology distortion on the image of the welded component body: Take the geometric center point of each solder joint as a node, and connect the geometric center points of adjacent solder joints in sequence according to the solder joint index order, and mark the corresponding θ_offset_i value on the connecting line segment; At the position where |G_i|>G_th, draw a red dot with a radius of 5 pixels at the midpoint of the connection line between the geometric center points of solder joints i and i+1; For the solder joints marked as the spatially coincident region, fill the region enclosed by the coordinate sequence of the solder joint contour edge with yellow with 50% transparency; Fill the independent region of morphology distortion with blue with 50% transparency; Fill the independent region of size distortion with green with 50% transparency to generate a complete distribution map of solder joint morphology distortion.

[0032] Step S130: Perform a quantification operation on the image of the solder joint area around the welded component to generate a map of abnormal fluctuations in the wetting and spreading profile. The map of abnormal fluctuations in the wetting and spreading profile includes marks of positions where the spreading profile is blocked and marks of the asymmetric extension direction of the spreading profile.

[0033] Step S131: Extract the pixel connected domain of the solder spreading area from the image of the solder joint area around the welded component to obtain a binary mask of the solder spreading connected domain. The pixel points with the first value in the binary mask of the solder spreading connected domain form the solder coverage area, and the pixel points with the second value form the non-solder coverage area.

[0034] Perform a quantification operation on the image of the solder joint area around the welded component. Calculate the grayscale histogram of the image, and find the grayscale value corresponding to the valley bottom between the two peaks as the segmentation threshold T_seg. Mark the pixels with grayscale value ≥ T_seg as solder candidate pixels, and mark the pixels with grayscale value < T_seg as non-solder pixels. Perform connected domain analysis on the solder candidate pixels using the four-neighborhood connectivity rule, calculate the pixel area of each connected domain, and retain the connected domain with an area > 500 pixels as the effective solder spreading connected domain. Generate a binary mask with the same size as the image of the solder joint area around the welded component. The pixels with a value of 1 form the solder coverage area, and the pixels with a value of 0 form the non-solder coverage area.

[0035] Step S132: Perform a boundary tracking algorithm on the solder coverage area in the binary mask of the solder spreading connected domain to generate a set of spreading profile boundary points. The set of spreading profile boundary points is arranged in the traversal order of the boundary points to form a closed spreading profile loop.

[0036] A Moore neighborhood boundary tracing algorithm is performed on the solder coverage area in the solder spread connected binary mask. The first pixel with a value of 1 is scanned from the top-left corner of the solder coverage area as the boundary starting point. Starting from the starting point, the eight neighbors of the current pixel are searched clockwise. The first neighboring pixel with a value of 1 is found as the next boundary point, and the current direction is recorded at this boundary point. The search continues to the next boundary point, each time starting from the direction perpendicular to the entry direction of the previous boundary point to avoid repeated tracing. A closed spread contour loop is formed when tracing back to the boundary starting point. The pixel coordinates of all boundary points during the tracing process are arranged in the tracing order to obtain a set of spread contour boundary points, where each boundary point contains row and column coordinates.

[0037] Step S133: Calculate the radial distance of each boundary point in the set of boundary points of the paving contour relative to the geometric centroid of the paving contour ring, and obtain a radial distance distribution sequence, which reflects the extension length of the paving contour in different angular directions.

[0038] Suppose that the set of boundary points of the unfolded contour contains K boundary points. The geometric centroid row coordinate G_center_row is equal to the sum of the row coordinates of all boundary points divided by K, i.e., G_center_row = (1 / K) × (B_1_row + B_2_row + ... + B_K_row); the geometric centroid column coordinate G_center_col is equal to the sum of the column coordinates of all boundary points divided by K, i.e., G_center_col = (1 / K) × (B_1_col + B_2_col + ... + B_K_col). The radial distances of all K boundary points are rearranged in ascending order of polar angle to obtain the radial distance distribution sequence {R_(1), R_(2), ..., R_(K)}, where R_(t) corresponds to the radial distance at the polar angle φ_(t).

[0039] Step S134: Identify local minimum points of radial distance from the radial distance distribution sequence, and mark the boundary point positions corresponding to the local minimum points of radial distance as positions where the spread profile diffusion is hindered. The positions where the spread profile diffusion is hindered indicate that the solder spread in the corresponding direction is obstructed and fails to extend sufficiently; and identify local maximum points of radial distance from the radial distance distribution sequence, and mark the boundary point positions corresponding to the local maximum points of radial distance as positions where the spread profile is abnormally convex. The positions where the spread profile is abnormally convex indicate that the solder has abnormally over-spread in that direction.

[0040] Identify local extreme points from the radial distance distribution sequence. The three - adjacent - point comparison method is adopted: for an interior point \(R_t\), if \(R_t < R_{t - 1}\) and \(R_t < R_{t+1}\), it is a local minimum point; if \(R_t > R_{t - 1}\) and \(R_t > R_{t+1}\), it is a local maximum point. Only the two end - points of the sequence are compared with one adjacent point respectively. Mark the boundary - point position corresponding to each local minimum point as the position where the spreading contour diffusion is blocked, and mark the boundary - point position corresponding to each local maximum point as the position where the spreading contour abnormally bulges.

[0041] Based on the positions where the spreading contour diffusion is blocked and the positions where the spreading contour abnormally bulges, calculate the fluctuation amplitude parameter and the asymmetric extension direction, and draw an image annotation to generate an abnormal fluctuation map of the wetting spreading contour. Specifically, it may include: Step S135: For each local maximum point of the radial distance, calculate the difference between its radial distance value and the radial distance values of the adjacent local minimum points of the radial distance, to obtain a spreading contour fluctuation amplitude parameter sequence, which records the degree of undulation of the spreading contour in different angular intervals; calculate the deviation of the direction angle of each local maximum point of the radial distance from the direction angle of the geometric center of the spreading contour ring, to obtain a set of asymmetric extension directions of the spreading contour.

[0042] For each local maximum point \(R_{max}\), search for the nearest local minimum points \(R_{min - left}\) and \(R_{min - right}\) in the radial distance distribution sequence to the left and right respectively, and calculate the fluctuation amplitude \(A_{fluctuation}=(R_{max}-R_{min - left}+R_{max}-R_{min - right}) / 2\). Obtain a spreading contour fluctuation amplitude parameter sequence \(\{A_1,A_2,\cdots,A_N\}\), where \(N\) is the total number of local maximum points.

[0043] Calculate the overall direction angle of the spreading contour ring. The row component of the direction vector \(V_{row}=\sum_{j = 1}^{K}R_j\cos(\varphi_j)\), the column component \(V_{col}=\sum_{j = 1}^{K}R_j\sin(\varphi_j)\), and the overall direction angle \(\varphi_{overall}=\arctan2(V_{col},V_{row})\). For each local maximum point, its direction - angle deviation \(\Delta\varphi_{dev}=\varphi_{max}-\varphi_{overall}\). Record the polar angle \(\varphi_{max}\) and the deviation \(\Delta\varphi_{dev}\) of the local maximum points with \(|\Delta\varphi_{dev}|>\pi / 6\) into the set of asymmetric extension directions of the spreading contour.

[0044] Step S136: Based on the angle range corresponding to the first preset number of peak fluctuations with the largest fluctuation amplitude in the spread profile fluctuation amplitude parameter sequence, determine the concentrated distribution angle range of abnormal fluctuations in the spread profile, and mark the spread profile arc segment corresponding to the concentrated distribution angle range as the violently fluctuating profile segment.

[0045] From the sequence of fluctuation amplitude parameters of the unfolded contour, identify the three peaks with the largest fluctuation amplitudes, A_max1, A_max2, and A_max3, and record the corresponding polar angles φ_max1, φ_max2, and φ_max3. Set the angle window width π / 12 centered on each polar angle to obtain three angle intervals: [φ_max1-π / 24, φ_max1+π / 24], [φ_max2-π / 24, φ_max2+π / 24], and [φ_max3-π / 24, φ_max3+π / 24]. Take the union of these intervals as the concentrated distribution angle range of abnormal fluctuations in the unfolded contour. Find all boundary points of the polar angles within this range on the unfolded contour ring, and label the contour arc segment connecting these boundary points as the severely fluctuating contour segment.

[0046] Step S137: Based on the angles of each asymmetric extension direction in the set of asymmetric extension directions of the spread profile, draw extension direction rays from the geometric centroid to each local maximum point of radial distance on the spread profile ring, and draw obstruction markers perpendicular to the radial direction at each local minimum point of radial distance. Superimpose the violently fluctuating profile segment, the extension direction rays, and the obstruction markers on the solder joint area image around the soldered components to generate an abnormal fluctuation map of the wetted spread profile.

[0047] To generate an abnormal fluctuation map of the wetting spread profile on the image of the solder joint area around the soldered components: connect the boundary points of the spread profile with a white line of 2 pixels in the tracing order to form a closed spread profile loop; for each element in the set of asymmetric extension directions of the spread profile, draw a red ray of length 1.2*R_max from the geometric centroid (G_center_row, G_center_col) along the polar angle φ_max, with an arrow at the end; for each local minimum point of radial distance, calculate the radial unit vector (u_row, u_col)=((B_min_row-G_center_row) / R_min, (B_min_col-G_center_col) / R_min), and the tangential unit vector (v_row, v_col)=(-u_col, u_row), and draw a blue line of length 10 pixels along the tangential direction at this point, with the midpoint at this point and extending 5 pixels at each end; outline the violently fluctuating profile segment with a yellow line of 3 pixels in width to generate an abnormal fluctuation map of the wetting spread profile.

[0048] Step S140: Combine the solder joint morphology distortion distribution map and the wetting spread contour abnormal fluctuation map to construct a spatial coupling correlation model of component welding defects, and output a spatial coupling defect correlation mapping map. The spatial coupling defect correlation mapping map records the coordinate set of the spatially overlapping area between solder joint morphology distortion and wetting spread contour abnormality.

[0049] Step S141: Project the solder joint position offset trajectory line in the solder joint morphology distortion distribution map and the extension direction ray in the wetting and spreading contour abnormal fluctuation map into the same spatial coordinate system to obtain a spatial direction vector superposition map, which includes the offset trajectory direction vector and the extension direction ray vector.

[0050] The physical coordinate system of the printed circuit board is adopted as a unified spatial coordinate system, with the origin at the lower left corner, the horizontal direction to the right as the positive x-axis, and the vertical direction upward as the positive y-axis. Pixel coordinates are converted to physical coordinates using camera calibration parameters: the horizontal axis conversion factor is S_x (mm / pixel), and the vertical axis conversion factor is S_y (mm / pixel). Points (P_row, P_col) on the solder joint offset trajectory line are converted to (X=P_col*S_x, Y=P_row*S_y). The geometric centroid of the starting point of the extension ray in the wetting spread contour anomaly fluctuation map is also converted to physical coordinates, while the ray direction angle remains unchanged. The converted solder joint offset trajectory line and the extension ray are plotted in the same physical coordinate system to obtain a spatial direction vector overlay map.

[0051] Step S142: Calculate the spatial distance between the inflection point of the offset trajectory of each solder joint in the solder joint morphology distortion distribution map and the abnormal protrusion position of each spreading profile in the abnormal fluctuation map of the wetting spreading profile, and pair the inflection points with spatial distances less than a preset distance threshold with the protrusion positions, record them as candidate associated feature pairs, count the number of candidate associated feature pairs, and obtain the number of directional associated feature pairs.

[0052] Let U be the number of inflection points on the weld point offset trajectory, and let (K_u_x, K_u_y) be the physical coordinates of each inflection point; let V be the number of abnormal protrusions in the spread contour, and let (P_v_x, P_v_y) be the physical coordinates of each protrusion. Calculate the spatial distance D_{u, v} between each inflection point and each protrusion. A preset distance threshold D_th = 2 mm is used. When D_{u, v} < 2 mm, pair the inflection point u with the protrusion v as a candidate associated feature pair and record (u, v, D_{u, v}). Count the total number of candidate associated feature pairs to obtain the number of directional associated feature pairs C_dir.

[0053] Step S143: Perform a spatial intersection operation on the boundary coordinates of the spatially overlapping region of morphological distortion and size distortion in the solder joint morphological distortion distribution map and the contour coordinates of the violently fluctuating contour segment in the abnormal fluctuation map of the wetting and spreading contour to obtain a set of polygons of spatially overlapping regions. The set of polygons of spatially overlapping regions records the spatial range in which the solder joint morphological distortion region and the spreading contour fluctuation region overlap.

[0054] The spatial intersection operation is performed between the boundary coordinates of the spatially overlapping regions in the solder joint morphology distortion distribution map (i.e., the solder joint contour edge coordinate sequence of the solder joints marked as spatially overlapping regions) and the contour coordinates of the violently fluctuating contour segments in the wetting spread contour anomaly fluctuation map. The polygon formed by the boundary coordinates of the spatially overlapping regions is denoted as polygon A, and the polygon formed by the contour coordinates of the violently fluctuating contour segments is denoted as polygon B. The intersection region of polygon A and polygon B is calculated, and the boundary coordinates of the intersection region form a new polygon. The intersection is calculated for each spatially overlapping region and each violently fluctuating contour segment, resulting in a set of spatially overlapping region polygons. This set of spatially overlapping region polygons records the spatial extent of the overlap between the solder joint morphology distortion region and the spread contour fluctuation region.

[0055] Step S144: Calculate the spatial distance between the center point coordinates of the candidate region of size distortion in the solder joint morphology distortion distribution map and the coordinates of the obstructed diffusion position in the abnormal fluctuation map of the wetting spread profile, and obtain the set of proximity distance parameters between the obstructed position and the size distortion region. The set of proximity distance parameters records the spatial interval between each obstructed diffusion position of the spread profile and the nearest candidate region of size distortion.

[0056] For each location where the spread contour diffusion is obstructed, its physical coordinates are (Q_w_x, Q_w_y). There are multiple candidate regions for size distortion, and the center point coordinates of each candidate region are (Z_r_x, Z_r_y). The center point coordinates are equal to the average of the coordinates of all pixels within that candidate region. Calculate the spatial distance between the obstructed location and the center point of each candidate region, and take the minimum value as the nearest neighbor parameter N_w = min_{r}sqrt((Q_w_x-Z_r_x)^2+(Q_w_y-Z_r_y)^2). Calculate this for each location where the spread contour diffusion is obstructed, obtaining the nearest neighbor parameter set {N_1, N_2, ..., N_W}, where W is the total number of locations where the spread contour diffusion is obstructed.

[0057] Step S145: Extract a preset number of feature points with the smallest spatial distance from the candidate associated feature pairs, or record all feature point pairs with a distance less than a preset threshold as directional coupling feature pairs. The directional coupling feature pairs characterize the spatial proximity relationship between the solder joint position offset and the abnormal spread of solder.

[0058] Extract the direction coupling feature pairs from the candidate associated feature pairs. Use one of the following two methods: Method 1: Sort the candidate associated feature pairs in ascending order of the spatial distance Du,v, and extract the first H pairs as the direction coupling feature pairs. H is set to 30% of the total number of candidate associated feature pairs. Method 2: Directly record all candidate associated feature pairs that satisfy Du,v < Dth as the direction coupling feature pairs. Each element in the direction coupling feature pair contains the inflection point index u, the bulge position index v, and the spatial distance Du,v, which characterizes the proximity relationship between the solder joint position offset and the abnormal solder spreading in terms of spatial position.

[0059] Based on the set of spatial overlap region polygons, the set of adjacent distance parameters, and the direction coupling feature pairs, extract the defect coupling regions and adjacent coupling defect pairs and draw the spatial coupling connection annotations. Specifically, it may include: Step S146: Extract the spatial overlap region polygons with an area exceeding the preset area threshold from the set of spatial overlap region polygons, and mark the extracted spatial overlap region polygons as target defect overlap regions. The target defect overlap regions characterize the defect coupling positions where the solder joint morphology distortion and the spreading profile abnormality coincide in space.

[0060] Calculate the area of each polygon in the set of spatial overlap region polygons. For a closed polygon with T vertices and vertex coordinates (x1, y1), (x2, y2),..., (xT, yT) in sequence, the area Area calculation formula is: Area = 0.5×|(x1×y2 - x2×y1) + (x2×y3 - x3×y2) +... + (xT×y1 - x1×yT)|. The preset area threshold Ath is set to 0.5 square millimeters. Extract the spatial overlap region polygons with Area ≥ 0.5 square millimeters and mark them as target defect overlap regions, which characterize the defect coupling positions where the solder joint morphology distortion and the spreading profile abnormality coincide in space.

[0061] Step S147: Extract the positions where the spreading profile diffusion is blocked and the candidate regions of size distortion with adjacent distance parameters less than the preset adjacent distance threshold from the set of adjacent distance parameters, and pair and record the extracted positions where the spreading profile diffusion is blocked and the candidate regions of size distortion as adjacent coupling defect pairs. The adjacent coupling defect pairs characterize the coupling relationship between the solder joint size distortion and the spreading blocked defects in terms of adjacent distribution in space.

[0062] Extract the positions where the spreading profile diffusion is blocked with \(N_w < N_{th}\) from the set of adjacent distance parameters, and set \(N_{th}\) to 1 mm. For each position where the spreading profile diffusion is blocked that meets the condition, find the candidate region of size distortion that minimizes its adjacent distance parameter, pair this blocked position with the candidate region of size distortion, and record it as an adjacent coupling defect pair. The adjacent coupling defect pair includes the coordinates of the position where the spreading profile diffusion is blocked, the coordinates of the center point of the candidate region of size distortion, and an index, which characterizes the coupling relationship between the solder joint size distortion and the spatially adjacent distribution of the spreading blockage defect.

[0063] Step S148: Pair and connect the inflection points of the solder joint position offset trajectory and the abnormally raised positions of the spreading profile in the direction coupling feature pairs with connecting line segments to obtain a set of direction coupling connecting lines. Each connecting line segment in the set of direction coupling connecting lines connects an inflection point of the solder joint position offset trajectory and an abnormally raised position of the spreading profile.

[0064] For each pair in the direction coupling feature pairs, obtain the physical coordinates \((K_{u_x}, K_{u_y})\) of the inflection point of the solder joint position offset trajectory and the physical coordinates \((P_{v_x}, P_{v_y})\) of the abnormally raised position of the spreading profile, and draw a line segment connecting these two points in the spatial direction vector superposition diagram. The two endpoints of the line segment are the inflection point and the raised position respectively. Each connecting line segment corresponds to a direction coupling feature pair, and all the connecting line segments form a set of direction coupling connecting lines.

[0065] Step S149: Mark the boundary of the target defect overlapping region with a highlighted closed curve, and connect the position where the spreading profile diffusion is blocked and the center point of the candidate region of size distortion in the adjacent coupling defect pair with a dashed line segment to obtain a set of adjacent coupling connecting lines.

[0066] In the spatial direction vector superposition diagram, mark the boundary of each target defect overlapping region with a highlighted closed curve. The highlighted closed curve is drawn with a magenta line 3 pixels wide. For each adjacent coupling defect pair, obtain the physical coordinates of the position where the spreading profile diffusion is blocked and the physical coordinates of the center point of the candidate region of size distortion, and connect these two points with a dashed line segment. The line type of the dashed line segment is set to a dotted line style and 2 pixels wide. All the dashed line segments form a set of adjacent coupling connecting lines.

[0067] Step S1410: Generate a spatial coupling defect association mapping diagram containing spatial coupling connection relationships and spatial overlapping region annotations based on the set of direction coupling connecting lines, the highlighted closed curves of the target defect overlapping regions, and the set of adjacent coupling connecting lines.

[0068] All solid line segments in the directional coupling connection line set, the highlighted magenta closed curve of the target defect overlap area, and all dashed line segments in the adjacent coupling connection line set are overlaid and plotted on the same physical coordinate system base map. The base map is a fused image obtained by stitching together the image of the soldered component body and the image of the solder joint area around the soldered component. Legends are added to the corners of the image: red solid lines represent directional coupling connection lines, magenta closed curves represent target defect overlap areas, and blue dashed lines represent adjacent coupling connection lines. A spatial coupling defect association mapping map containing spatial coupling connection relationships and spatial overlap area annotations is generated.

[0069] Step S150: Call the welding defect type decision tree to perform defect category determination on the spatial coupling defect association mapping map, and generate component welding defect identification results containing defect category labels and defect area boundary coordinates.

[0070] Step S151: Extract the total number of directional coupling lines in the set of directional coupling lines from the spatial coupling defect association mapping diagram to obtain the directional coupling line quantity parameter.

[0071] Extract the set of directional coupling connection lines from the spatial coupling defect association map, and count the total number of connection lines in the set, denoted as C_dir_line.

[0072] Step S152: Extract the total area of ​​the region enclosed by the highlighted closed curve of the target defect overlapping region from the spatial coupling defect association mapping map, and obtain the target overlapping region area parameter. The target overlapping region area parameter is equal to the cumulative sum of the surface areas of all target defect overlapping regions.

[0073] Extract the highlighted closed curves of all target defect overlapping regions from the spatial coupling defect association map, calculate the area of ​​the polygon enclosed by each closed curve, and sum all the faces to obtain the target overlapping region area parameter A_total, that is, A_total=Area_1+Area_2+...+Area_K, where K is the total number of target defect overlapping regions.

[0074] Step S153: Extract the total number of neighboring coupling lines in the set of neighboring coupling lines from the spatial coupling defect association mapping map to obtain the neighboring coupling line quantity parameter.

[0075] Extract the set of adjacent coupling lines from the spatial coupling defect association map, and count the total number of lines in the set, denoted as C_prox_line.

[0076] Step S154: Perform a first numerical comparison between the number parameter of the directional coupling connection lines and a preset directional coupling number threshold, and select the first-layer branch path of the welding defect type decision tree according to the result of the first numerical comparison. The first-layer branch path includes a path with excessive directional coupling number and a path with non-excessive directional coupling number.

[0077] Preset directional coupling number threshold C_dir_th. Compare C_dir_line with C_dir_th: If C_dir_line ≥ C_dir_th, select the path with excessive directional coupling number; if C_dir_line < C_dir_th, select the path with non-excessive directional coupling number.

[0078] Based on the results of each layer of numerical comparison, input the parameter combination into the corresponding leaf node to obtain the defect category label and output the recognition result, which may specifically include: Step S155: When the result of the first numerical comparison indicates selecting the path with excessive directional coupling number, perform a second numerical comparison between the target overlapping area parameter and a preset overlapping area threshold, and select the second-layer branch path of the welding defect type decision tree according to the result of the second numerical comparison. The second-layer branch path includes a path with excessive overlapping area and a path with non-excessive overlapping area.

[0079] Preset overlapping area threshold A_th. Compare A_total with A_th: If A_total ≥ A_th, select the path with excessive overlapping area; if A_total < A_th, select the path with non-excessive overlapping area.

[0080] Step S156: When the result of the second numerical comparison indicates selecting the path with excessive overlapping area, perform a third numerical comparison between the number parameter of the adjacent coupling connection lines and a preset adjacent coupling number threshold, and select the third-layer branch path of the welding defect type decision tree according to the result of the third numerical comparison. The third-layer branch path includes a path with excessive adjacent coupling number and a path with non-excessive adjacent coupling number.

[0081] Preset adjacent coupling number threshold C_prox_th. Compare C_prox_line with C_prox_th: If C_prox_line ≥ C_prox_th, select the path with excessive adjacent coupling number; if C_prox_line < C_prox_th, select the path with non-excessive adjacent coupling number.

[0082] Step S157: When the result of the third numerical comparison indicates that the path with excessive adjacent coupling is selected, the combination of the directional coupling connection line quantity parameter, the target overlapping area parameter, and the adjacent coupling connection line quantity parameter is input into the terminal leaf node of the welding defect type decision tree. The terminal leaf node outputs a first defect category label, which is used to identify spatially fully coupled welding defects.

[0083] Input the triple (C_dir_line, A_total, C_prox_line) into the leaf node at the end of the decision tree. The leaf node outputs the category label L1="spatial fully coupled welding defect".

[0084] Step S158: When the result of the third numerical comparison indicates that the number of adjacent couplings has not exceeded the standard, the combination of the directional coupling connection line quantity parameter and the target overlapping area parameter is input into the end leaf node of the welding defect type decision tree. The end leaf node outputs a second defect category label, which is used to identify directional overlapping coupling type welding defects.

[0085] Input the tuple (C_dir_line, A_total) into the leaf node at the end of the decision tree. The leaf node outputs the category label L2="Directional Overlap Coupling Welding Defect".

[0086] Step S159: When the result of the second numerical comparison indicates that the selected path with the overlapping area does not exceed the standard, the combination of the directional coupling connection line density parameter and the adjacent coupling connection line density parameter is input into the terminal leaf node of the welding defect type decision tree. The terminal leaf node outputs a third defect category label, which is used to identify directional adjacent coupling type welding defects.

[0087] Calculate the directional coupling connection line density D_dir = C_dir_line / S_area, where S_area is the total area of ​​the weld point and its surrounding region in the spatial coupling defect association mapping map. Calculate the proximity coupling connection line density D_prox = C_prox_line / S_area. Input (D_dir, D_prox) into the leaf node at the end of the decision tree, and output the category label L3 = "directional proximity coupling type welding defect".

[0088] Step S1510: When the result of the first numerical comparison indicates that the number of directional couplings has not exceeded the standard, the target overlapping area parameter is compared with the preset overlapping area threshold for the fourth time, and another branch path of the welding defect type decision tree is selected according to the result of the fourth numerical comparison. The other branch path includes the overlapping area exceeding the standard path and the overlapping area not exceeding the standard path.

[0089] Compare A_total with A_th: If A_total ≥ A_th, select the path with an excessive overlapping area; if A_total < A_th, select the path with a non-excessive overlapping area. When selecting the path with an excessive overlapping area, output the category label L4 = "isolated overlapping area defect"; when selecting the path with a non-excessive overlapping area, output the category label L5 = "no significant coupling defect".

[0090] Step S1511: Use the defect category label output by the terminal leaf node of the welding defect type decision tree as the defect category label in the component welding defect recognition result, and use the spatial region boundary coordinates corresponding to the defect category label in the spatial coupling defect association mapping diagram as the defect region boundary coordinates in the component welding defect recognition result.

[0091] Use the corresponding one of the defect category labels L1 to L5 output by the decision tree as the defect category label of the component welding defect recognition result. Determine the corresponding spatial region according to the defect category label: If the label is L1 or L2 or L4, extract the boundary coordinates of all target defect overlapping regions; if the label is L3, extract the boundary coordinates of the regions near the endpoints of all direction coupling connection lines and within 1 mm of the regions near the endpoints of adjacent coupling connection lines; if the label is L5, extract the boundary coordinates of the candidate positions of morphological distortion and the candidate regions of dimensional distortion. Use the above boundary coordinates as the defect region boundary coordinates, and jointly form the component welding defect recognition result with the defect category label.

[0092] Step S210: Extract the total number of all direction coupling connection lines in the direction coupling connection line set from the spatial coupling defect association mapping diagram to obtain the direction coupling connection line quantity parameter, compare the direction coupling connection line quantity parameter with the preset direction coupling quantity threshold, and generate a direction coupling anomaly determination flag according to the comparison result to determine whether there is a spatial direction coupling anomaly between the solder joint offset direction and the solder spreading direction.

[0093] Extract C_dir_line from the spatial coupling defect association mapping diagram. Preset C_dir_th = 3. If C_dir_line ≥ 3, generate the direction coupling anomaly determination flag F_dir = 1, indicating that there is a direction coupling anomaly; if C_dir_line < 3, generate F_dir = 0, indicating that there is no direction coupling anomaly.

[0094] Step S220: Extract the area enclosed by the highlighted closed curve of the target defect overlapping region from the spatial coupling defect association mapping map, sum the areas enclosed by the highlighted closed curve of the target defect overlapping region to obtain the total area parameter of the target defect overlapping region, and compare the total area parameter with the preset overlapping region area threshold. Based on the comparison result, determine whether there is a spatial overlap coupling anomaly between the solder joint morphology distortion region and the spread contour fluctuation region, and generate an overlap coupling anomaly determination mark.

[0095] Extract the area of ​​all overlapping regions of target defects and sum them up to obtain A_total. The default value of A_th is 2.5 square millimeters. If A_total ≥ 2.5, generate an overlap coupling anomaly judgment flag F_overlap = 1, indicating that an overlap coupling anomaly exists; if A_total < 2.5, generate F_overlap = 0, indicating that no overlap coupling anomaly exists.

[0096] Step S230: Extract the total number of all adjacent coupling lines in the adjacent coupling connection line set from the spatial coupling defect association mapping map to obtain the adjacent coupling connection line quantity parameter, and compare the adjacent coupling connection line quantity parameter with the preset adjacent coupling quantity threshold. Based on the comparison result, determine whether there is a spatial adjacent coupling anomaly between the solder joint size distortion area and the spread contour diffusion obstruction position, and generate an adjacent coupling anomaly determination mark.

[0097] Extract C_prox_line. The default value is C_prox_th=4. If C_prox_line≥4, generate a proximity coupling anomaly flag F_prox=1, indicating the presence of a proximity coupling anomaly; if C_prox_line<4, generate F_prox=0, indicating the absence of a proximity coupling anomaly.

[0098] The directional coupling anomaly determination identifier, the overlapping coupling anomaly determination identifier, and the adjacent coupling anomaly determination identifier are input into the welding defect type determination matrix. Based on the combined state, the defect type code is output as the identification result, which may specifically include: Step S240: Input the directional coupling anomaly determination identifier, the overlapping coupling anomaly determination identifier, and the adjacent coupling anomaly determination identifier into the welding defect type determination matrix. The welding defect type determination matrix includes three determination dimensions: directional coupling dimension, overlapping coupling dimension, and adjacent coupling dimension. Each determination dimension corresponds to a binary input port. Based on the combination state of the binary input values ​​of the three determination dimensions, output the corresponding defect type code from the output port of the welding defect type determination matrix.

[0099] A welding defect type determination matrix is ​​constructed, which is a 3-input, 1-output lookup table. The input ports sequentially receive F_dir, F_overlap, and F_prox. The output port outputs the corresponding defect type code based on the input combination. The mapping relationship between the input combination and the output code is as follows: (1, 1, 1) output code E1; (1, 1, 0) output code E2; (1, 0, 1) output code E3; (0, 1, 1) output code E4; (1, 0, 0) output code E5; (0, 1, 0) output code E6; (0, 0, 1) output code E7; (0, 0, 0) output code E8.

[0100] Step S250: When the value of the directional coupling anomaly determination identifier is a first directional coupling state value, the value of the overlapping coupling anomaly determination identifier is a first overlapping coupling state value, and the value of the adjacent coupling anomaly determination identifier is a first adjacent coupling state value, the welding defect type determination matrix outputs a first defect type code, which corresponds to a directional overlapping adjacent full coupling type welding defect; when the value of the directional coupling anomaly determination identifier is a first directional coupling state value, the value of the overlapping coupling anomaly determination identifier is a first overlapping coupling state value, and the value of the adjacent coupling anomaly determination identifier is a second adjacent coupling state value, the welding defect type determination matrix outputs a second defect type code, which corresponds to a directional overlapping double coupling type welding defect. Welding defects; when the value of the directional coupling anomaly determination identifier is a first directional coupling state value, the value of the overlapping coupling anomaly determination identifier is a second overlapping coupling state value, and the value of the adjacent coupling anomaly determination identifier is a first adjacent coupling state value, the welding defect type determination matrix outputs a third defect type code, the third defect type code corresponding to a directional adjacent double coupling type welding defect; when the value of the directional coupling anomaly determination identifier is a second directional coupling state value, the value of the overlapping coupling anomaly determination identifier is a first overlapping coupling state value, and the value of the adjacent coupling anomaly determination identifier is a first adjacent coupling state value, the welding defect type determination matrix outputs a fourth defect type code, the fourth defect type code corresponding to an overlapping adjacent double coupling type welding defect.

[0101] According to the mapping relationship in step S240: when (F_dir, F_overlap, F_prox) = (1, 1, 1), output E1 corresponds to "directional overlapping adjacent fully coupled welding defect"; when (1, 1, 0), output E2 corresponds to "directional overlapping double coupled welding defect"; when (1, 0, 1), output E3 corresponds to "directional adjacent double coupled welding defect"; when (0, 1, 1), output E4 corresponds to "overlapping adjacent double coupled welding defect". For the other combinations (1, 0, 0), output E5 corresponds to "isolated directional coupled welding defect"; (0, 1, 0), output E6 corresponds to "isolated overlapping coupled welding defect"; (0, 0, 1), output E7 corresponds to "isolated adjacent coupled welding defect"; and (0, 0, 0), output E8 corresponds to "uncoupled defect".

[0102] Step S260: Use the defect type code output by the welding defect type determination matrix as the defect category label in the component welding defect identification result, and use the spatial region boundary coordinates corresponding to the defect type code in the spatial coupling defect association mapping map as the defect region boundary coordinates in the component welding defect identification result.

[0103] The corresponding defect type codes E1 to E8 output by the decision matrix are used as the defect category labels for the component welding defect identification results. Based on the codes, the corresponding spatial regions are determined: E1, E2, and E4 extract the union of the boundaries of all overlapping areas of target defects and the boundaries of the areas covered by directional coupling lines; E3 extracts the union of the areas covered by all directional coupling lines and the areas covered by adjacent coupling lines; E5 extracts the region within 1.5 mm of the endpoints of directional coupling lines; E6 extracts the boundaries of overlapping areas of target defects; E7 extracts the region within 1.5 mm of the endpoints of adjacent coupling lines; E8 extracts the boundaries of candidate locations for morphological distortion and candidate regions for dimensional distortion. These boundary coordinates are used as the defect region boundary coordinates, and together with the defect category codes, they form the component welding defect identification results.

[0104] Step S310: Obtain the image of the soldered component body at the first acquisition time in the continuous acquisition image sequence as a reference image, and obtain the image of the soldered component body at the second acquisition time in the continuous acquisition image sequence as a comparison image. The second acquisition time is later than the first acquisition time in time sequence. The reference image and the comparison image contain the surface state records of the same electronic component at different soldering stages.

[0105] The 10th frame of the soldered component body image is selected from the continuously acquired image sequence as the reference image, and the 50th frame of the soldered component body image is selected as the comparison image. The two frames correspond to the surface state records of the same surface mount component during the soldering stage at the 10th and 50th seconds.

[0106] Step S320: Perform feature extraction operation on the solder joint position offset trajectory line of the solder joint morphology distortion distribution map in the reference image to obtain the inflection point coordinate sequence of the reference solder joint position offset trajectory line and the boundary coordinate set of the offset direction change region, and record the inflection point coordinate sequence of the reference solder joint position offset trajectory line and the boundary coordinate set of the offset direction change region as the reference morphology distortion feature set.

[0107] The reference image is processed according to steps S121 to S128 to obtain the reference solder joint morphological distortion distribution map. From this map, the inflection point coordinate sequence {K1_u} of the solder joint position offset trajectory line and the boundary coordinate set {B1_m} of the abrupt shift in offset direction regions are extracted. {K1_u} and {B1_m} are merged and recorded as the reference morphological distortion feature set F_base.

[0108] Step S330: Perform feature extraction operation on the solder joint position offset trajectory line of the solder joint morphology distortion distribution map in the comparison image to obtain the inflection point coordinate sequence of the solder joint position offset trajectory line and the boundary coordinate set of the offset direction change region, and record the inflection point coordinate sequence of the solder joint position offset trajectory line and the boundary coordinate set of the offset direction change region as the comparison morphology distortion feature set.

[0109] The comparison images were processed in the same way to obtain a comparison solder joint morphology distortion distribution map. The inflection point coordinate sequence {K2_u} and the boundary coordinate set {B2_m} of the abrupt shift in offset direction region were extracted and merged into a comparison morphology distortion feature set F_comp.

[0110] Step S340: After spatially aligning the reference image with the comparison image, perform a matching operation based on spatial proximity between the inflection point coordinate sequence of the reference solder joint position offset trajectory line in the reference morphological distortion feature set and the inflection point coordinate sequence of the comparison solder joint position offset trajectory line in the comparison morphological distortion feature set. Calculate the spatial displacement vector between each set of matched inflection point coordinates to obtain a set of inflection point spatial displacement vectors. The set of inflection point spatial displacement vectors records the position movement direction and movement distance of each inflection point from the first acquisition time to the second acquisition time.

[0111] A spatial nearest neighbor matching algorithm is employed: for each inflection point coordinate K1_u in the reference image, the nearest inflection point coordinate K2_v in the comparison image is found based on Euclidean distance; if the distance is less than 0.5 mm, a match is considered to have been made. For each matched pair of inflection points, the spatial displacement vector ΔK_u = (K2_v_x - K1_u_x, K2_v_y - K1_u_y) is calculated. The displacement vectors of all matched inflection points constitute the inflection point spatial displacement vector set {ΔK_1, ΔK_2, ..., ΔK_U}, where U is the number of successfully matched inflection points.

[0112] Step S350: Calculate the region overlap between the boundary coordinate set of the offset direction abrupt change region in the reference morphological distortion feature set and the boundary coordinate set of the offset direction abrupt change region in the comparison morphological distortion feature set, obtain the overlap parameter of the offset direction abrupt change region, and determine the change trend of the offset direction abrupt change region in the welding process based on the overlap parameter, and generate an evolution direction identifier for the abrupt change region.

[0113] For the set of boundary regions of abrupt change in the reference offset direction {B1_m} and the set of boundary regions of abrupt change in the comparison offset direction {B2_n}, calculate the overlap IoU for each pair of regions: IoU = area(B1_m∩B2_n) / area(B1_m∪B2_n). Take the maximum IoU_max among all region pairs. If IoU_max ≥ 0.6, generate a mutation region evolution direction identifier T_region = "stable"; if 0.3 ≤ IoU_max < 0.6, generate T_region = "drifting"; if IoU_max < 0.3, generate T_region = "new or disappeared".

[0114] Based on the set of inflection point spatial displacement vectors and the evolution direction identifiers of the abrupt change regions, stability parameters and evolution rates are calculated, and trend vectors are constructed to predict evolutionary trends. Specifically, this may include: Step S360: Extract the set of solder joint indexes marked as candidate regions for size distortion from the reference morphological distortion feature set, extract the set of solder joint indexes marked as candidate regions for size distortion from the comparison morphological distortion feature set, calculate the intersection of the two solder joint index sets, and obtain the set of solder joint indexes marked as candidate regions for size distortion at both acquisition times. The stability parameter of the solder joint size distortion position is obtained based on the number or proportion of solder joint indexes in the intersection.

[0115] Let S_base be the set of solder joint indices corresponding to the candidate regions of size distortion in the baseline image, and S_comp be the set of solder joint indices corresponding to these regions in the comparison image. Calculate the intersection S_stable = S_base ∩ S_comp. The stability parameter P_stable = |S_stable| / |S_base|, where |·| represents the cardinality of the set.

[0116] Step S370: Sum the movement distances of all inflection points in the inflection point spatial displacement vector set to obtain the total cumulative displacement of the inflection points. Divide the total cumulative displacement of the inflection points by the total number of inflection points in the inflection point spatial displacement vector set to obtain the average displacement distance of the inflection points. Based on the comparison result between the average displacement distance of the inflection points and the preset displacement threshold, generate a weld joint morphology distortion evolution rate identifier.

[0117] The total cumulative displacement of the inflection points, D_total, is equal to the sum of the magnitudes of the displacement vectors of all matched inflection points, i.e., D_total = |ΔK_1| + |ΔK_2| + ... + |ΔK_U|, where |ΔK_u| = sqrt(ΔK_u_x^2 + ΔK_u_y^2). The average displacement distance of the inflection points, D_avg, is D_total / U. The preset displacement threshold, D_th, is 0.3 mm. If D_avg ≥ 0.3 mm, the evolution rate flag R_rate is generated as "rapid evolution"; if D_avg < 0.3 mm, R_rate is generated as "slow evolution".

[0118] Step S380: Based on the evolution direction identifier of the mutation region, the stability parameter of the solder joint size distortion position, and the evolution rate identifier of the solder joint morphological distortion, construct a time evolution trend vector of solder joint morphological distortion. The time evolution trend vector includes the evolution direction component of the mutation region, the size distortion stability component, and the morphological distortion rate component.

[0119] Construct a three-dimensional time evolution trend vector V_trend=(T_region, P_stable, R_rate). Where T_region is a discrete value that can be "stable", "drifting" or "new or disappearing", P_stable is a continuous value between 0 and 1, and R_rate is a discrete value that can be "rapid evolution" or "slow evolution".

[0120] Step S390: Input the time evolution trend vector into a pre-trained welding defect evolution trend classifier. The welding defect evolution trend classifier outputs the evolution trend prediction category of the welding defect in the subsequent welding stage based on the numerical combination of each component in the time evolution trend vector. The evolution trend prediction category is attached to the component welding defect identification result to generate an extended component welding defect identification result containing defect category label, defect region boundary coordinates, and evolution trend prediction category. The evolution trend prediction category includes defect deterioration trend category, defect stabilization trend category, and defect mitigation trend category.

[0121] The welding defect evolution trend classifier employs a three-layer fully connected neural network structure: 3 nodes in the input layer, 8 nodes in the hidden layer using the ReLU activation function, and 3 nodes in the output layer using the Softmax activation function to output three class probabilities. The classifier is pre-trained using a training set containing 2000 labeled samples. The optimizer is Adam, with a learning rate of 0.001, a batch size of 32, and 100 training epochs. The classifier is input with V_trend, and the class with the highest output probability is used as the predicted evolution trend class. If the output is "defect worsening trend class," it indicates that the welding defect will worsen in the subsequent stage; if it is "defect stabilizing trend class," it indicates that the defect state remains unchanged; if it is "defect mitigation trend class," it indicates that the defect may improve spontaneously. This predicted class is appended to the component welding defect identification results to generate extended identification results.

[0122] Step S3100: Obtain the solder joint morphology distortion distribution map of all soldered component body images in the continuously acquired image sequence, and stack the solder joint position offset trajectory lines in all solder joint morphology distortion distribution maps according to the order of acquisition time to obtain a three-dimensional spatiotemporal stack of solder joint position offset trajectory lines. The three-dimensional spatiotemporal stack forms a three-dimensional coordinate system with spatial horizontal coordinate, spatial vertical coordinate and acquisition time coordinate.

[0123] The continuously acquired image sequence contains T frames of images of the soldered components. For each frame, steps S121 to S128 are used to generate a solder joint morphology distortion distribution map, from which the solder joint position offset trajectory lines are extracted. A three-dimensional coordinate system is constructed using the spatial horizontal coordinate X, spatial vertical coordinate Y, and acquisition time coordinate t. All T trajectory lines are stacked in chronological order, with each trajectory line located on the t=ti plane, forming a three-dimensional spatiotemporal stack.

[0124] Step S3110: In the three-dimensional spatiotemporal stack of the solder joint position offset trajectory line, for the spatial position corresponding to each solder joint index, extract its offset direction angle at each acquisition time to form the offset direction angle time series of each solder joint index, and obtain the offset direction angle time series of each solder joint index. The offset direction angle time series records the offset direction angle of the solder joint index at different acquisition times.

[0125] For solder joint index i, extract θ_offset_i(t) from the set of solder joint position offset vectors at each time point, and arrange them in chronological order to form a time series {θ_i(t1), θ_i(t2), ..., θ_i(tT)}.

[0126] Step S3120: Calculate the first time derivative of the time function of the offset direction angle of each spatial location point as a function of time, obtain the time distribution sequence of the offset direction angle change rate, and generate the offset direction oscillation frequency parameter based on the alternation frequency of positive and negative values ​​in the time distribution sequence of the offset direction angle change rate.

[0127] For the θ_i(t) sequence, calculate the first-order time derivative dθ_i / dt≈(θ_i(t_{k+1})-θ_i(t_k)) / (t_{k+1}-t_k), obtaining the rate sequence {r_i(t1), r_i(t2), ..., r_i(t_{T-1})}. Count the number of alternations between positive and negative values ​​in the rate sequence, let the number of alternations be Alt_i, the total time interval length be T_total=t_T-t_1, and the oscillation frequency parameter f_swing_i=Alt_i / T_total. Take the average over all solder joints to obtain f_swing_avg.

[0128] Step S3130: Extract the coordinate distribution of the inflection points of the solder joint position offset trajectory line in three-dimensional space from the three-dimensional spatiotemporal stack of the solder joint position offset trajectory line, project the three-dimensional coordinates of each inflection point onto the acquisition time coordinate axis to obtain a discrete distribution set of inflection point occurrence time points, and generate an inflection point time interval sequence according to the time interval between adjacent inflection point occurrence time points in the discrete distribution set.

[0129] Extract the 3D coordinates (K_x, Ky, t_k) of all inflection points from the 3D spatiotemporal stack, and project them onto the t-axis to obtain the set of inflection point occurrence time points {t_k}. Sort the time points in ascending order, and calculate the interval between adjacent time points Δt_j = t_{j+1} - t_j to obtain the inflection point time interval sequence {Δt_1, Δt_2, ..., Δt_{J-1}}, where J is the total number of inflection points.

[0130] Step S3140: Calculate the average inflection point occurrence frequency based on the inflection point time interval sequence, compare the offset direction swing frequency parameter with the average inflection point occurrence frequency, calculate their ratio or difference, and obtain the closeness parameter between the swing frequency and the inflection point occurrence frequency.

[0131] The average frequency of inflection points is f_kp = 1 / avg(Δt_j). The ratio of the oscillation frequency to the frequency of inflection points, R_freq = f_swing_avg / f_kp, is calculated to obtain the proximity parameter.

[0132] Based on the proximity parameter between the swing frequency and the inflection point occurrence frequency, a dynamic evolution feature triplet is constructed and input into a classification model to obtain a dynamic type identifier, which may specifically include: Step S3150: Extract the solder joint size deformation parameter corresponding to the gradient peak in the solder joint size change gradient sequence from the solder joint morphology distortion distribution map, arrange the solder joint size deformation parameter into a size deformation time series according to the chronological order of acquisition time, and perform a difference operation on the size deformation time series to obtain the size deformation change sequence.

[0133] Extract the gradient peak value ΔR_peak(t) from the gradient sequence of weld joint size changes at each time point, and arrange them in chronological order as {ΔR_peak(t1), ΔR_peak(t2), ..., ΔR_peak(tT)}. Perform first-order difference to obtain the change Δ(ΔR_peak)(t_k) = ΔR_peak(t_{k+1}) - ΔR_peak(t_k), thus obtaining the sequence of size deformation changes.

[0134] Step S3160: Based on the proximity parameter between the oscillation frequency and the inflection point occurrence frequency, the maximum value of the change in the sequence of dimensional deformation amount, and the oscillation frequency parameter of the offset direction, construct a dynamic evolution feature triplet for solder joint morphology distortion. The dynamic evolution feature triplet includes frequency proximity element, deformation change element, and oscillation frequency element.

[0135] Construct a dynamic evolution feature triple V_dynamic=(R_freq,ΔR_max,f_swing_avg), where ΔR_max is the maximum value in the sequence of changes in size deformation.

[0136] Step S3170: Input the dynamic evolution feature triplet of the weld joint morphology distortion into the pre-trained welding defect dynamic classification model. The welding defect dynamic classification model performs weighted aggregation processing on the three elements in the dynamic evolution feature triplet and outputs the dynamic type identifier of the welding defect. The dynamic type identifier includes periodic fluctuation defect type, monotonically deteriorating defect type and random fluctuation defect type.

[0137] The dynamic classification model for welding defects employs a support vector machine (SVM) classifier with a radial basis function (RBF) kernel. The model is pre-trained using 1500 feature triplets and corresponding labels, with a kernel parameter gamma=0.1 and a regularization parameter C=1.0. Inputting V_dynamic into the model, the model calculates the decision function value and outputs the category: Category 1 corresponds to "periodic fluctuation defect type," indicating that the defect fluctuates regularly with the welding cycle; Category 2 corresponds to "monotonically worsening defect type," indicating that the defect continuously worsens over time; and Category 3 corresponds to "random fluctuation defect type," indicating that the defect changes without a clear pattern.

[0138] Step S3180: Combine the dynamic type identifier of the welding defect with the defect category label in the component welding defect identification result to generate a composite welding defect identification result that includes static defect category and dynamic defect type. The composite welding defect identification result records the spatial distribution characteristics and temporal evolution characteristics of the welding defect.

[0139] The dynamic type identifier output by the dynamic classification model is concatenated with the defect category label output in step S1511 to generate a composite recognition result, in the format of (static category, dynamic type, defect area boundary coordinates).

[0140] Step S3190: Output the composite welding defect identification result to the control system of the production inspection line, trigger the defect handling operation corresponding to the composite welding defect identification result, the defect handling operation includes at least one of the following operations: adjusting welding parameters, marking defective components, and triggering an alarm.

[0141] The composite identification result is sent to the production inspection line control system. If the dynamic type is "monotonically deteriorating defect type" and the static category is "spatially fully coupled welding defect", the control system triggers an alarm operation and stops the production line; if the dynamic type is "periodic fluctuation defect type", the control system triggers an operation to adjust welding parameters, reducing the reflow oven temperature by 5% proportionally; if the dynamic type is "random fluctuation defect type" and the static category is "directionally overlapping coupled welding defect", the control system triggers an operation to mark the defective component, controlling the inkjet printer to spray a red defect mark on the edge of the printed circuit board.

[0142] For example, the method may further include: step S410: obtaining the spatial distribution coordinates of all directional coupling lines in the directional coupling connection line set in the spatial coupling defect association mapping diagram, and recording the spatial coordinates of the two endpoints of each directional coupling connection line as the first endpoint coordinates and the second endpoint coordinates, respectively. The first endpoint coordinates correspond to the spatial position of the inflection point of the weld point position offset trajectory, and the second endpoint coordinates correspond to the spatial position of the abnormal protrusion of the spread contour.

[0143] Read the set of directional coupling connection lines from the spatial coupling defect association mapping diagram. For each connection line L_i, record its endpoint coordinates: A_i coordinates are the inflection points of the weld point position offset trajectory (K_u_x, K_u_y), and B_i coordinates are the abnormal protrusion positions of the spread contour (P_v_x, P_v_y).

[0144] Step S420: Calculate the Euclidean space distance between the coordinates of the first endpoint and the second endpoint of each directional coupling connection line to obtain the set of spatial span parameters of the directional coupling connection line, and record the maximum spatial span parameter and the minimum spatial span parameter in the set of spatial span parameters as the maximum extension distance and the minimum extension distance of the directional coupling connection line, respectively.

[0145] For each directional coupling connection line, calculate the spatial span S_i = sqrt((A_i_x - B_i_x)^2 + (A_i_y - B_i_y)^2), obtaining the set {S_1, S_2, ..., S_{C_dir_line}}. Take the maximum value S_max = max{S_i} and the minimum value S_min = min{S_i}.

[0146] Step S430: Calculate the standard deviation of the spatial span parameters of all directionally coupled lines in the set of directionally coupled lines to obtain the spatial span discrete parameters of the directionally coupled lines, and generate the spatial span variation coefficient of the directionally coupled lines based on the ratio of the spatial span discrete parameters to the average spatial span parameters.

[0147] Calculate the average spatial span S_avg, i.e., S_avg = (S_1 + S_2 + ... + S_C_dir_line) / C_dir_line. Calculate the standard deviation S_std = sqrt([(S_1 - S_avg)^2 + (S_2 - S_avg)^2 + ... + (S_C_dir_line - S_avg)^2] / C_dir_line. Calculate the coefficient of variation of spatial span CV_span = S_std / S_avg.

[0148] Step S440: Extract the spatial distribution coordinates of all adjacent coupling lines in the set of adjacent coupling lines from the spatial coupling defect association mapping map. Record the spatial coordinates of the two endpoints of each adjacent coupling line as the coordinates of the position where the spreading profile diffusion is blocked and the coordinates of the center point of the candidate region of size distortion, respectively. Calculate the Euclidean spatial distance between the coordinates of the two endpoints to obtain the set of spatial interval parameters of adjacent coupling lines.

[0149] For each adjacent coupled connection line, endpoint C_i is the location where the spread profile diffusion is hindered (Q_w_x, Q_w_y), and endpoint D_i is the center point of the candidate region for size distortion (Z_r_x, Z_r_y). The spatial interval I_i = sqrt((C_i_x - D_i_x)^2 + (C_i_y - D_i_y)^2) is calculated to obtain the set {I_1, I_2, ..., I_{C_prox_line}}.

[0150] Step S450: The average spatial span parameter of the directional coupling connection line is compared with the minimum spatial interval parameter in the set of spatial interval parameters of the adjacent coupling connection lines to obtain the spatial scale comparison parameter of directional coupling and adjacent coupling. The spatial scale comparison parameter reflects the degree of scale difference between the range of directional coupling and the range of adjacent coupling.

[0151] Calculate the average spatial span of directional coupling, S_avg, and calculate the minimum spatial interval of proximity coupling, I_min = min{I_i}. The spatial scale comparison parameter is R_scale = S_avg / I_min.

[0152] Step S460: Extract the shape features of the region enclosed by the highlighted closed curve of the target defect overlapping area from the spatial coupling defect association mapping map, calculate the length of the long side and the length of the short side of the minimum bounding rectangle of the region enclosed by the highlighted closed curve, and divide the length of the long side by the length of the short side to obtain the aspect ratio parameter of the target defect overlapping area.

[0153] For each overlapping polygon of the target defect, calculate its minimum bounding rectangle: find the minimum x_min, maximum x_max, minimum y_min, and maximum y_max of the polygon's vertices. The longer side of the rectangle is L_long = max(x_max - x_min, y_max - y_min), and the shorter side is L_short = min(x_max - x_min, y_max - y_min). The aspect ratio AR = L_long / L_short. If there are multiple overlapping regions, take the average AR of all regions, AR_avg.

[0154] Step S470: Based on the spatial span variation coefficient of the directional coupling connection line, the spatial scale comparison parameter of the directional coupling and adjacent coupling, and the aspect ratio parameter of the overlapping area of ​​the target defect, construct a spatial coupling structure feature vector. The spatial coupling structure feature vector includes a three-dimensional feature combination of span variation coefficient component, scale comparison component, and aspect ratio component.

[0155] Construct a three-dimensional spatial coupling structure feature vector V_struct=(CV_span, R_scale, AR_avg).

[0156] Step S480: Input the spatial coupling structure feature vector into a pre-trained spatial coupling pattern recognition network. The spatial coupling pattern recognition network performs a nonlinear mapping transformation on the three-dimensional feature combination of the spatial coupling structure feature vector and outputs a spatial coupling pattern category identifier. The spatial coupling pattern category identifier includes a centralized spatial coupling pattern identifier, a decentralized spatial coupling pattern identifier, and a chain spatial coupling pattern identifier.

[0157] The spatial coupling pattern recognition network employs a three-layer radial basis function neural network structure: 3 nodes in the input layer, 10 nodes in the hidden layer using Gaussian radial basis functions, and 3 nodes in the output layer using linear activation functions. The network is pre-trained using 1200 sets of feature vectors and annotations. When V_struct is input into the network, the network calculates the activation values ​​of the hidden layer nodes, and the maximum value of the weighted summation of the output layer values ​​corresponds to the category. For example, if category 1 corresponds to "concentrated spatial coupling pattern identifier," it indicates that the directional coupling lines are concentrated in a small spatial region; category 2 corresponds to "dispersed spatial coupling pattern identifier," it indicates that the lines are dispersed; and category 3 corresponds to "chain-like spatial coupling pattern identifier," it indicates that the lines are arranged in a chain-like sequence.

[0158] Step S490: Associate and store the spatial coupling mode category identifier with the defect category label and defect area boundary coordinates in the component welding defect identification result to generate an extended defect identification record containing spatial coupling structure information. The extended defect identification record is used to describe the spatial coupling connection mode and coupling area morphology of the welding defect.

[0159] The spatial coupling mode category identifier M_struct is associated with the defect category label L_defect and the defect region boundary coordinate B_defect output in step S1511 and stored as an extended defect identification record Record_ext=(L_defect, B_defect, M_struct).

[0160] Step S4100: When a new spatial coupling defect association mapping map of an electronic component is obtained, a new spatial coupling structure feature vector is extracted and input into the spatial coupling pattern recognition network to obtain a new spatial coupling pattern category identifier. The new spatial coupling pattern category identifier is compared with the spatial coupling pattern category identifier in the extended defect recognition record. Based on the comparison result, it is determined whether the new electronic component and the recorded electronic component have the same spatial coupling defect structure type.

[0161] For a new electronic component, extract V_struct_new according to steps S410 to S470, and input it into the spatial coupling pattern recognition network to obtain M_struct_new. Compare M_struct_new with the M_struct in the stored Record_ext: if they are the same, it is determined that the new component and the recorded component have the same spatial coupling defect structure type, and a matching success flag is output; if they are different, a matching failure flag is output.

[0162] For example, the method may further include: Step S510: Obtain the solder joint size deformation amount parameter in the solder joint morphology distortion distribution map of all the welded component body images in the continuously acquired image sequence, arrange the solder joint size deformation amount parameters of the same solder joint index at different acquisition times in chronological order to obtain the size deformation amount time trajectory of a single solder joint, and generate an independent size deformation amount time trajectory curve for each solder joint.

[0163] For solder joint index i from 1 to 16, extract the values of this solder joint at each moment from the solder joint size deformation amount parameters ΔR_i(t) at each moment, and arrange them in chronological order into a time trajectory {ΔR_i(t1), ΔR_i(t2),..., ΔR_i(tT)}. Taking time as the horizontal axis and ΔR_i as the vertical axis, plot the size deformation amount time trajectory curve of each solder joint.

[0164] Step S520: Extract the starting point ordinate value and the ending point ordinate value of the trajectory curve from the size deformation amount time trajectory curve of each solder joint, subtract the starting point ordinate value from the ending point ordinate value to obtain the net change value of the size deformation amount of each solder joint, and divide the solder joints into an expanding solder joint set and a shrinking solder joint set according to the positive or negative sign of the net change value.

[0165] For solder joint i, the starting point ordinate value ΔR_i_start = ΔR_i(t1), and the ending point ordinate value ΔR_i_end = ΔR_i(tT). The net change value ΔNet_i = ΔR_i_end - ΔR_i_start. If ΔNet_i > 0, classify solder joint i into the expanding solder joint set S_expand; if ΔNet_i < 0, classify it into the shrinking solder joint set S_shrink.

[0166] Step S530: Count the total number of solder joints in the expanding solder joint set and the total number of solder joints in the shrinking solder joint set, divide the total number of solder joints in the expanding solder joint set by the total number of solder joints in the shrinking solder joint set to obtain the directional proportion parameter of the solder joint size change, and generate a dominant direction identifier for the size evolution of the solder joint population according to the comparison result between the directional proportion parameter and the preset direction proportion threshold.

[0167] Let |S_expand| = N_exp and |S_shrink| = N_shr. The directional proportion parameter R_dir = N_exp / N_shr. The preset direction proportion threshold R_dir_th = 1.2. If R_dir ≥ 1.2, generate the dominant direction identifier D_dom = "expansion dominant"; if R_dir ≤ 0.8, generate D_dom = "shrinkage dominant"; if 0.8 < R_dir < 1.2, generate D_dom = "balanced evolution".

[0168] Step S540: Extract the slope change sequence of the trajectory curve from the dimensional deformation time trajectory curve of each weld point, count the number of first inflection points where the slope value changes from positive to negative and the number of second inflection points where the slope value changes from negative to positive in the slope change sequence, divide the sum of the number of the first inflection points and the number of the second inflection points by the total time length of the time trajectory curve to obtain the dimensional fluctuation frequency parameter of each weld point.

[0169] For the trajectory curve of solder point i, calculate the slope between adjacent time points: k_i(t) = (ΔR_i(t_{j+1}) - ΔR_i(t_j)) / (t_{j+1} - t_j), to obtain the slope sequence. Scan the slope sequence; when k_i(t_j) > 0 and k_i(t_{j+1}) < 0, it is counted as the first turning point; when k_i(t_j) < 0 and k_i(t_{j+1}) > 0, it is counted as the second turning point. The total number of turning points is N_turn_i = N_pos2neg + N_neg2pos. The dimensional fluctuation frequency parameter f_fluc_i = N_turn_i / (t_T - t_1).

[0170] Step S550: Calculate the average value of the size fluctuation frequency parameters of all solder joints in the expansion type solder joint set to obtain the average fluctuation frequency of the expansion type solder joints. Calculate the average value of the size fluctuation frequency parameters of all solder joints in the contraction type solder joint set to obtain the average fluctuation frequency of the contraction type solder joints. Divide the average fluctuation frequency of the expansion type solder joints by the average fluctuation frequency of the contraction type solder joints to obtain the fluctuation frequency ratio of solder joints in different evolution directions.

[0171] The average fluctuation frequency f_exp_avg of an expanding solder joint is equal to the sum of the dimensional fluctuation frequencies of all expanding solder joints in the set, divided by the number of expanding solder joints, N_exp. The average fluctuation frequency f_shr_avg of a contracting solder joint is equal to the sum of the dimensional fluctuation frequencies of all contracting solder joints in the set, divided by the number of contracting solder joints, N_shr. The fluctuation frequency ratio R_fluc = f_exp_avg / f_shr_avg.

[0172] Step S560: Based on the dominant direction identifier of the solder joint group size evolution, the average fluctuation frequency of the expanding solder joint, and the average fluctuation frequency of the contracting solder joint, construct a group behavior feature vector of the solder joint group size evolution. The group behavior feature vector includes a dominant direction component, an expanding group frequency component, and a contracting group frequency component.

[0173] Construct a three-dimensional group behavior feature vector V_swarm=(D_dom, f_exp_avg, f_shr_avg), where D_dom is a discrete value, and f_exp_avg and f_shr_avg are continuous values.

[0174] Step S570: Input the group behavior feature vector into the pre-trained solder joint group evolution pattern recognition network. The solder joint group evolution pattern recognition network outputs a solder joint group evolution pattern identifier, which includes a cooperative expansion evolution pattern identifier, a cooperative contraction evolution pattern identifier, and a differentiation evolution pattern identifier.

[0175] The solder joint group evolution pattern recognition network uses a random forest classifier, containing 50 decision trees, each with a maximum depth of 10 and a feature sampling number of sqrt(3). The network is trained using 800 sets of group behavior feature vectors. When V_swarm is input into the network, the network outputs the category with the highest probability: Category 1 corresponds to "cooperative expansion evolution pattern identifier", indicating that the size of most solder joints increases synchronously; Category 2 corresponds to "cooperative contraction evolution pattern identifier", indicating that the size of most solder joints decreases synchronously; Category 3 corresponds to "differentiation evolution pattern identifier", indicating that some solder joints expand and some solder joints contract, with inconsistent evolution directions.

[0176] Step S580: Attach the solder joint group evolution mode identifier to the component welding defect identification result to generate a group behavior enhanced welding defect identification result containing defect category labels, defect region boundary coordinates, and solder joint group evolution mode identifier.

[0177] The population evolution pattern identifier M_swarm output in step S570 is appended to the component welding defect identification result output in step S1511 to generate a population behavior enhanced identification result, in the format (L_defect, B_defect, M_swarm).

[0178] Step S610: Obtain two images of the soldered component body at adjacent acquisition times in the continuous acquisition image sequence, and record the two images of the soldered component body at adjacent acquisition times as the preceding time image and the subsequent time image, respectively, and extract the trajectory line of the preceding solder joint position offset from the preceding solder joint morphology distortion distribution map of the preceding time image.

[0179] Select adjacent frames t and t+1 from the continuously acquired image sequence, showing the soldered component body. Frame t is the previous time-phase image, and frame t+1 is the subsequent time-phase image. Process the previous time-phase image according to steps S121 to S128, and extract the trajectory line L_prev of the previous solder joint position offset.

[0180] Step S620: Extract the subsequent solder joint position offset trajectory line from the subsequent solder joint morphology distortion distribution map from the subsequent time image, and superimpose the preceding solder joint position offset trajectory line and the subsequent solder joint position offset trajectory line into the same spatial coordinate system to obtain a trajectory line superimposed comparison map. The trajectory line superimposed comparison map contains the spatial position relationship between the preceding trajectory line segment and the subsequent trajectory line segment.

[0181] Process subsequent time-stamped images using the same steps to extract the subsequent solder joint position offset trajectory line L_next. Overlay L_prev and L_next onto the printed circuit board physical coordinate system to obtain a trajectory line overlay comparison image, which simultaneously displays the preceding and subsequent trajectory lines.

[0182] Step S630: Calculate the vertical distance between each preceding trajectory line segment and the subsequent trajectory line segment at the corresponding spatial position in the trajectory line superimposed comparison image to obtain the trajectory line spatial offset distance sequence, and generate the spatial drift amplitude parameter of the solder joint position offset trajectory line based on the maximum offset distance in the trajectory line spatial offset distance sequence.

[0183] For each solder joint index, the point set on the preceding trajectory segment is L_prev_i, and the point set on the subsequent trajectory segment is L_next_i. For each point on L_prev_i, the shortest vertical distance to L_next_i is calculated, and the average of all distances is taken as the spatial offset distance D_offset_i for that solder joint trajectory segment. This results in the sequence {D_offset_1, D_offset_2, ..., D_offset_16}. The spatial drift amplitude parameter D_drift_max = max(D_offset_i).

[0184] Step S640: Calculate the angle difference between the tangent direction angle of each preceding trajectory line segment and the tangent direction angle of the subsequent trajectory line segment at the corresponding spatial position in the trajectory line superimposed comparison image, obtain the trajectory line direction change angle sequence, and generate the direction twisting degree parameter of the solder joint position offset trajectory line based on the average angle difference in the trajectory line direction change angle sequence.

[0185] For each solder joint index, calculate the tangent angle α_prev_i of the preceding trajectory segment at the geometric center point, and calculate the tangent angle α_next_i of the subsequent trajectory segment at the corresponding spatial position. The angle difference Δα_i = min(|α_next_i - α_prev_i|, 2π - |α_next_i - α_prev_i|). This yields the sequence {Δα_1, Δα_2, ..., Δα_16}. The directional twist parameter α_twist_avg = (1 / 16) * ΣΔα_i.

[0186] Step S650: Arrange the offset distances in the trajectory line spatial offset distance sequence into a spatial offset distribution curve according to their spatial position, calculate the area under the curve of the spatial offset distribution curve to obtain the cumulative offset of the trajectory line spatial offset, and obtain the average offset gradient of the trajectory line spatial offset based on the ratio of the cumulative offset to the length of the trajectory line spatial offset distance sequence.

[0187] Arrange D_offset_i in solder joint index order, and plot the spatial offset distribution curve with solder joint index as the horizontal axis and D_offset_i as the vertical axis. The area under the curve, AUC_offset, can be approximated using the trapezoidal rule, for example: AUC_offset≈Σ{i=1}^{15}((D_offset_i+D_offset{i+1})×0.5).

[0188] Step S660: Based on the spatial drift amplitude parameter of the solder joint position offset trajectory line, the directional twisting degree parameter of the solder joint position offset trajectory line, and the average offset gradient of the trajectory line spatial offset, construct the inter-frame evolution feature vector of the solder joint trajectory line. The inter-frame evolution feature vector includes spatial drift amplitude component, directional twisting degree component, and average offset gradient component.

[0189] Construct a three-dimensional inter-frame evolution feature vector V_frame=(D_drift_max, α_twist_avg, G_offset_avg).

[0190] Step S670: Input the inter-frame evolution feature vector into the pre-trained solder joint trajectory evolution pattern recognition model. The solder joint trajectory evolution pattern recognition model outputs the inter-frame evolution pattern identifier of the solder joint trajectory. The inter-frame evolution pattern identifier of the solder joint trajectory includes a unidirectional drift evolution pattern identifier, a reciprocating swing evolution pattern identifier, and a random walk evolution pattern identifier.

[0191] The weld joint trajectory evolution pattern recognition model employs a Long Short-Term Memory (LSTM) network. The network structure includes: 3 nodes in the input layer, 32 nodes in the hidden layer using LSM units, and 3 nodes in the output layer using the Softmax activation function. The model is trained using 1000 sets of time-series data with a sequence length of 20, using the Adam optimizer and a learning rate of 0.001. When a V_frame is input into the model, it outputs three probabilities, with the maximum value corresponding to the category: Category 1 is a "unidirectional drift evolution pattern identifier," indicating that the trajectory continuously moves in the same direction; Category 2 is a "reciprocating oscillation evolution pattern identifier," indicating that the trajectory oscillates back and forth; and Category 3 is a "random walk evolution pattern identifier," indicating that the trajectory moves in an irregular direction.

[0192] Step S680: Associate and store the inter-frame evolution mode identifier of the solder joint trajectory with the defect category label in the component welding defect identification result to generate a trajectory evolution enhanced welding defect identification result containing the defect category label and the inter-frame evolution mode identifier.

[0193] The inter-frame evolution mode identifier M_frame is associated with the defect category label L_defect output in step S1511 and stored to generate a trajectory evolution enhanced recognition result in the format (L_defect, B_defect, M_frame).

[0194] Step S710: Extract the region enclosed by the highlighted closed curve of the target defect overlapping region from the spatial coupling defect association mapping map, perform boundary segmentation operation on the region enclosed by the highlighted closed curve, divide the highlighted closed curve into multiple boundary arc segments according to the curvature change points, and obtain the set of boundary arc segments of the overlapping region.

[0195] The highlighted closed curves of the overlapping regions of each target defect are extracted from the spatial coupling defect association map. For each closed curve, the curvature κ = |x'y''-y'x''| / (x'^2+y'^2)^{3 / 2} at each point on the curve is calculated. Points with drastic curvature changes are identified as segmentation points, with a curvature change threshold Δκ_th = 0.1. The closed curves are cut at the segmentation points to obtain multiple boundary arc segments, forming a set of boundary arc segments of the overlapping region.

[0196] Step S720: Calculate the arc length and arc radius of curvature of each boundary arc segment in the set of boundary arc segments of the overlapping region, divide the arc length by the arc radius of curvature to obtain the curvature parameter of each boundary arc segment, and divide the boundary arc segments into a straight arc segment subset and a curved arc segment subset according to the comparison result of the curvature parameter and the preset curvature threshold.

[0197] For each boundary arc segment, the arc length L_arc is equal to the sum of the distances between adjacent points on the arc segment. The arc radius of curvature R_curv = 1 / avg(κ). The curvature parameter B_degree = L_arc / R_curv. The preset curvature threshold B_th = 1.5. If B_degree < 1.5, the arc segment is assigned to the straight arc segment subset S_straight; if B_degree ≥ 1.5, it is assigned to the curved arc segment subset S_curved.

[0198] Step S730: Count the total number of boundary arc segments in the straight arc segment subset and the total number of boundary arc segments in the curved arc segment subset, and divide the total number of boundary arc segments in the curved arc segment subset by the total number of boundary arc segments in the straight arc segment subset to obtain the curvature ratio parameter of the boundary arc segments in the overlapping area.

[0199] Let |S_curved|=N_curved, |S_straight|=N_straight. The curvature ratio parameter R_curve_ratio=N_curved / N_straight.

[0200] Step S740: Extract the radius of curvature of the boundary arc segments in all curved arc segment subsets from the set of boundary arc segments of the overlapping region, calculate the average value of the radius of curvature of the boundary arc segments in all curved arc segment subsets to obtain the average radius of curvature parameter of the curved arc segments, and calculate the standard deviation of the radius of curvature of the boundary arc segments in all curved arc segment subsets to obtain the radius of curvature dispersion parameter of the curved arc segments.

[0201] Extract the radii of curvature of all arc segments from S_curved, denoted as R_curv_1, R_curv_2, ..., R_curv_N_curved. The average radius of curvature R_curv_avg is equal to the sum of these radii of curvature divided by N_curved. The standard deviation of the radius of curvature R_curv_std = sqrt([(R_curv_1-R_curv_avg)^2+...+(R_curv_N_curved-R_curv_avg)^2] / N_curved).

[0202] Step S750: Extract the spatial distribution coordinates of all directional coupling lines in the set of directional coupling lines from the spatial coupling defect association mapping map, calculate the distance between the straight line containing each directional coupling line and the nearest boundary arc of the overlapping area of ​​the target defect, and obtain the distance sequence from the directional coupling line to the boundary of the overlapping area.

[0203] For each directional coupling connection line L_i, calculate its equation. For each boundary arc segment of the overlapping region of each target defect, calculate the shortest distance from the line to the arc segment (the minimum distance from a point to a line segment). Take the minimum distance among all arc segments as the distance d_i from the connection line to the boundary of the overlapping region. This yields the distance sequence {d_1, d_2, ..., d_{C_dir_line}}.

[0204] Step S760: Based on the bending ratio parameter of the boundary arc segment of the overlapping region, the average radius of curvature parameter of the bending arc segment, the radius of curvature dispersion parameter of the bending arc segment, and the average value of the distance sequence from the directional coupling connection line to the boundary of the overlapping region, construct the boundary morphology feature vector of the overlapping region. The boundary morphology feature vector includes a bending ratio component, an average radius of curvature component, a radius of curvature dispersion component, and a distance component from the coupling line to the boundary.

[0205] Calculate the average distance d_avg from the directional coupling connection line to the boundary of the overlapping region, i.e., d_avg=(d_1+d_2+...+d_C_dir_line) / C_dir_line. Construct a four-dimensional boundary morphology feature vector V_boundary=(R_curve_ratio, R_curv_avg, R_curv_std, d_avg).

[0206] Step S770: Input the feature vector of the boundary morphology of the overlapping region into the pre-trained welding defect boundary morphology classifier. The welding defect boundary morphology classifier outputs a boundary morphology category identifier, which includes a smooth arc boundary identifier, a sawtooth boundary identifier, and a multi-arc spliced ​​boundary identifier.

[0207] The welding defect boundary morphology classifier employs a gradient boosting decision tree classifier, containing 100 regression trees with a learning rate of 0.1 and a maximum depth of 5. The classifier is trained using 600 sets of boundary morphology feature vectors. Inputting V_boundary into the classifier yields the following output categories: Category 1 is "Smooth Circular Arc Boundary Marker," indicating a boundary with a single smooth circular arc shape; Category 2 is "Jagged Boundary Marker," indicating a boundary with sharply jagged undulations; and Category 3 is "Multi-Arc Segmented Boundary Marker," indicating a boundary composed of multiple arc segments with different curvatures.

[0208] Step S780: Use the boundary morphology category identifier as the boundary attribute information of the component welding defect identification result, and generate a boundary morphology enhanced component welding defect identification result containing defect category label, defect area boundary coordinates and boundary morphology category identifier.

[0209] The boundary shape category identifier M_boundary output in step S770 is appended to the component welding defect identification result output in step S1511 to generate an enhanced boundary shape identification result, in the format (L_defect, B_defect, M_boundary).

[0210] In one exemplary embodiment, a deep learning-based production anomaly detection system is provided, which may be a terminal, server, etc., and its internal structure diagram may be as follows. Figure 2As shown, the system includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, near-field communication, or other technologies. When the computer program is executed by the processor, it implements a deep learning-based method for detecting and identifying production anomalies. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, or a button, trackball, or touchpad set on the shell of a deep learning-based production inspection anomaly recognition system, or an external keyboard, touchpad, or mouse, etc.

[0211] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A production anomaly detection method based on deep learning, characterized in that, The method includes: The continuous acquisition image sequence output from the online inspection station of the electronic component production line is obtained. The continuous acquisition image sequence includes images of the soldered component body and images of the solder joint area around the soldered component. An abnormal solder joint shape is captured by performing a solder joint shape orientation capture operation on the image of the soldered component body, and a solder joint shape distortion distribution map of the soldered component body is generated. The solder joint shape distortion distribution map is marked with the solder joint position offset trajectory and the solder joint size change gradient. A wetting and spreading behavior quantification operation is performed on the image of the solder joint area around the soldered components to generate an abnormal fluctuation map of the wetting and spreading contour. The abnormal fluctuation map of the wetting and spreading contour includes a marker of the location where the spreading contour diffusion is blocked and a marker of the direction of asymmetric extension of the spreading contour. By integrating the solder joint morphology distortion distribution map and the wetting spread profile abnormal fluctuation map, a spatial coupling correlation model of component welding defects is constructed, and a spatial coupling defect correlation mapping map is output. The spatial coupling defect correlation mapping map records the coordinate set of the spatially overlapping area between solder joint morphology distortion and wetting spread profile abnormality. The welding defect type decision tree is invoked to perform defect category determination on the spatially coupled defect association mapping, generating component welding defect identification results containing defect category labels and defect region boundary coordinates.

2. The production inspection anomaly identification method based on deep learning according to claim 1, characterized in that, The step of performing solder joint morphology anomaly directional capture operation on the image of the soldered component body to generate a solder joint morphology distortion distribution map of the soldered component body includes: From the image of the soldered component body, the local image blocks where each solder joint is located are segmented to obtain a set of local image blocks of solder joints. Each local image block of solder joints in the set of local image blocks of solder joints contains the complete outline information of a single solder joint. For each local image block in the set of local image blocks of solder joints, solder joint contour edge tracking processing is performed to obtain a sequence of solder joint contour edge coordinates. The sequence of solder joint contour edge coordinates is arranged in the order of connection of edge points to form a closed contour curve. Calculate the coordinates of the geometric center point of the weld point contour edge coordinate sequence, and compare the coordinates of the geometric center point with the preset standard weld point position coordinates by spatial offset vector to obtain a set of weld point position offset vectors. The set of weld point position offset vectors includes the offset direction angle and the offset distance length. Extract the maximum inscribed circle radius and minimum circumscribed circle radius of the solder joint contour from each local image block in the set of local image blocks of the solder joint, calculate the radial difference between the maximum inscribed circle radius and the minimum circumscribed circle radius, and generate solder joint size deformation parameters. Based on the set of solder joint position offset vectors and the solder joint size deformation parameters, a trajectory line and gradient sequence are constructed and superimposed with the identified distortion candidate regions to generate a solder joint morphology distortion distribution map of the soldered component body.

3. The production inspection anomaly identification method based on deep learning according to claim 1, characterized in that, The step of performing a wetting and spreading behavior quantification operation on the image of the solder joint area around the soldered components to generate an abnormal fluctuation map of the wetting and spreading contour includes: Extract the pixel connected regions of the solder spreading area from the image of the solder joint area around the soldered components to obtain a solder spreading connected region binary mask. In the solder spreading connected region binary mask, the pixels with a value of the first value constitute the solder covered area and the pixels with a value of the second value constitute the non-solder covered area. A boundary tracing algorithm is performed on the solder coverage area in the solder spread connected binary mask to generate a set of spread contour boundary points. The set of spread contour boundary points is arranged in the order of traversal of the boundary points to form a closed spread contour loop. Calculate the radial distance of each boundary point in the set of boundary points of the paving profile relative to the geometric centroid of the paving profile ring to obtain a radial distance distribution sequence, which reflects the extension length of the paving profile in different angular directions; Local minimum points of radial distance are identified from the radial distance distribution sequence, and the boundary points corresponding to the local minimum points are marked as locations where the spread profile diffusion is hindered. These locations indicate that the solder spread in the corresponding direction is obstructed and fails to extend sufficiently. Additionally, local maximum points of radial distance are identified from the radial distance distribution sequence, and the boundary points corresponding to the local maximum points are marked as locations where the spread profile is abnormally convex. These locations indicate that the solder exhibits abnormal overspreading in that direction. Based on the locations where the spreading profile diffusion is obstructed and the locations where the spreading profile is abnormally convex, the fluctuation amplitude parameters and the asymmetric extension direction are calculated, and image annotations are drawn to generate an abnormal fluctuation map of the wetting spreading profile.

4. The production detection anomaly identification method based on deep learning according to claim 1, characterized in that, The method integrates the solder joint morphology distortion distribution map and the wetting spread contour abnormal fluctuation map to construct a spatial coupling correlation model of component welding defects, and outputs a spatial coupling defect correlation mapping map, including: The solder joint position offset trajectory line in the solder joint morphology distortion distribution map and the extension direction ray in the wetting and spreading profile abnormal fluctuation map are projected onto the same spatial coordinate system to obtain a spatial direction vector superposition map, which includes the offset trajectory direction vector and the extension direction ray vector. Calculate the spatial distance between the inflection point of the offset trajectory of each solder joint in the solder joint morphology distortion distribution map and the abnormal protrusion position of each spreading profile in the abnormal fluctuation map of the wetting spreading profile, and pair the inflection point with the protrusion position whose spatial distance is less than a preset distance threshold with the protrusion position, and record them as candidate associated feature pairs. Count the number of candidate associated feature pairs to obtain the number of directional associated feature pairs. The boundary coordinates of the spatially overlapping region of morphological and dimensional distortion in the solder joint morphological distortion distribution map are spatially intersected with the contour coordinates of the violently fluctuating contour segment in the abnormal fluctuation map of the wetting and spreading contour to obtain a set of polygons of spatially overlapping region. The set of polygons of spatially overlapping region records the spatial range in which the solder joint morphological distortion region and the spreading contour fluctuation region overlap. The spatial distance between the center point coordinates of the candidate region of size distortion in the solder joint morphology distortion distribution map and the coordinates of the obstructed diffusion position of the spreading profile in the abnormal fluctuation map of the wetting spreading profile is calculated to obtain the set of proximity distance parameters between the obstructed position and the size distortion region. The set of proximity distance parameters records the spatial interval distance between each obstructed spreading profile position and the nearest candidate region of size distortion. From the candidate associated feature pairs, extract a preset number of feature points with the smallest spatial distance, or record all feature point pairs with a distance less than a preset threshold as directional coupling feature pairs. The directional coupling feature pairs characterize the spatial proximity relationship between solder joint position offset and abnormal solder spread. Based on the set of polygons in the spatially overlapping region, the set of neighboring distance parameters, and the directional coupling feature pairs, the defect coupling region and the neighboring coupled defect pairs are extracted, and spatial coupling connection annotations are drawn.

5. The production inspection anomaly identification method based on deep learning according to claim 4, characterized in that, The step of calling the welding defect type decision tree to perform defect category determination on the spatially coupled defect association mapping map generates component welding defect identification results containing defect category labels and defect region boundary coordinates, including: The total number of directional coupling lines in the set of directional coupling lines is extracted from the spatial coupling defect association mapping to obtain the directional coupling line quantity parameter. The total area of ​​the region enclosed by the highlighted closed curve of the target defect overlapping region is extracted from the spatial coupling defect association map to obtain the target overlapping region area parameter, which is equal to the cumulative sum of the surface areas of all target defect overlapping regions. The total number of neighboring coupling lines in the set of neighboring coupling lines is extracted from the spatial coupling defect association mapping to obtain the neighboring coupling line number parameter. The number of directional coupling connection lines is compared with the preset directional coupling number threshold for the first time, and the first-level branch path of the welding defect type decision tree is selected based on the result of the first numerical comparison. The first-level branch path includes paths where the number of directional coupling exceeds the standard and paths where the number of directional coupling does not exceed the standard. Based on the results of numerical comparisons at each layer, the parameter combinations are input into the corresponding leaf nodes to obtain defect category labels and output the recognition results.

6. The production inspection anomaly identification method based on deep learning according to claim 4, characterized in that, The method further includes: The total number of all directional coupling lines in the directional coupling connection line set is extracted from the spatial coupling defect association mapping to obtain the directional coupling connection line number parameter. The directional coupling connection line number parameter is compared with the preset directional coupling number threshold. Based on the comparison result, it is determined whether there is a spatial directional coupling anomaly between the solder joint offset direction and the solder spreading direction, and a directional coupling anomaly determination identifier is generated. Extract the area enclosed by the highlighted closed curve of the target defect overlapping region from the spatial coupling defect association mapping map, sum the areas enclosed by the highlighted closed curve of the target defect overlapping region to obtain the total area parameter of the target defect overlapping region, and compare the total area parameter with the preset overlapping region area threshold. Based on the comparison result, determine whether there is spatial overlap coupling anomaly between the solder joint morphology distortion region and the spread contour fluctuation region, and generate an overlap coupling anomaly determination mark. The total number of all adjacent coupling lines in the adjacent coupling connection line set is extracted from the spatial coupling defect association mapping to obtain the adjacent coupling connection line number parameter. The adjacent coupling connection line number parameter is compared with the preset adjacent coupling number threshold. Based on the comparison result, it is determined whether there is a spatial adjacent coupling anomaly between the solder joint size distortion area and the spread profile diffusion obstruction position, and an adjacent coupling anomaly determination mark is generated. The directional coupling anomaly determination identifier, the overlapping coupling anomaly determination identifier, and the adjacent coupling anomaly determination identifier are input into the welding defect type determination matrix, and the defect type code is output based on the combined state and used as the identification result.

7. The production inspection anomaly identification method based on deep learning according to claim 1, characterized in that, The method further includes: The image of the soldered component body at the first acquisition time in the continuous acquisition image sequence is used as a reference image, and the image of the soldered component body at the second acquisition time in the continuous acquisition image sequence is used as a comparison image. The second acquisition time is later than the first acquisition time in time sequence. The reference image and the comparison image contain the surface state records of the same electronic component at different soldering stages. A feature extraction operation is performed on the solder joint position offset trajectory line in the solder joint morphology distortion distribution map of the reference image to obtain the inflection point coordinate sequence of the reference solder joint position offset trajectory line and the boundary coordinate set of the offset direction change region. The inflection point coordinate sequence of the reference solder joint position offset trajectory line and the boundary coordinate set of the offset direction change region are recorded as the reference morphology distortion feature set. A feature extraction operation is performed on the solder joint position offset trajectory line in the solder joint morphology distortion distribution map of the comparison image to obtain the inflection point coordinate sequence of the comparison solder joint position offset trajectory line and the boundary coordinate set of the offset direction change region. The inflection point coordinate sequence of the comparison solder joint position offset trajectory line and the boundary coordinate set of the offset direction change region are recorded as the comparison morphology distortion feature set. After spatially aligning the reference image with the comparison image, the inflection point coordinate sequence of the reference solder joint position offset trajectory line in the reference morphological distortion feature set and the inflection point coordinate sequence of the comparison solder joint position offset trajectory line in the comparison morphological distortion feature set are matched based on spatial proximity. The spatial displacement vector between each set of matched inflection point coordinates is calculated to obtain the inflection point spatial displacement vector set. The inflection point spatial displacement vector set records the position movement direction and movement distance of each inflection point from the first acquisition time to the second acquisition time. The boundary coordinate set of the offset direction abrupt region in the reference morphological distortion feature set and the boundary coordinate set of the offset direction abrupt region in the comparison morphological distortion feature set are used to calculate the region overlap, obtain the overlap parameter of the offset direction abrupt region, and determine the change trend of the offset direction abrupt region in the welding process based on the overlap parameter, and generate the evolution direction identifier of the abrupt region. Based on the set of spatial displacement vectors at the inflection points and the evolution direction identifiers of the mutation regions, stability parameters and evolution rates are calculated, and trend vectors are constructed to predict evolution trends.

8. The production inspection anomaly identification method based on deep learning according to claim 1, characterized in that, The method further includes: The solder joint morphology distortion distribution map of all soldered component body images in the continuously acquired image sequence is obtained. The solder joint position offset trajectory lines in all solder joint morphology distortion distribution maps are stacked and arranged in the order of acquisition time to obtain a three-dimensional spatiotemporal stack of solder joint position offset trajectory lines. The three-dimensional spatiotemporal stack is composed of a three-dimensional coordinate system with spatial horizontal coordinate, spatial vertical coordinate and acquisition time coordinate. In the three-dimensional spatiotemporal stack of the solder joint position offset trajectory line, for the spatial position corresponding to each solder joint index, the offset direction angle at each acquisition time is extracted to form a sequence of the offset direction angle of each solder joint index changing with time, and the offset direction angle time series of each solder joint index is obtained. The offset direction angle time series records the offset direction angle of the solder joint index at different acquisition times. Calculate the first time derivative of the time function of the offset direction angle of each spatial location point as a function of time to obtain the time distribution sequence of the offset direction angle change rate, and generate the offset direction oscillation frequency parameter based on the alternation frequency of positive and negative values ​​in the time distribution sequence of the offset direction angle change rate. The coordinate distribution of the inflection points of the solder joint position offset trajectory line in three-dimensional space is extracted from the three-dimensional spatiotemporal stack of the solder joint position offset trajectory line. The three-dimensional coordinates of each inflection point are projected onto the acquisition time coordinate axis to obtain a discrete distribution set of inflection point occurrence time points. Based on the time interval between adjacent inflection point occurrence time points in the discrete distribution set, an inflection point time interval sequence is generated. The average inflection point occurrence frequency is calculated based on the inflection point time interval sequence. The offset direction swing frequency parameter is compared with the average inflection point occurrence frequency, and their ratio or difference is calculated to obtain the parameter of the closeness between the swing frequency and the inflection point occurrence frequency. Based on the proximity parameter between the swing frequency and the inflection point occurrence frequency, a dynamic evolution feature triplet is constructed and input into the classification model to obtain the dynamic type identifier.

9. A production inspection anomaly identification system based on deep learning, characterized in that, include: processor; A machine-readable storage medium for storing machine-executable instructions of the processor; The processor is configured to execute the deep learning-based production detection anomaly identification method according to any one of claims 1 to 8 by executing the machine-executable instructions.

10. A machine-readable storage medium, characterized in that, Used to store the processor's machine-executable instructions; The processor is configured to execute the deep learning-based production detection anomaly identification method according to any one of claims 1 to 8 by executing the machine-executable instructions.

Images