Circuit board defect detection method and system based on deep learning

Abstract

The invention provides a circuit board defect detection method and system based on deep learning, and the method comprises the steps: firstly obtaining a layered image set of a to-be-detected circuit board, which comprises a welding spot imaging layer, a line imaging layer and an element imaging layer; calling a pre-trained circuit board defect hierarchical detection model, inputting the hierarchical image set into a model defect pre-association layer, and generating a hierarchical defect association map which marks the position mapping relation of the same defect in different imaging layers; on the basis of the layered defect correlation map, extracting defect area exclusive features of each imaging layer to form a layered exclusive defect feature set, and inputting a model hierarchical feature enhancement layer to generate a hierarchical enhanced defect feature set. And finally, calling the model scene adaptive classification layer to carry out defect type matching, and generating a circuit board defect detection report containing defect types, hierarchical positions and association relationships in combination with the position mapping relationship, so that the circuit board defects can be comprehensively and accurately detected, and the detection efficiency and quality are improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a method and system for detecting circuit board defects based on deep learning. Background Technology

[0002] In the circuit board manufacturing process, defect detection is a crucial step in ensuring product quality. Traditional circuit board defect detection methods mainly rely on manual visual inspection and simple image processing techniques. Manual visual inspection has many limitations. On the one hand, due to human visual fatigue and subjective judgment differences, missed or false detections are prone to occur, especially when facing large-scale, high-precision circuit board production, where the efficiency and accuracy of manual inspection are difficult to guarantee. On the other hand, manual inspection cannot effectively identify some small, hidden defects, such as cold solder joints or minor breaks in the circuit.

[0003] While detection methods based on simple image processing techniques improve efficiency to some extent, they typically only detect defects on a single layer of the circuit board or specific types of defects. For example, some methods focus solely on the appearance of solder joints, using fixed thresholds to determine the presence of defects, while ignoring the relationship between solder joints and surrounding circuitry and components. For circuit defect detection, they simply check the continuity of the circuitry, failing to accurately identify potential problems reflected in variations in circuit texture. Furthermore, existing detection methods lack the analysis and utilization of spatial correspondences between defects in different imaging layers, making it difficult to comprehensively and accurately locate and identify various defects in complex circuit boards, thus failing to meet the demands of modern circuit board production for high-quality, high-efficiency inspection. Summary of the Invention

[0004] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a circuit board defect detection method based on deep learning, the method comprising: A layered image set of the circuit board to be inspected is obtained. The layered image set includes a solder joint imaging layer, a circuit imaging layer and a component imaging layer. Each imaging layer corresponds to an image unit of a different structural area of ​​the circuit board, and contains image units of all solder joints, conductive lines and surface mount components. A pre-trained circuit board defect hierarchical detection model is invoked, and the hierarchical image set is input into the model defect pre-association layer. The spatial correspondence between defect regions in each imaging layer is established through the intra-layer structure association module, and a hierarchical defect association map is generated. The hierarchical defect association map marks the positional mapping relationship of the same defect in different imaging layers. Based on the layered defect association map, the specific features of the defect region in each imaging layer are extracted. Solder joint morphology association parameters are extracted in the solder joint imaging layer, line texture continuity parameters are extracted in the line imaging layer, and component pin offset association parameters are extracted in the component imaging layer, forming a set of layered specific defect features. The layered specific defect feature set is input into the model layer-level feature enhancement layer. Through the layer's built-in defect feature attention module, each parameter in the layered specific defect feature set is subjected to targeted enhancement processing to generate a layered enhanced defect feature set. The model scene adaptation classification layer is invoked to perform circuit board-specific defect type matching on the hierarchical enhanced defect feature set, outputting solder joint, circuit and component defect results. Combined with the position mapping relationship of the hierarchical defect association map, a circuit board defect detection report containing defect type, hierarchical position and association relationship is generated.

[0005] In another aspect, embodiments of the present invention also provide a circuit board defect detection system based on deep learning, including a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0006] Based on the above, this embodiment of the invention acquires a layered image set of the circuit board to be inspected, comprehensively covering image information of different structural regions of the circuit board. It then calls a pre-trained hierarchical defect detection model for the circuit board and utilizes an intra-layer structural association module to establish spatial correspondences between defect regions in each imaging layer, generating a layered defect association map. This map can label the positional mapping relationships of the same defect in different imaging layers, effectively solving the problem of traditional methods struggling to handle complex spatial correlations of defects. Based on the layered defect association map, it extracts specific features of defect regions in each imaging layer, forming a layered specific defect feature set. This specifically captures defect features in different structural regions, improving the accuracy and effectiveness of feature extraction. The layered specific defect feature set is input into the model's hierarchical feature enhancement layer. Through the layer's built-in defect feature attention module, each parameter is specifically enhanced, generating a layered enhanced defect feature set. This further highlights key defect features and enhances the model's defect recognition ability. Finally, the model's scene adaptation classification layer is called to match circuit board-specific defect types. Combined with the positional mapping relationships of the layered defect association map, a circuit board defect detection report containing defect type, hierarchical position, and association relationships is generated. This achieves comprehensive and accurate detection of circuit board defects, helping to improve detection efficiency and reduce false negative and false positive rates. Attached Figure Description

[0007] Figure 1This is a schematic diagram of the execution flow of the circuit board defect detection method based on deep learning provided in an embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of exemplary hardware and software components of a deep learning-based circuit board defect detection system provided in an embodiment of the present invention. Detailed Implementation

[0009] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating a deep learning-based circuit board defect detection method according to an embodiment of the present invention. The following is a detailed description of the deep learning-based circuit board defect detection method.

[0010] Step S110: Obtain a layered image set of the circuit board to be inspected. The layered image set includes a solder joint imaging layer, a circuit imaging layer, and a component imaging layer. Each imaging layer corresponds to an image unit of a different structural area of ​​the circuit board, and includes image units of all solder joints, conductive lines, and surface mount components.

[0011] In this embodiment, the object to be inspected is a printed circuit board used in automated control equipment. This printed circuit board has numerous solder joints, a complex network of conductive lines, and various types of surface-mount components. The integrity of these structures directly affects the normal operation of the circuit board. To accurately acquire image information of each structural area, a layered image acquisition system consisting of a high-definition industrial camera, an adjustable light source array, a precision stage, and an image acquisition card is used for layered image acquisition.

[0012] During the acquisition process, the circuit board to be inspected is first fixed on a precision stage to ensure that it does not shift during acquisition. For the acquisition of the solder joint imaging layer, the lens focal length and aperture parameters of the high-definition industrial camera are adjusted to enable the camera to clearly capture the morphological details of the solder joints, including their outlines and surface gloss. Simultaneously, the light source array is controlled to illuminate the solder joint area at an oblique angle, highlighting the boundary between the solder joint and its surrounding area through differences in light reflection. The acquired solder joint imaging layer consists of multiple image units, each corresponding to an independent solder joint on the circuit board. These image units are arranged according to the actual distribution of the solder joints on the circuit board.

[0013] For the circuit imaging layer, the light source was adjusted to a vertical illumination mode to reduce the impact of reflections from the conductive circuit surface on image quality. Camera parameter settings focused on enhancing the contrast between the circuit and the substrate, making the edges of the conductive circuits more clearly discernible. The image units of the circuit imaging layer were divided according to the direction and segmentation of the conductive circuits, with each image unit corresponding to a continuous segment of the conductive circuit, ensuring that the texture features and continuity of the circuit could be fully presented.

[0014] The acquisition of the component imaging layer needs to capture both the component body and its pins. By adjusting the camera's shooting angle and focal length, the outline of the component body, the shape of the pins, and the connection points between the pins and the circuitry can all be clearly imaged. The light source adopts a ring-shaped distribution pattern, illuminating the component from multiple directions to avoid shadows caused by the component body obstructing the image and affecting image quality. The image unit of the component imaging layer is based on each surface-mount component, containing image information of the component body and all its pins.

[0015] After acquisition, the image acquisition card performs preliminary processing on the raw image data, including removing noise points and correcting geometric distortions. The processed solder joint imaging layer, circuit imaging layer, and component imaging layer are stored sequentially to form a layered image set of the circuit board under inspection.

[0016] Step S120: Call the pre-trained circuit board defect hierarchical detection model, input the hierarchical image set into the model defect pre-association layer, establish the spatial correspondence between defect regions in each imaging layer through the intra-layer structure association module, and generate a hierarchical defect association map. The hierarchical defect association map marks the positional mapping relationship of the same defect in different imaging layers.

[0017] In this embodiment, the pre-trained circuit board defect hierarchical detection model is built on a convolutional neural network architecture. It is trained using a large amount of circuit board layered image data labeled with defect type and location information, and can effectively identify and associate defect regions in different imaging layers. During defect detection, the layered image set obtained in step S110 is first converted into a digital image format that the model can recognize, and then input into the model's defect pre-association layer.

[0018] The structural association module within the defect pre-association layer is the core component for establishing spatial correspondences between imaging layers. This module includes sub-modules such as layer adaptation units and spatial mapping units. Through the collaborative work of these sub-modules, a series of operations are completed, from image coordinate calibration to association map generation. The generation process of the layered defect association map not only considers the location information of the defect region in each imaging layer but also incorporates the physical structural characteristics of the circuit board, ensuring that the map accurately reflects the positional mapping relationship of the same defect in different imaging layers, laying the foundation for subsequent defect feature extraction and classification.

[0019] Step S121: Input the solder joint imaging layer, circuit imaging layer and component imaging layer in the layered image set into the layer adaptation unit of the structure association module respectively, and perform coordinate calibration on the image units of each imaging layer to keep the image units of different imaging layers aligned, so as to obtain the coordinate aligned imaging layer.

[0020] In this embodiment, the layer adaptation unit of the structure association module adopts a coordinate calibration method based on feature point matching. First, a series of stable feature points are extracted from the solder joint imaging layer, the circuit imaging layer, and the component imaging layer, respectively. These feature points can be easily identifiable structures such as positioning holes on the circuit board, pads of specific shapes, and corners of components.

[0021] For the solder joint imaging layer, the center position of each solder joint and some fixed substrate markers around it are extracted as feature points. In the circuit imaging layer, the intersections and turning points of conductive lines are selected as feature points. The component imaging layer uses the corners of components and the endpoints of pins as feature points. After extracting the feature points, the layer adaptation unit matches these feature points to find the feature points at the same physical location in different imaging layers.

[0022] Based on the matched feature point pairs, a coordinate transformation matrix is ​​calculated between each imaging layer. This transformation matrix describes the geometric transformation relationships between different imaging layers, including translation, rotation, and scaling. Image units in the circuit imaging layer and component imaging layer are transformed using this coordinate transformation matrix to ensure they are consistent with the image units in the solder joint imaging layer in the coordinate system. After coordinate calibration, image units corresponding to the same physical location in different imaging layers can be accurately aligned, resulting in coordinate-aligned imaging layers. This ensures that subsequent spatial correlation analysis is performed under a unified coordinate system.

[0023] Step S122: Locate the solder joint area in the solder joint imaging layer of the coordinate alignment imaging layer, extract the center coordinates and contour range of all solder joint image units, and form a solder joint coordinate set; locate the circuit area in the circuit imaging layer, extract the center line coordinates and width parameters of all conductive circuit image units, and form a circuit coordinate set; locate the component area in the component imaging layer, extract the pin coordinates and component body contour of all surface mount component image units, and form a component coordinate set.

[0024] In this embodiment, after obtaining the coordinate-aligned imaging layer, region localization and parameter extraction are performed on each imaging layer. For the solder joint imaging layer, a method based on threshold segmentation and morphological operations is used to locate the solder joint region. First, based on the difference in grayscale values ​​between the solder joint and the substrate, an appropriate grayscale threshold is set to segment the solder joint imaging layer, resulting in a binarized image of the solder joint region. Then, morphological dilation and erosion operations are used to remove small noise regions from the segmented image, making the contour of the solder joint region more complete.

[0025] In the processed binarized image, contour extraction is performed on each independent solder joint region to determine the contour range of each solder joint image unit. The center coordinates of each solder joint are obtained by calculating the geometric center of the contour. After organizing the center coordinates and corresponding contour range information of all solder joints, a set of solder joint coordinates is formed.

[0026] In the circuit imaging layer, edge detection and line extraction algorithms are used to locate the conductive circuit region. First, an edge detection operator is applied to the circuit imaging layer to obtain the edge image of the conductive circuit. Then, the centerline of the conductive circuit is extracted based on the edge image, and the direction of the circuit is described by calculating the coordinate point sequence of the centerline. Simultaneously, the width of the circuit is measured in the vertical direction of the centerline to obtain the width parameter at each location. The centerline coordinates and width parameters of all conductive circuit image units are collected to form a circuit coordinate set.

[0027] For the component imaging layer, component region localization is performed using a combination of template matching and feature extraction. First, corresponding templates are created based on the appearance characteristics of different types of surface mount components. Template matching is then performed in the component imaging layer to initially determine the component's location and type. Next, for each located surface mount component, the endpoint coordinates of its pins and the contour information of the component body are extracted. Pin coordinates are obtained by detecting the endpoints of the pin edges, while the component body contour is obtained using a contour extraction algorithm. After organizing the pin coordinates and component body contours of all surface mount components, a set of component coordinates is formed.

[0028] Step S123: Through the spatial mapping unit of the structural association module, based on the physical stacking relationship of the circuit board, the center coordinates of each solder joint in the solder joint coordinate set are mapped to the circuit imaging layer to determine the coordinates of the corresponding conductive line endpoints and establish the first spatial association pair between the solder joint and the circuit; the pin coordinates of each component in the component coordinate set are mapped to the circuit imaging layer to determine the coordinates of the corresponding conductive line connection points and establish the second spatial association pair between the component and the circuit.

[0029] In this embodiment, the spatial mapping unit of the structural association module performs coordinate mapping based on the physical stacking relationship of the circuit board. The physical stacking relationship of the circuit board determines that solder joints are usually located at the endpoints of conductive lines, while the pins of components are connected to conductive lines.

[0030] For establishing the first spatial association pair, the spatial mapping unit projects the center coordinates of each solder joint in the solder joint coordinate set onto the circuit imaging layer according to the coordinate alignment mapping relationship. The endpoints of the conductive lines are searched near the projected positions, and the endpoint coordinates of the conductive lines corresponding to the center coordinates of the solder joint are determined by comparing them with the centerline coordinates in the circuit coordinate set. The center coordinates of each solder joint are then bound to the corresponding endpoint coordinates of the conductive lines, forming the first spatial association pair between the solder joint and the circuit.

[0031] When establishing the second spatial association pair, the coordinates of each component pin in the component coordinate set are also mapped to the circuit imaging layer. Connection points on the conductive lines are located around the mapped locations; these connection points are typically the contact points between the pins and the lines. By analyzing the line routing and position information in the circuit coordinate set, the coordinates of the conductive line connection points corresponding to each component pin are determined. The component pin coordinates are then associated with the corresponding conductive line connection point coordinates to form the second spatial association pair between the component and the circuit.

[0032] Step S124: Based on the first spatial association pair and the second spatial association pair, construct an association matrix. The row dimension of the matrix corresponds to the solder joint / component identifier, the column dimension corresponds to the circuit identifier, and the element represents the spatial distance parameter between the two.

[0033] In this embodiment, after obtaining the first spatial association pair and the second spatial association pair, the association matrix is ​​constructed. First, solder joints, components, and circuits are uniquely identified. Each solder joint and component has a corresponding unique number as an identifier, and each conductive circuit also has its own unique identifier number.

[0034] The row dimension of the correlation matrix covers all solder joint and component identifiers, arranged in the order of solder joint identifier first, then component identifier. The column dimension corresponds to all circuit identifiers, arranged in the order of circuit distribution on the circuit board. Each element in the correlation matrix represents the spatial distance parameter between the solder joint or component in the corresponding row and the circuit in the corresponding column.

[0035] For each solder joint and circuit combination in the first spatial association pair, calculate the Euclidean distance between the center coordinates of the solder joint and the coordinates of the corresponding conductive circuit endpoint, and use this distance value as the element value of the corresponding position in the association matrix. For each component and circuit combination in the second spatial association pair, calculate the Euclidean distance between the component pin coordinates and the coordinates of the corresponding conductive circuit connection point, and similarly fill this distance value into the corresponding position in the association matrix. For solder joint / component and circuit combinations without association relationships, their corresponding matrix element values ​​can be set to a larger default value to distinguish them from elements with association relationships.

[0036] Step S125: Perform threshold filtering on the spatial distance parameters in the association matrix, retaining association pairs that are less than the preset association threshold to form a valid association set.

[0037] In this embodiment, the preset association threshold is determined based on the circuit board's design parameters and manufacturing process requirements. This preset association threshold represents the maximum spatial distance range under normal connection relationships between solder joints and circuits, and between components and circuits. In practical applications, the preset association threshold can be obtained through statistical analysis of a large number of normal circuit board samples, ensuring accurate differentiation between normal association relationships and abnormal non-association relationships.

[0038] Each element in the association matrix, i.e., the spatial distance parameter, is checked one by one. When the spatial distance parameter is less than a preset association threshold, it indicates that there is a valid physical connection between the corresponding solder joint and the circuit, or between the component and the circuit, and the association pair is retained. When the spatial distance parameter is greater than or equal to the preset association threshold, it is considered that there is no valid association between the corresponding two, and the association pair is removed. After threshold filtering, all the retained association pairs form a valid association set, which reflects the actual connection relationship between the various structures on the circuit board.

[0039] Step S126: Combine the effective association set with the coordinates of the suspected defect areas in each imaging layer, mark the corresponding positions of the same defect in each imaging layer, arrange them in the order of the physical structure of the circuit board, and generate a layered defect association map.

[0040] In this embodiment, the coordinates of suspected defect areas in each imaging layer are obtained through preliminary defect detection by aligning the imaging layers with coordinates. In the solder joint imaging layer, the coordinate range of suspected solder joint defect areas is determined by analyzing the morphological characteristics of the solder joints, such as irregular contours and abnormal surface depressions. In the circuit imaging layer, the coordinates of suspected circuit defect areas are marked by detecting phenomena such as circuit interruptions, short circuits, and irregular edges. In the component imaging layer, the coordinates of suspected component defect areas are determined based on characteristics such as component position offset and pin deformation.

[0041] The correlation pairs in the effective correlation set are compared and analyzed with the coordinates of these suspected defect areas. For each suspected defect area, the structures of other imaging layers associated with it in the effective correlation set are searched. For example, when a suspected defect area is found in the solder joint imaging layer, the associated lines are found through the effective correlation set, thereby determining the corresponding location of the defect in the line imaging layer.

[0042] After marking the corresponding locations of the same defect in different imaging layers, these location information and relationships are organized and arranged according to the physical structure of the circuit board, i.e., from the solder joint layer, circuit layer to the component layer. This information is then visualized in the form of a map, which includes the identifier of each imaging layer, the coordinates of the suspected defect area, and the connecting lines of the relationships, ultimately generating a layered defect association map.

[0043] Step S130: Based on the layered defect association map, extract the specific features of the defect region of each imaging layer, extract the solder joint morphology association parameters in the solder joint imaging layer, extract the line texture continuity parameters in the line imaging layer, and extract the component pin offset association parameters in the component imaging layer to form a set of layered specific defect features.

[0044] In this embodiment, the layered defect correlation map clearly shows the location of suspected defect regions in each imaging layer and the relationships between them. Based on this map, specific defect feature parameters are extracted for different imaging layers.

[0045] In the solder joint imaging layer, the focus is on the morphological changes of the solder joints, as their shape directly reflects the welding quality. By analyzing images of areas suspected of solder joint defects, parameters describing the morphological characteristics of the solder joints are extracted. These parameters can indicate whether defects such as cold solder joints or voids exist. In the circuit imaging layer, the continuity of the conductive circuit texture is crucial for determining whether defects such as open circuits or short circuits exist; therefore, parameters related to the continuity of the circuit texture are extracted. In the component imaging layer, the offset of component leads directly affects the connection quality between the component and the circuit; therefore, parameters related to component lead offset are extracted.

[0046] The extracted solder joint morphology correlation parameters, circuit texture continuity parameters, and component pin offset correlation parameters are classified and organized according to the imaging layer category to form feature subsets corresponding to different structural regions. These feature subsets are then combined to form a layered exclusive defect feature set.

[0047] Step S131: Separate the solder joint association sub-map, circuit association sub-map, and component association sub-map from the layered defect association map, which respectively contain defect association information of solder joint-circuit, circuit itself, and component-circuit.

[0048] In this embodiment, the layered defect association map contains various types of defect association information. To facilitate the targeted extraction of features specific to each imaging layer, it is necessary to separate the map into sub-maps. The separation process is based on the type of association in the map and the imaging layers involved.

[0049] The separation of the solder joint association sub-map primarily focuses on the association information related to the solder joints. By filtering the portions of the layered defect association map that involve the relationship between solder joints and circuits, these association pairs, the corresponding coordinates of suspected defect areas, and related structural identifiers are extracted to form the solder joint association sub-map. This solder joint association sub-map specifically reflects the defect association between solder joints and circuits.

[0050] The line association sub-map focuses on the defect association information of the line itself. It extracts defect association content related to the line's continuity and the intersection relationships between lines from the layered defect association map, including the coordinates of suspected defect areas and the connection relationships between line segments, to form the line association sub-map for analyzing the texture continuity characteristics of the line.

[0051] The separation of the component association sub-map targets the association information between components and circuits. From the layered defect association map, association pairs between component pins and circuit connection points, coordinates of suspected component defect areas, and corresponding circuit information are extracted to form the component association sub-map, which is used to extract component pin offset association parameters.

[0052] Step S132: Perform morphological analysis on the suspected defective areas of the solder joints in the solder joint correlation sub-map, extract the outline roundness, the length of the solder joint melt trace, and the proportion of the solder joint void area for each area, and calculate the change rate of the outline roundness, the deviation value of the melt trace length, and the fluctuation value of the void area proportion of the same solder joint under different shooting frames, which are respectively used as independent feature items in the solder joint morphological correlation parameters.

[0053] In this embodiment, morphological analysis is performed on each suspected defect area of ​​a solder joint in the solder joint correlation sub-map. First, high-resolution image data of the area is acquired, and then image segmentation technology is used to separate the solder joint area from the background.

[0054] Step S1321: Extract the contour coordinates of the suspected defect area of ​​each weld point from the weld point correlation sub-map, calculate the contour perimeter and area based on the coordinates, and obtain the contour roundness.

[0055] Edge detection algorithms are used to extract the contour coordinates of suspected solder joint defects. These coordinates describe the contour boundaries in pixel form. Based on these coordinates, a contour tracking algorithm is used to calculate the perimeter of the contour, which is the sum of the distances between all consecutive pixels on the contour. Simultaneously, a polygon approximation method is used to calculate the area enclosed by the contour. By approximating the contour as a polygon, the area value is obtained using the polygon area calculation formula. The roundness of the contour is obtained by dividing the square of the contour perimeter by the contour area by a fixed coefficient. The smaller this value, the closer the solder joint shape is to a standard circle.

[0056] Step S1322: Perform grayscale analysis on the image unit of the suspected solder joint defect area, identify the area with grayscale value lower than the preset melting threshold as the melting trace area, and extract the longest diameter of the area as the solder joint melting trace length.

[0057] Images of suspected solder joint defects are converted to grayscale images. A preset melting threshold is determined through grayscale histogram analysis, based on the grayscale difference between normal solder joints and melting trace areas. Thresholding segmentation is performed on the grayscale images, marking areas with grayscale values ​​below the preset melting threshold as melting trace areas. A region growing algorithm is used to perform morphological processing on the marked areas, eliminating noise interference and connecting disconnected regions. The longest diameter of the melting trace area is detected using Hough transform, and the pixel length of this diameter is converted into the actual physical length, which is taken as the solder joint melting trace length.

[0058] Step S1323: Binarize the image units of the suspected solder joint defect area to distinguish the solder joint foreground area from the solder joint background area. Identify the black pixel area in the solder joint foreground area as the hole area using the hole detection algorithm. Calculate the ratio of the hole area to the solder joint foreground area as the solder joint hole area ratio.

[0059] An adaptive threshold segmentation algorithm is used to binarize the image of the suspected solder joint defect area, dividing the image into black and white regions. The white region represents the foreground area of ​​the solder joint, and the black region represents the background area. Morphological opening operations are performed on the binarized image to remove noise points in the foreground area. Connectivity analysis is used to detect black pixel regions in the foreground area of ​​the solder joint; these regions are identified as voids. The total area of ​​the voids and the total area of ​​the foreground area of ​​the solder joint are calculated separately, and the ratio of the two is taken as the proportion of the void area of ​​the solder joint.

[0060] Step S1324: Obtain the outline roundness, solder melt trace length, and solder cavity area ratio of the same solder joint under different shooting frames, calculate the ratio of the outline roundness difference between adjacent frames to the roundness of the previous frame, and obtain the outline roundness change rate.

[0061] Image data of the same solder joint in multiple consecutive frames are acquired from the image acquisition system, and the contour roundness of each frame is calculated. For two adjacent frames, the difference between the contour roundness of the later frame and the contour roundness of the previous frame is calculated, and then this difference is divided by the contour roundness of the previous frame to obtain the contour roundness change rate between adjacent frames. The same calculation is performed on all adjacent frame pairs to form a sequence of contour roundness change rates for the solder joint.

[0062] Step S1325: Calculate the average length of the solder joint melt trace in all shooting frames of the same solder joint. Take the absolute value of the difference between the length in each shooting frame and the average value as the melt trace length deviation value.

[0063] Collect the solder joint melt trace length data for the same solder joint across all shooting frames, and calculate the arithmetic mean of these data. For the melt trace length of each shooting frame, calculate the difference between its value and the average value, and take the absolute value of this difference to obtain the melt trace length deviation value for each shooting frame. Arrange all deviation values ​​in the order of the shooting frames to form the melt trace length deviation sequence for that solder joint.

[0064] Step S1326: Calculate the standard deviation of the proportion of void area of ​​the solder joint in all shooting frames of the same solder joint, and use it as the void area proportion fluctuation value.

[0065] Based on the percentage of void area of ​​the same solder joint across all shooting frames, the standard deviation of these data is calculated. The standard deviation is calculated by first squared the deviation of each data point from the mean, then averaging these squared values, and finally taking the square root. This standard deviation represents the fluctuation value of the void area percentage, reflecting the degree of dispersion of the void area percentage across different shooting frames.

[0066] Step S1327: Arrange the contour roundness change rate, melt trace length deviation value, and void area ratio fluctuation value of the same weld point in the calculation order to form weld point morphology related parameters.

[0067] The contour roundness change rate sequence obtained in step S1324, the melt trace length deviation sequence obtained in step S1325, and the void area ratio fluctuation value obtained in step S1326 are integrated. These parameters are arranged into a multi-dimensional vector in the order of contour roundness change rate, melt trace length deviation value, and void area ratio fluctuation value, with each dimension corresponding to a parameter sequence, forming the morphological correlation parameters of the weld point.

[0068] Step S133: Perform texture analysis on the suspected defect areas of the line in the line association sub-map, extract the line edge smoothness, line width uniformity and line grayscale consistency of each area, calculate the edge smoothness difference, width uniformity deviation and grayscale consistency fluctuation of the same line segment at different imaging positions, and combine them into line texture continuity parameters.

[0069] In this embodiment, a multi-dimensional texture analysis method is used to extract feature parameters for suspected line defect areas in the line-related sub-map. By analyzing the pixel distribution and morphological features of the line image, key indicators reflecting line quality are obtained.

[0070] Step S1331: Extract the line edge coordinates of each suspected defect area from the line association sub-map, fit the edge curve based on the coordinates, calculate the curvature change of the curve, and use the reciprocal of the curvature change as the line edge smoothness.

[0071] The Canny edge detection algorithm is used to extract the left and right edge coordinates of suspected line defects, resulting in a pixel sequence of the edges. A B-spline curve is then fitted to the edge coordinates to obtain a continuous edge curve. Multiple sampling points are uniformly selected on the fitted curve, and the curvature value at each sampling point is calculated. The change in curvature is obtained by calculating the difference between the curvature values ​​of adjacent sampling points. The reciprocal of this change is used as the line edge smoothness; a larger value indicates a smoother edge.

[0072] Step S1332: Select multiple measurement points evenly along the extension direction of the suspected defect area. Cut a line segment perpendicular to the extension direction at each measurement point. The distance between the intersection of the measurement line segment and the edge of the line is taken as the line width of the measurement point. Calculate the variance of the line width of all measurement points and take the reciprocal of the variance as the line width uniformity.

[0073] Measurement points are selected at fixed intervals along the line extension direction, with the interval determined based on the line length and image resolution. At each measurement point, a line segment is drawn perpendicular to the line extension direction, intersecting the left and right edges of the line at two points. The pixel distance between these two intersection points is calculated and converted into the actual physical width, which is taken as the line width for that measurement point. The variance of the line width at all measurement points is calculated, and the reciprocal of the variance is used as the line width uniformity; a larger value indicates a more uniform width.

[0074] Step S1333: Select multiple sampling areas evenly within the suspected defect area of ​​the line, calculate the average gray value of each sampling area, and then calculate the variance of the average gray value of all sampling areas. Use the reciprocal of the variance as the line gray value consistency.

[0075] Within the suspected defect area of ​​the line, sampling areas are uniformly selected according to a grid division method, with each sampling area being a square or rectangle. The average gray value of all pixels within each sampling area is calculated, resulting in multiple average gray value data. The variance of these average gray values ​​is calculated, and the reciprocal of the variance is used as the line gray-level consistency; a larger value indicates a more uniform gray-level distribution.

[0076] Step S1334: Obtain the line edge smoothness, line width uniformity, and line grayscale consistency of the same line segment at different imaging positions, and calculate the absolute value of the smoothness difference between adjacent imaging positions as the edge smoothness difference.

[0077] Image data of the same line segment at multiple consecutive imaging positions are acquired, and the line edge smoothness at each position is calculated. For two adjacent imaging positions, the difference between the edge smoothness at the latter position and the edge smoothness at the former position is calculated, and the absolute value of this difference is taken as the edge smoothness difference value. This calculation is performed on all pairs of adjacent imaging positions to form an edge smoothness difference value sequence.

[0078] Step S1335: Calculate the average value of the uniformity of the line width at all imaging positions of the same line segment. Take the absolute value of the difference between the uniformity at each imaging position and the average value as the width uniformity deviation.

[0079] Collect line width uniformity data for the same line segment at all imaging locations and calculate the arithmetic mean of these data. For the width uniformity at each imaging location, calculate the difference between its value and the mean, and take the absolute value as the width uniformity deviation. Arrange all deviation values ​​in order of imaging location to form a width uniformity deviation sequence.

[0080] Step S1336: Calculate the standard deviation of line grayscale consistency at all imaging positions of the same line segment, as the grayscale consistency fluctuation.

[0081] Based on the line grayscale consistency data of the same line segment at all imaging locations, the standard deviation of these data is calculated. The standard deviation is obtained by first calculating the square of the deviation of each data point from the mean, then calculating the average of the squared values, and finally taking the square root. This standard deviation is the grayscale consistency fluctuation, reflecting the degree of change in grayscale consistency at different imaging locations.

[0082] Step S1337: Arrange the edge smoothness difference, width uniformity deviation, and grayscale consistency fluctuation of the same line segment in the order of calculation to form the line texture continuity parameters.

[0083] The edge smoothness difference sequence obtained in step S1334, the width uniformity deviation sequence obtained in step S1335, and the grayscale consistency fluctuation value obtained in step S1336 are integrated. These parameters are arranged into a multi-dimensional vector in the order of edge smoothness difference, width uniformity deviation, and grayscale consistency fluctuation to form the texture continuity parameters of the line segment.

[0084] Step S134: Perform pin location analysis on the suspected defect areas of the components in the component correlation sub-map, extract the pin spacing, pin alignment with the line connection end and pin bending angle of each area, calculate the pin spacing variation, alignment deviation and bending angle fluctuation of the same component in different imaging layers, and use them as independent feature terms in the component pin offset correlation parameters.

[0085] In this embodiment, pin location analysis is performed on each suspected defect area in the component association sub-map. First, high-resolution image data of the area is acquired. Image segmentation and edge detection techniques are used to separate the pin portions of the component, clarifying the contour and position information of each pin. For pin spacing extraction, the central axis or edge feature points of each adjacent pin are determined, and the distance between corresponding feature points of adjacent pins is measured to obtain the pin spacing. This parameter reflects whether the distance between adjacent pins meets the design requirements.

[0086] When extracting the alignment between a pin and a circuit connection, the first step is to determine the position of the pin's end and the corresponding circuit connection. Image analysis is used to locate the edge points of the pin's end and the circuit connection's edge points, and the offset of these points in a predetermined direction is calculated. This offset is compared to a preset standard alignment distance to obtain the alignment degree between the pin and the circuit connection. A higher alignment degree indicates a more accurate connection position.

[0087] The pin bending angle is determined by fitting the pin's central axis. First, the edge coordinates of the pin are extracted, and the central axis of the pin is fitted based on these coordinates. This central axis can be a straight line or a curve. Then, the standard axis direction that the pin should have is determined according to the design requirements, and the angle between the actual central axis and the standard axis is calculated. This angle is the pin bending angle, and this parameter reflects whether the degree of pin bending is within the allowable range.

[0088] Since the imaging angle and position of components may differ in different imaging layers, for the same component, the pin spacing, pin-to-line connection alignment, and pin bending angle are obtained in different imaging layers. The difference in pin spacing between adjacent imaging layers is calculated to obtain the pin spacing variation, which reflects the change in pin spacing in different imaging layers.

[0089] Calculate the average alignment of pins and line connections under all imaging layers of the same component. Then, subtract the average alignment from the alignment of each imaging layer and take the absolute value to obtain the alignment deviation value for each imaging layer. This alignment deviation value reflects the degree of deviation of the alignment in each imaging layer from the average level.

[0090] Calculate the standard deviation of the pin bending angle across all imaging layers for the same element, and use this as the bending angle fluctuation value. This fluctuation value reflects the stability of the pin bending angle variation across different imaging layers. Treat the pin pitch variation, alignment deviation, and bending angle fluctuation value as independent characteristic terms in the element pin offset correlation parameters.

[0091] Step S135: Classify the solder joint morphology association parameters, circuit texture continuity parameters, and component pin offset association parameters according to the imaging layer, and assign them to the solder joint feature subset, circuit feature subset, and component feature subset, respectively.

[0092] In this embodiment, after extracting the defect feature parameters specific to each imaging layer, they need to be categorized and organized according to the type of imaging layer. All weld point morphology-related parameters obtained in step S132, including independent feature items such as the change rate of contour roundness of the suspected defect area of ​​each weld point, the deviation value of melt trace length, and the fluctuation value of void area ratio, are grouped into a weld point feature subset. The parameters within this weld point feature subset are sorted according to the weld point's identifier, with the parameters corresponding to each weld point forming a sub-item, facilitating subsequent feature processing and analysis.

[0093] The line texture continuity parameters extracted in step S133, namely the edge smoothness difference, width uniformity deviation, and grayscale consistency fluctuation of each suspected defect area, are uniformly classified into a line feature subset. The parameters within the line feature subset are organized according to the identification of the line segment, with the parameters corresponding to each line segment forming a sub-item, ensuring the orderly management of line features.

[0094] The component pin offset correlation parameters obtained in step S134, including independent feature items such as the pin spacing change, alignment deviation, and bending angle fluctuation in each suspected defect area of ​​a component, are categorized into component feature subsets. The parameters within these subsets are arranged according to the component's identifier, with each component's corresponding parameter forming a sub-item, thus achieving categorized storage of component features. This categorization and organization clarifies the defect feature parameters of different imaging layers, laying the foundation for subsequent feature set integration.

[0095] Step S136: Integrate the solder joint feature subset, circuit feature subset, and component feature subset according to the circuit board structure hierarchy to form a layered exclusive defect feature set. Each feature subset in the layered exclusive defect feature set corresponds to an exclusive defect feature of a structural region of the circuit board.

[0096] In this embodiment, after classifying each feature subset, they are integrated according to the structural hierarchy of the circuit board. The structural hierarchy of the circuit board typically starts from the bottom conductive lines, followed by solder joints and surface mount components, which conforms to the physical structure and signal transmission path of the circuit board.

[0097] First, a subset of circuit features is placed at the beginning of the integration sequence as the base layer features, because conductive circuits are fundamental to signal transmission on the circuit board, and their defects can affect the normal operation of solder joints and components. Next, a subset of solder joint features is integrated into the intermediate layer. Solder joints connect conductive circuits and components, occupying an intermediate connection position, and their features are closely related to the circuit features. Finally, a subset of component features is integrated into the top layer. Components are the core components that enable the circuit board to achieve specific functions, and their defect features depend on their connection relationship with the circuits and solder joints.

[0098] During the integration process, a corresponding structural region identifier is added to each feature subset to clarify its corresponding imaging layer and structural type. Simultaneously, an association index is established between the feature subsets. This index allows for the rapid identification of feature parameters with relationships between different feature subsets, such as the circuit feature parameters and component feature parameters corresponding to a specific solder joint feature parameter. The resulting layered, dedicated defect feature set contains the unique defect features of each structural region of the circuit board, and the relationships between the feature subsets are clear, facilitating subsequent feature enhancement processing by the model.

[0099] Step S140: Input the layered specific defect feature set into the model layer-level feature enhancement layer. Through the layer's built-in defect feature attention module, perform targeted enhancement processing on each parameter in the layered specific defect feature set to generate a layered enhanced defect feature set.

[0100] In this embodiment, the hierarchical dedicated defect feature set includes dedicated defect feature parameters for three structural regions: solder joints, circuits, and components. However, these parameters differ in their importance for defect detection. The role of the model-level feature enhancement layer is to use the defect feature attention module to perform targeted enhancement processing on these parameters, highlighting features that are more valuable for defect classification.

[0101] The defect feature attention module comprises multiple branches, each corresponding to a different feature subset. Each branch automatically assigns different weights based on the importance of the feature. During processing, the module analyzes the correlation between each feature parameter and historical defect data. Parameters that have proven to contribute significantly to defect type determination in past defect detection are given higher weights to enhance their influence within the feature set; parameters with lower contributions are given lower weights to reduce their impact.

[0102] Through the aforementioned targeted enhancement processes, the effective features in the hierarchical, dedicated defect feature set are made more prominent, while noisy features are suppressed. The processed feature parameters are reorganized according to the original hierarchical structure to form a hierarchically enhanced defect feature set. The feature parameters in this set have higher recognition in defect classification, which can improve the accuracy of subsequent defect type matching.

[0103] Step S141: Input the solder joint feature subset, circuit feature subset, and component feature subset from the layered exclusive defect feature set into the solder joint attention branch, circuit attention branch, and component attention branch of the defect feature attention module, respectively.

[0104] In this embodiment, the defect feature attention module includes three independent branches: solder joint attention branch, circuit attention branch, and component attention branch. Each branch specifically processes its corresponding feature subset. During the input process, the layered dedicated defect feature set is first parsed to separate the solder joint feature subset, circuit feature subset, and component feature subset.

[0105] After the subset of solder joint features is converted according to a preset input format, it is input into the solder joint attention branch. The solder joint attention branch contains a weight calculation and feature processing unit for the solder joint features, which can identify parameters in the solder joint features that are key to defect detection, such as the fluctuation value of the void area ratio.

[0106] After format conversion, the subset of line features is input into the line attention branch. The processing unit of the line attention branch is optimized according to the characteristics of the line features, and can focus on key items in the line texture continuity parameters, such as edge smoothness difference. These parameters are crucial for judging defects such as line open circuits and short circuits.

[0107] The component feature subset, after format conversion, is input into the component attention branch. This branch focuses on processing component pin offset-related parameters, identifying parameters that significantly impact component defect judgment, such as pin alignment deviations. Through this branching process, each branch can be specifically optimized based on the characteristics of its corresponding feature subset, improving the efficiency and accuracy of the attention mechanism.

[0108] Step S142: In the solder joint attention branch, the solder joint feature weight calculation unit within the branch standardizes and weights the contour roundness change rate, melt trace length deviation value, and void area ratio fluctuation value in the solder joint shape association parameters to generate a set of solder joint feature vectors.

[0109] In this embodiment, the solder joint feature weight calculation unit within the solder joint attention branch first standardizes the input solder joint morphology-related parameters. The purpose of standardization is to eliminate the differences in dimensions between different parameters, enabling them to be compared and processed on the same scale. For the three parameters—profile roundness change rate, melt trace length deviation, and void area ratio fluctuation—the mean and standard deviation of each parameter across all solder joint feature samples are calculated. Then, the mean is subtracted from each parameter value, and the result is divided by the standard deviation to obtain the standardized parameter value.

[0110] After standardization, the process moves to the weighting stage. The solder joint feature weight calculation unit learns the importance weight of each parameter in solder joint defect classification by analyzing a large amount of historical defect data. Parameters that are more discriminative in defect detection, such as the fluctuation value of void area ratio, are assigned higher weights; relatively less important parameters are assigned lower weights. Each standardized parameter value is multiplied by its corresponding weight to obtain the weighted parameter value.

[0111] The weighted rate of change of contour roundness, deviation value of melt trace length, and fluctuation value of void area ratio corresponding to each solder joint are arranged in order to form a multi-dimensional vector, with each vector representing the reinforcement feature of a solder joint. The vectors of all solder joints are combined to generate a set of solder joint feature vectors, in which the vectors highlight the feature information that is more important for judging solder joint defects.

[0112] Step S143: In the line attention branch, the line feature weight calculation unit within the branch is used to standardize and weight the line edge smoothness difference, width uniformity deviation, and grayscale consistency fluctuation in the line texture continuity parameters to generate a set of line feature vectors.

[0113] In this embodiment, the line feature weight calculation unit within the line attention branch first standardizes the line texture continuity parameters. For the three parameters—line edge smoothness difference, width uniformity deviation, and grayscale consistency fluctuation—the mean and standard deviation are calculated for each across all line feature samples. Then, the specific value of each parameter is subtracted from its corresponding mean and divided by its standard deviation to complete the standardization transformation, ensuring that different parameters have the same scale.

[0114] After standardization, a weighted processing is performed. The line feature weight calculation unit determines the weight of each parameter based on historical defect detection data. For example, the difference in line edge smoothness is more important for judging whether there are defects such as burrs or breaks in the line, so it is given a higher weight; while the fluctuation of grayscale consistency has a relatively small impact on defect judgment in some cases, so it has a lower weight. Each standardized parameter value is multiplied by its corresponding weight to obtain the weighted parameter value.

[0115] The weighted differences in line edge smoothness, width uniformity deviation, and grayscale consistency fluctuations corresponding to each line segment are sequentially combined to form a multi-dimensional vector, with each vector representing the enhancement feature of a line segment. The vectors from all line segments are then aggregated to generate a line feature vector set, which more effectively reflects the defect characteristics of the line.

[0116] Step S144: In the component attention branch, the component feature weight calculation unit within the branch standardizes and weights the pin spacing change, alignment deviation, and bending angle fluctuation in the component pin offset correlation parameters to generate a set of component feature vectors.

[0117] In this embodiment, the component feature weight calculation unit within the component attention branch first standardizes the component pin offset associated parameters. For the three parameters—pin pitch variation, alignment deviation, and bending angle fluctuation—the mean and standard deviation are calculated across all component feature samples. Then, the actual value of each parameter is subtracted from the mean and divided by the standard deviation to obtain the standardized parameter value, thus eliminating dimensional differences between different parameters.

[0118] Next, a weighted processing step is performed. The component feature weight calculation unit learns the importance weight of each parameter based on historical data. The alignment deviation value directly reflects the accuracy of the pin and circuit connection and has a significant impact on whether the component can work properly, so it is given a high weight. The pin pitch variation and bending angle fluctuation value are also assigned corresponding weights according to their contribution to defect judgment. Each standardized parameter value is multiplied by its corresponding weight to obtain the weighted parameter value.

[0119] The weighted pin pitch variation, alignment deviation, and bending angle fluctuation for each component are arranged sequentially to form a multi-dimensional vector, with each vector representing a component's reinforcement feature. All component vectors are combined to generate a component feature vector set, which highlights key features related to component pin offset.

[0120] Step S145: Perform feature dimension standardization processing on the solder joint feature vector set, circuit feature vector set, and component feature vector set respectively to obtain a standardized feature vector set.

[0121] In this embodiment, although the parameters within each feature vector set after weighted processing by each attention branch have been standardized, the feature vector dimensions and numerical ranges between different sets may differ. To facilitate subsequent cross-layer association and fusion, feature dimension standardization processing needs to be performed on each of these three feature vector sets.

[0122] For the set of solder joint feature vectors, analyze the dimensionality of each vector to ensure that all solder joint feature vectors have the same number of dimensions. If there are inconsistencies in dimensions, adjust them by padding with zeros or feature filtering to make the dimensions of each vector uniform. At the same time, calculate the mean and standard deviation of all vector elements in the entire set of solder joint feature vectors, subtract the mean from each vector element and then divide by the standard deviation to make the numerical range of the vector elements uniform.

[0123] The same method is used to process the circuit feature vector set, unifying the dimension of the circuit feature vectors and standardizing the vector elements to eliminate differences in numerical range. The same dimension unification and element standardization operations are performed on the component feature vector set to ensure that the three feature vector sets are consistent in both dimension and numerical scale. The resulting standardized feature vector set contains solder joint, circuit, and component feature vectors with unified dimensions and consistent numerical scale.

[0124] Step S146: Based on the spatial correspondence of the hierarchical defect association map, the standardized feature vector set is cross-layer associated and fused to generate a cross-layer fused feature vector.

[0125] In this embodiment, cross-layer correlation fusion is based on the spatial mapping relationship recorded in the hierarchical defect correlation map to achieve the organic integration of feature vectors from different imaging layers and enhance the feature's ability to represent complex defects.

[0126] Step S1461: Extract the identification information of all defect association pairs from the layered defect association map. Each defect association pair contains a solder joint identifier, a circuit identifier, and a component identifier, corresponding to the same defect area.

[0127] The data structure of the layered defect association map is analyzed to extract information on all recorded defect association pairs. Each defect association pair contains unique solder joint identifiers, circuit identifiers, and component identifiers, which correspond one-to-one with the actual structures on the circuit board. By matching the identifiers, the physical defect region corresponding to each association pair is determined, ensuring that the fused feature vectors come from different imaging layers of the same defect region.

[0128] Step S1462: Based on the identification information, select three types of vectors corresponding to the same defect association pair from the standardized feature vector set to form an association feature group.

[0129] Based on the identification information of defect association pairs, vector filtering is performed in the standardized feature vector set. For each association pair, the solder joint feature vector corresponding to the solder joint identifier, the circuit feature vector corresponding to the circuit identifier, and the component feature vector corresponding to the component identifier are found respectively. These three vectors are combined to form an association feature group, and each association feature group contains standardized features of the same defect area in three imaging layers.

[0130] Step S1463: Perform feature concatenation on the three types of vectors in the associated feature group to obtain the concatenated cross-layer feature vector.

[0131] A feature concatenation method is used to fuse the three types of vectors in the associated feature group, concatenating the vectors end-to-end in the order of solder joint feature vector, circuit feature vector, and component feature vector. During the concatenation process, the dimensions and feature order of each vector remain unchanged, forming a cross-layer feature vector whose dimension is the sum of the dimensions of the three types of vectors. This cross-layer feature vector contains comprehensive feature information of the defect region at different imaging layers.

[0132] Step S1464: Extract the key feature elements after standardization from the concatenated cross-layer feature vectors. The key feature elements correspond to the key parameters for defect determination.

[0133] By analyzing prior knowledge of circuit board defect detection, key feature parameters that play a crucial role in defect judgment are identified, such as the proportion of void area in solder joints and the difference in edge smoothness of circuits. Feature elements corresponding to these key parameters are extracted from the cross-layer feature vector after connection, forming a key feature subset. The extraction of key feature elements is based on feature indexing, with the index position predetermined according to the definition of the standardized feature vector.

[0134] Step S1465: Perform feature enhancement processing on key feature elements, while keeping non-key feature elements unchanged, to obtain the adjusted cross-layer feature vector.

[0135] Key feature elements are enhanced by feature scaling, which proportionally enlarges their numerical range to increase their weight in the feature vector. The values ​​of non-key feature elements remain unchanged to ensure no loss of original feature information. The adjusted cross-layer feature vector retains complete feature information while highlighting the influence of key features.

[0136] Step S1466: Bind the adjusted cross-layer feature vector to the corresponding defect association pair identification information to generate a cross-layer fusion feature vector, wherein the cross-layer fusion feature vector contains the association identifier and the enhanced cross-layer feature information.

[0137] The adjusted cross-layer feature vectors are associated with the corresponding defects to bind the identification information, which includes solder joint identifiers, circuit identifiers, component identifiers, and the coordinate information of the defect area. The bound cross-layer fused feature vectors not only contain enhanced feature data but also carry the location and correlation information of the defect area, providing complete feature support for subsequent defect classification.

[0138] Step S147: Integrate all cross-layer fusion feature vectors according to the order of the circuit board structure regions to form a hierarchical reinforcement defect feature set. Each hierarchical reinforcement defect feature in the hierarchical reinforcement defect feature set corresponds to the reinforcement feature of an associated defect region of the circuit board.

[0139] In this embodiment, after generating multiple cross-layer fusion feature vectors, they are integrated according to the structural region order of the circuit board. The structural region order of the circuit board is arranged from the circuit region, through the solder joint region to the component region, which is consistent with the physical layout and signal flow of the circuit board.

[0140] First, cross-layer fusion feature vectors related to the circuit region are categorized and integrated. These vectors mainly contain the association features between circuits and solder joints, and between circuits and components. Then, cross-layer fusion feature vectors related to the solder joint region are categorized. These vectors highlight the association features between solder joints and circuits, as well as the reinforcement features of the solder joints themselves. Finally, cross-layer fusion feature vectors related to the component region are integrated. These vectors contain the association features between components and circuits, and the reinforcement features of component pins.

[0141] During the integration process, a structural region identifier is added to each cross-layer fusion feature vector to clarify its corresponding circuit board structural region. Simultaneously, the vectors are sorted according to the actual physical distribution order of each structural region on the circuit board, ensuring that the arrangement order of the hierarchical reinforcement defect feature set corresponds to the actual layout of the circuit board. For example, cross-layer fusion feature vectors corresponding to circuit regions located at the edge of the circuit board are arranged at the beginning of the set; vectors corresponding to component-dense areas located in the center of the circuit board are arranged in the middle of the set.

[0142] After integration, a consistency check is performed on the hierarchical reinforcement defect feature set to ensure that each associated defect region has a corresponding reinforcement feature vector, and that there are no duplicate or missing vectors. Through this integration method, the resulting hierarchical reinforcement defect feature set can systematically present the reinforcement characteristics of each associated defect region on the circuit board, providing structured feature data support for subsequent defect type matching and inspection report generation.

[0143] Step S150: Call the model scene adaptation classification layer to perform circuit board-specific defect type matching on the hierarchical enhanced defect feature set, output solder joint, circuit and component defect results, and generate a circuit board defect detection report containing defect type, hierarchical position and correlation relationship by combining the position mapping relationship of the hierarchical defect association map.

[0144] In this embodiment, the model scene adaptation classification layer is the output layer of the circuit board defect hierarchical detection model. It is specifically used for accurate matching of defect types and output of results based on the hierarchical enhanced defect feature set. This classification layer contains multiple classification units for defects in different structural regions, which can analyze and identify the defect features of solder joints, circuits, and components respectively.

[0145] First, the hierarchical enhanced defect feature set is input into the scene adaptation classification layer. The classification layer assigns the feature vectors to corresponding classification units based on their structural region identifiers. Each classification unit processes the input feature vectors using a pre-trained classification model and determines the defect type corresponding to each feature vector through a feature-defect type matching algorithm.

[0146] After obtaining the defect types for each structural region, the specific location of each defect in each imaging layer of the circuit board is determined by combining the positional mapping relationship in the layered defect correlation map. The defect type, corresponding layer position, and correlation relationship between each defect are integrated to generate a circuit board defect detection report according to a preset report format. The report clearly presents detailed information on various defects, providing a basis for circuit board repair and quality assessment.

[0147] Step S151: Input the hierarchical enhanced defect feature set into the feature splitting unit in the model scene adaptation classification layer, and split it into solder joint-circuit association feature subset and component-circuit association feature subset according to the defect association type corresponding to the cross-layer fusion feature vector.

[0148] In this embodiment, the feature splitting unit within the model scene adaptation classification layer first parses each cross-layer fusion feature vector in the hierarchical enhancement defect feature set, extracting the defect association type identifier contained in the vector. The defect association type identifier records the structural association relationship corresponding to the vector, such as the association between solder joints and circuits, or the association between components and circuits.

[0149] Based on these identifiers, the feature splitting unit divides the hierarchical reinforcement defect feature set into two main feature subsets. The solder joint-circuit association feature subset contains all cross-layer fused feature vectors identified as solder joint-circuit associations, which collectively reflect the association defect features between solder joints and circuits. The component-circuit association feature subset contains cross-layer fused feature vectors identified as component-circuit associations, focusing on the association defect features between components and circuits.

[0150] During the splitting process, the feature splitting unit rigorously verifies the association type identifier of each vector to ensure that the vector is accurately assigned to the corresponding subset. After splitting, the two feature subsets are checked for dimensionality to ensure that the vectors in the subsets have consistent dimensions, preparing for subsequent classification processing. Through this splitting process, different types of associated defect features can be processed by the corresponding classification units, improving the accuracy of defect type matching.

[0151] Step S152: Input the solder joint-circuit association feature subset into the solder joint defect classification unit in the scene adaptation classification layer, classify it through the solder joint defect classification sub-network in the unit, and output the solder joint defect type. The solder joint defect type includes solder joint void, solder joint cold solder joint, and solder joint solder overflow.

[0152] In this embodiment, the solder joint defect classification unit uses a deep learning classification network to accurately identify the type of solder joint defect. The input features are processed and classified step by step through a multi-layer network structure.

[0153] Step S1521: Input the solder joint-circuit association feature subset into the input layer of the solder joint defect classification subnetwork, and compress the dimension of the cross-layer fusion feature vector to obtain the compressed solder joint feature vector.

[0154] The solder joint-circuit association feature subset contains multiple cross-layer fused feature vectors, which are input into the input layer of the solder joint defect classification sub-network. The input layer performs dimensionality compression on the feature vectors through a fully connected layer, reducing the feature dimensionality while retaining key information. The compression process is achieved through matrix multiplication, transforming the high-dimensional cross-layer fused feature vectors into low-dimensional compressed solder joint feature vectors, thus reducing the computational complexity of the network.

[0155] Step S1522: Input the compressed solder joint feature vector into the hidden layer of the sub-network, and perform feature transformation through two fully connected layers within the layer to generate the transformed solder joint feature vector. The ReLU function is used as the activation function of the fully connected network.

[0156] The compressed solder joint feature vector is fed into the hidden layer of the sub-network, which consists of two fully connected layers. The first fully connected layer performs a linear transformation on the input vector, mapping it to a new feature space through a weight matrix, and then performs a nonlinear transformation using the ReLU activation function to enhance the network's feature representation capability. The transformed feature vector is then fed into the second fully connected layer for further feature transformation and nonlinear processing, generating a transformed solder joint feature vector that contains more abstract defect feature information.

[0157] Step S1523: Input the transformed solder joint feature vector into the output layer of the sub-network, and calculate the probability values ​​of solder joint voids, solder joint cold solder joints, and solder overflow corresponding to the transformed solder joint feature vector through the softmax activation function in the layer to obtain the solder joint defect probability distribution.

[0158] The transformed solder joint feature vector is input into the output layer of the sub-network. The output layer is a fully connected layer, and the number of output nodes is the same as the number of solder joint defect types. Through linear transformation of the fully connected layer, the feature vector is mapped to a probability space, and then the probability value of each defect type is calculated by the Softmax activation function. The sum of the probability values ​​is 1, forming the solder joint defect probability distribution, reflecting the likelihood that the feature vector belongs to each defect type.

[0159] Step S1524: Extract the defect type with the highest probability value from the solder joint defect probability distribution, and use it as the solder joint defect type corresponding to the cross-layer fusion feature vector.

[0160] The probability distribution of solder joint defects is analyzed to identify the defect type with the highest probability value, which is then determined as the solder joint defect type corresponding to the cross-layer fusion feature vector. When multiple conditions have the same maximum probability value, the defect type is selected according to a preset priority order, which is determined based on the severity and commonality of the defect.

[0161] Step S1525: Statistically analyze the solder joint defect types corresponding to all cross-layer fusion feature vectors, and record the number of each solder joint defect type and the corresponding defect association pair identifier.

[0162] Iterate through all cross-layer fused feature vectors in the solder joint-circuit associated feature subset and count the frequency of each solder joint defect type. Simultaneously, record the defect association pair identifiers corresponding to each defect type to establish a correspondence between defect types and defect areas, facilitating subsequent defect localization and report generation.

[0163] Step S1526: Bind the statistically analyzed solder joint defect types with the corresponding defect association pairs and defect level locations to form solder joint defect results.

[0164] The statistically obtained solder joint defect types are bound to their corresponding defect association tags and defect level location information obtained from the layered defect association map. The bound solder joint defect results contain complete information such as defect type, defect location, and association tags, providing basic data for generating circuit board defect inspection reports.

[0165] Step S153: Input the component-circuit associated feature subset into the circuit defect classification unit and component defect classification unit in the scene adaptation classification layer. The circuit defect classification unit classifies the circuit defects through the circuit defect classification sub-network within the unit and outputs the circuit defect types, including circuit short circuit, circuit open circuit, and circuit corrosion. The component defect classification unit classifies the components through the component defect classification sub-network within the unit and outputs the component defect types, including component offset, pin deformation, and component missing.

[0166] For example, in step S1531: the component-line associated feature subset is input into the feature preprocessing unit of the line defect classification subnetwork, and the cross-layer fusion feature vector is normalized to obtain the normalized line feature vector.

[0167] In this embodiment, after receiving the component-line associated feature subset, the feature preprocessing unit performs normalization processing on the cross-layer fusion feature vector. First, it calculates the mean and standard deviation of all elements in the cross-layer fusion feature vector. Then, it subtracts the mean from each element in the vector and divides it by the standard deviation, ensuring that the values ​​of each dimension of the processed feature vector are on the same order of magnitude. Normalization eliminates the influence of differences in units between different feature dimensions, ensuring that subsequent network layers can learn feature information more effectively. The normalized line feature vector obtained after processing has a more stable numerical distribution, which is beneficial for improving the classification accuracy of the sub-network.

[0168] Step S1532: Input the normalized line feature vector into the sub-network convolutional feature extraction unit, and perform convolution operation on the vector using two convolution kernels of different sizes to generate two convolutional line feature maps. The number of channels in the convolutional line feature map is the same as the number of convolution kernels.

[0169] In this embodiment, the convolutional feature extraction unit is the core component of the line defect classification sub-network for feature extraction. This unit is configured with two convolutional kernels of different sizes, used to capture feature information at different scales from the normalized line feature vector. When the normalized line feature vector is input, the first convolutional kernel performs a sliding window calculation on the vector, extracting local features through convolution to generate the first convolutional line feature map. Subsequently, the second convolutional kernel of a different size performs convolution operations on the normalized line feature vector in the same way, extracting another set of local features to generate the second convolutional line feature map. The number of channels in each convolutional line feature map corresponds to the number of convolutional kernels used, ensuring that the extracted feature information is completely preserved.

[0170] Step S1533: Concatenate the two convolutional line feature maps along the channel dimension to form a multi-channel line feature map, and perform global average pooling on the multi-channel feature map to obtain the global line feature vector.

[0171] In this embodiment, after obtaining two convolutional line feature maps, they are concatenated along the channel dimension. During the concatenation process, the spatial dimension of the feature maps remains unchanged, and the channel information of the two feature maps is merged to form a multi-channel line feature map. This multi-channel feature map integrates feature information extracted by different convolutional kernels, containing richer line defect-related features. Next, global average pooling is performed on the multi-channel line feature map to calculate the average value of all elements in each channel, converting the multi-channel feature map into a fixed-length vector, i.e., the global line feature vector. Global average pooling can reduce the feature dimension, reduce the amount of computation, and at the same time retain the key global information in the feature map.

[0172] Step S1534: Input the global line feature vector into the sub-network classification output unit, and calculate the probability values ​​of line short circuit, line open circuit and line corrosion corresponding to the vector through the fully connected layer in the layer to obtain the line defect probability distribution.

[0173] In this embodiment, the global line feature vector is input into the fully connected layer of the classification output unit. The fully connected layer performs matrix operations with the global line feature vector through weight parameters, and after processing by an activation function, outputs three probability values, corresponding to three defect types: line short circuit, line open circuit, and line corrosion, respectively. The sum of these probability values ​​is 1, reflecting the likelihood that the global line feature vector belongs to each defect type, and together they constitute the line defect probability distribution. The weight parameters of the fully connected layer are optimized through the backpropagation algorithm during the sub-network training process, which can accurately capture the correlation between the global line feature vector and various line defects.

[0174] Step S1535: Extract the defect type with the highest probability value from the line defect probability distribution, and use it as the line defect type corresponding to the cross-layer fusion feature vector.

[0175] In this embodiment, the obtained line defect probability distribution is compared among three probability values. The defect type corresponding to the probability value with the highest probability is identified and determined as the line defect type corresponding to the cross-layer fusion feature vector. This process is based on the principle of maximum probability, assuming that the defect type with the highest probability best matches the actual line defect situation reflected by the cross-layer fusion feature vector. Through the above method, the mapping from feature vector to specific defect type is realized, and each cross-layer fusion feature vector is given a clear defect type label.

[0176] Step S1536: Statistically analyze the line defect types corresponding to all cross-layer fusion feature vectors, and record the quantity of each type and the corresponding defect association pair identifier.

[0177] In this embodiment, after determining the circuit defect type corresponding to each cross-layer fusion feature vector, all defect types are statistically analyzed. During the statistical analysis, the number of three defect types—short circuit, open circuit, and corrosion—is recorded. Simultaneously, a corresponding defect association pair identifier is associated with each defect type, recording which cross-layer fusion feature vector determined the defect type and which defect association pair that cross-layer fusion feature vector corresponds to. Through statistical analysis and recording, the distribution and specific location association information of various circuit defects on the circuit board under inspection can be clearly understood.

[0178] Step S1537: Bind the statistically analyzed line defect types with the corresponding defect association pairs and defect level locations to form line defect results.

[0179] In this embodiment, the statistically obtained circuit defect types are bound to their corresponding defect association markers and the defect level locations extracted from the layered defect association map. The defect level location clarifies the specific location of the defect in the physical coordinate system of the circuit board, including coordinate information within the circuit imaging layer. Through this binding operation, each circuit defect type is associated with a specific association and location information, forming a complete circuit defect result. The circuit defect result includes defect type, quantity, association marker, and location information, providing detailed circuit defect data for generating the final circuit board defect inspection report.

[0180] Step S154: Extract the imaging layer coordinates of the solder joint imaging layer, circuit imaging layer, and component imaging layer corresponding to each defect association pair from the layered defect association map, and convert the imaging layer coordinates into the defect level position in the physical coordinate system of the circuit board. Each imaging layer coordinate corresponds to the defect level position of the solder joint, circuit, and component, respectively.

[0181] In this embodiment, the layered defect association map records the specific coordinate information of each defect association pair in different imaging layers. These coordinates are based on the pixel coordinate system of the imaging layers. To convert these coordinates into positions in the physical coordinate system of the circuit board, it is first necessary to establish the transformation relationship between the pixel coordinate system of the imaging layers and the physical coordinate system of the circuit board.

[0182] By using the calibration parameters of the image acquisition system, the proportional relationship between pixels and physical dimensions is obtained, i.e., the actual physical length corresponding to each pixel. Simultaneously, the position coordinates of the imaging layer origin in the circuit board's physical coordinate system are determined as the reference point for coordinate transformation. For the coordinates in the solder joint imaging layer, based on the proportional relationship and the reference point position, the pixel coordinates are converted into physical coordinates in millimeters, obtaining the layer position of the solder joint defect in the circuit board's physical coordinate system.

[0183] The same method is used to transform the coordinates in the circuit imaging layer and the component imaging layer. For the coordinates of the suspected circuit defect area in the circuit imaging layer, the transformation yields the specific location of the circuit defect in the physical coordinate system of the circuit board; for the coordinates of the suspected component defect area in the component imaging layer, the transformation is used to convert the component defect to its hierarchical location in the physical coordinate system of the circuit board.

[0184] During the conversion process, the accuracy of each coordinate point is verified to ensure that the converted physical coordinates accurately reflect the actual location of the defect on the circuit board. Through the above conversion, the coordinates of different imaging layers are unified under the physical coordinate system of the circuit board.

[0185] Step S155: Associate the solder joint defect type with the solder joint defect level and location to form the solder joint defect result. Similarly, form the circuit defect result and component defect result.

[0186] In this embodiment, after obtaining the solder joint defect type and the corresponding solder joint defect level location, the two are bound together by a defect association identifier. Each solder joint defect type has a corresponding defect association identifier, which is consistent with the association identifier of the solder joint defect level location. By matching the association identifier, the solder joint defect type is associated with the corresponding physical location information.

[0187] For example, for a defect type identified as a solder joint void, the corresponding solder joint defect level coordinates are found based on its associated identifier. The solder joint void type is then bound to this coordinate information, forming a solder joint defect result entry that includes the defect type and physical location. All solder joint defect types are processed in the same way to form a complete set of solder joint defect results.

[0188] To generate line defect results, the line defect types output by the line defect classification unit are associated with the line defect hierarchical location through association identifiers. For example, the line short circuit type is bound to the corresponding line physical location coordinates to form line defect result entries, and all line defect entries constitute the line defect result set.

[0189] The process of forming component defect results is similar. Component defect types are associated with their hierarchical locations using association identifiers. For example, component offset types are bound to the component's physical coordinates on the circuit board, forming component defect result entries, which in turn constitute a set of component defect results. This association method ensures that each defect type has a clearly defined location.

[0190] Step S156: Extract the defect association relationships in the layered defect association map, mark the association pairs between solder joint defects and circuit defects, and between component defects and circuit defects, and integrate the association relationships with the solder joint defect results, circuit defect results, and component defect results.

[0191] In this embodiment, the defect relationships in the layered defect association map exist in the form of association pairs, each containing two associated defect region identifiers. All association pairs between solder joint defect regions and circuit defect regions, as well as association pairs between component defect regions and circuit defect regions, are extracted from the map. These association pairs reflect the causal relationships or locational associations between defects in different structural regions.

[0192] The extracted association pairs are labeled to clarify the correspondence between solder joint defects and circuit defects in each association pair. For example, a solder joint void defect is associated with a circuit break defect in a certain section of the circuit. At the same time, the correspondence between component defects and circuit defects is labeled. For example, a component pin deformation defect is associated with a circuit short circuit defect in a certain section of the circuit.

[0193] The annotated relationships are then integrated with the previously generated solder joint defect results, circuit defect results, and component defect results. During the integration process, a related defect identifier is added to each defect result entry, recording information about other related defect result entries. For example, an identifier for a corresponding circuit open circuit defect result entry is added to the solder joint void defect result entry; an identifier for a corresponding circuit short circuit defect result entry is added to the component lead deformation defect result entry.

[0194] By integrating the above methods, a network of relationships between defects is established, so that the results of various defects not only include their own type and location information, but also their relationship information with other defects.

[0195] Step S157: Arrange the integrated defect results and their relationships into a structured report according to the order of the circuit board structure areas, add the circuit board model identifier and inspection time information, and generate a circuit board defect inspection report.

[0196] In this embodiment, after integrating the defect results and their relationships, the integrated information is arranged according to the structural regions of the circuit board. The arrangement order follows the sequence from the circuit area to the solder joint area and then to the component area, consistent with the physical layout of the circuit board and the arrangement order of the hierarchical reinforcement defect feature set.

[0197] The structured report first lists the defect results and their relationships for each circuit area, detailing the defect type, location, and associated solder joint and component defect information for each circuit. Next, following the same format, it lists the defect results and their relationships for each solder joint area, explaining the defect type, location, and associated circuit defect information for each solder joint. Finally, it lists the defect results and their relationships for each component area, presenting the defect type, location, and associated circuit defect information for each component.

[0198] The report clearly illustrates the relationships between defects using connecting lines or corresponding numbers, allowing readers to intuitively understand the connections between different defects. Additionally, the model number of the circuit board under inspection is added at the beginning of the report, determined by the silkscreen information on the board or a pre-defined product number; the inspection time, the system time for completing the defect inspection, is added at the end of the report. After formatting, the report content undergoes format verification to ensure accurate text descriptions, clear charts, and complete information. This process generates a formal circuit board defect inspection report.

[0199] Figure 2The illustration shows exemplary hardware and software components of a deep learning-based circuit board defect detection system 100 that can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the deep learning-based circuit board defect detection system 100 and to perform the functions in this application.

[0200] For example, a deep learning-based circuit board defect detection system 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the deep learning-based circuit board defect detection system 100 may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The deep learning-based circuit board defect detection system 100 also includes an I / O interface 150 between the computer and other input / output devices.

[0201] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A deep learning-based circuit board defect detection method, characterized by, The method comprises: acquiring a layered image set of a circuit board to be detected, the layered image set comprising a solder joint imaging layer, a circuit imaging layer and a component imaging layer, each imaging layer corresponding to an image unit of a different structural region of the circuit board and comprising image units of all solder joints, conductive circuits and patch components respectively; calling a pre-trained circuit board defect hierarchical detection model, inputting the layered image set into a model defect pre-association layer, establishing spatial correspondence of defect regions between the imaging layers through an intra-layer structure association module, generating a layered defect association graph, and the layered defect association graph labeling position mapping relationships of the same defect in different imaging layers; based on the layered defect association graph, extracting defect region exclusive features of each imaging layer, extracting solder joint morphology association parameters in the solder joint imaging layer, extracting circuit texture continuity parameters in the circuit imaging layer, and extracting component pin offset association parameters in the component imaging layer to form a layered exclusive defect feature set; inputting the layered exclusive defect feature set into a model hierarchical feature strengthening layer, and through an intra-layer built-in defect feature attention module, performing targeted strengthening processing on each parameter in the layered exclusive defect feature set to generate a hierarchical strengthened defect feature set; calling a model scene adaptation classification layer, performing circuit board exclusive defect type matching on the hierarchical strengthened defect feature set, outputting solder joint, circuit and component defect results, combining the position mapping relationships of the layered defect association graph, and generating a circuit board defect detection report containing defect types, hierarchical positions and association relationships.

2. The deep learning-based circuit board defect detection method according to claim 1, characterized by, The calling of the pre-trained circuit board defect hierarchical detection model, the inputting of the layered image set into the model defect pre-association layer, the establishment of spatial correspondence of defect regions between the imaging layers through the intra-layer structure association module, and the generation of the layered defect association graph comprise: inputting the solder joint imaging layer, the circuit imaging layer and the component imaging layer in the layered image set into the layer adaptation unit of the structure association module respectively, performing coordinate calibration on the image units of each imaging layer, keeping the image units of different imaging layers aligned, and obtaining coordinate-aligned imaging layers; performing solder joint region positioning on the solder joint imaging layer in the coordinate-aligned imaging layer, extracting the center coordinates and contour ranges of all solder joint image units to form a solder joint coordinate set; performing circuit region positioning on the circuit imaging layer, extracting the center line coordinates and width parameters of all conductive circuit image units to form a circuit coordinate set; and performing component region positioning on the component imaging layer, extracting the pin coordinates and component body contours of all patch component image units to form a component coordinate set; through the spatial mapping unit of the structure association module, based on the physical layer stacking relationship of the circuit board, mapping each solder joint center coordinate in the solder joint coordinate set to the circuit imaging layer to determine corresponding conductive circuit endpoint coordinates, establishing a first spatial association pair of the solder joint and the circuit; and mapping each component pin coordinate in the component coordinate set to the circuit imaging layer to determine corresponding conductive circuit connection point coordinates, establishing a second spatial association pair of the component and the circuit. Based on the first spatial correlation pair and the second spatial correlation pair, a correlation matrix is constructed, the matrix row dimension corresponds to the solder point / component identification, the column dimension corresponds to the circuit identification, and the element represents the spatial distance parameter; The spatial distance parameters in the correlation matrix are threshold filtered, and the correlation pairs smaller than the preset correlation threshold are retained to form an effective correlation set; The effective correlation set is combined with the coordinates of the defect suspected area of each imaging layer to mark the corresponding position of the same defect in each imaging layer, arranged in the order of the physical structure of the circuit board, and a layered defect correlation graph is generated. 3.The deep learning-based circuit board defect detection method of claim 1, wherein, Based on the layered defect correlation graph, the exclusive features of the defect area of each imaging layer are extracted, the solder shape correlation parameters are extracted in the solder imaging layer, the circuit texture continuity parameters are extracted in the circuit imaging layer, and the component pin offset correlation parameters are extracted in the component imaging layer to form a layered exclusive defect feature set, including: The solder correlation sub-graph, the circuit correlation sub-graph and the component correlation sub-graph are separated from the layered defect correlation graph, respectively containing the defect correlation information of solder-circuit, circuit itself and component-circuit; The shape of the solder defect suspected area in the solder correlation sub-graph is analyzed, the outline roundness, the solder melting trace length and the solder hollow area proportion of each area are extracted, the outline roundness change rate, the melting trace length deviation value and the hollow area proportion fluctuation value of the same solder under different shooting frames are calculated, and they are respectively taken as independent feature items in the solder shape correlation parameters; The texture of the circuit defect suspected area in the circuit correlation sub-graph is analyzed, the circuit edge smoothness, the circuit width uniformity and the circuit gray consistency of each area are extracted, the edge smoothness difference, the width uniformity deviation and the gray consistency fluctuation of the same circuit segment at different imaging positions are calculated, and they are respectively taken as independent feature items in the circuit texture continuity parameters; The pin positioning analysis of the component defect suspected area in the component correlation sub-graph is performed, the pin spacing, the alignment of the pin and the circuit connection end and the pin bending angle of each area are extracted, the pin spacing change amount, the alignment deviation value and the bending angle fluctuation value of the same component in different imaging layers are calculated, and they are respectively taken as independent feature items in the component pin offset correlation parameters; The solder shape correlation parameters, the circuit texture continuity parameters and the component pin offset correlation parameters are classified according to the imaging layer and respectively belong to the solder feature subset, the circuit feature subset and the component feature subset; The solder feature subset, the circuit feature subset and the component feature subset are integrated in the order of the structure level of the circuit board to form a layered exclusive defect feature set, and each feature subset in the layered exclusive defect feature set corresponds to the exclusive defect features of a structure region of the circuit board.

4. The deep learning-based circuit board defect detection method according to claim 3, characterized by, The shape of the solder defect suspected area in the solder correlation sub-graph is analyzed, the outline roundness, the solder melting trace length and the solder hollow area proportion of each area are extracted, the outline roundness change rate, the melting trace length deviation value and the hollow area proportion fluctuation value of the same solder under different shooting frames are calculated, and they are combined as the solder shape correlation parameters, including: Extract the contour coordinates of each welding defect suspected area from the welding point correlation sub-atlas, calculate the contour perimeter and area based on the coordinates, and obtain the contour roundness; Perform gray scale analysis on the welding defect suspected area image unit, identify the area with a gray scale value lower than the preset melting threshold as the melting trace area, and extract the longest diameter of the area as the welding melting trace length; Perform binaryzation processing on the welding defect suspected area image unit, distinguish the welding point foreground area from the welding point background area, identify the black pixel area in the welding point foreground area as the hollow area through the hollow detection algorithm, and calculate the ratio of the hollow area to the welding point foreground area as the welding hollow area proportion; Obtain the contour roundness, welding melting trace length, and welding hollow area proportion of the same welding point under different shooting frames, calculate the difference value of the contour roundness between adjacent frames and the ratio of the roundness of the previous frame, and obtain the contour roundness change rate; Calculate the average value of the welding melting trace length of the same welding point under all shooting frames, take the absolute value of the difference between the length under each shooting frame and the average value as the melting trace length deviation value; Calculate the standard deviation of the welding hollow area proportion of the same welding point under all shooting frames as the hollow area proportion fluctuation value; Arrange the contour roundness change rate, melting trace length deviation value, and hollow area proportion fluctuation value of the same welding point in the order of calculation to form the welding shape correlation parameter.

5. The deep learning-based circuit board defect detection method according to claim 3, characterized by, The line edge coordinates of each line defect suspected area are extracted from the line correlation sub-atlas, the edge curve is fitted based on the coordinates, the curvature change amount of the curve is calculated, and the reciprocal of the curvature change amount is taken as the line edge smoothness; A plurality of measurement points are uniformly selected along the line extension direction of the line defect suspected area, a line segment is intercepted perpendicularly at each measurement point, the distance between the intersection points of the line segment and the line edge is taken as the line width of the measurement point, the variance of the line widths of all measurement points is calculated, and the reciprocal of the variance is taken as the line width uniformity; A plurality of sampling areas are uniformly selected within the line range of the line defect suspected area, the average gray scale value of each sampling area is calculated, and the variance of the average gray scale values of all sampling areas is calculated, and the reciprocal of the variance is taken as the line gray scale consistency; The line edge smoothness, line width uniformity, and line gray scale consistency of the same line segment at different imaging positions are obtained, the absolute value of the smoothness difference value between adjacent imaging positions is calculated as the edge smoothness difference value; The average value of the line width uniformity of the same line segment at all imaging positions is calculated, the absolute value of the difference between the uniformity at each imaging position and the average value is taken as the width uniformity deviation; The standard deviation of the line gray scale consistency of the same line segment at all imaging positions is calculated as the gray scale consistency fluctuation; The edge smoothness difference value, width uniformity deviation, and gray scale consistency fluctuation of the same line segment are arranged in the order of calculation to form the line texture continuity parameter. ​ 6.The deep learning-based circuit board defect detection method of claim 1, wherein The inputting of the hierarchical exclusive defect feature set into the model level feature enhancement layer is to respectively perform targeted enhancement processing on each parameter in the hierarchical exclusive defect feature set through an in-layer defect feature attention module, to generate a level-enhanced defect feature set, including: The solder feature subset, the circuit feature subset, and the element feature subset in the hierarchical exclusive defect feature set are respectively input into the solder attention branch, the circuit attention branch, and the element attention branch of the defect feature attention module; In the solder attention branch, the solder feature weight calculation unit in the branch is used to respectively perform standardization and weighting processing on the profile roundness change rate, the molten trace length deviation value, and the hollow area proportion fluctuation value in the solder shape correlation parameter, to generate a solder feature vector set; In the circuit attention branch, the circuit feature weight calculation unit in the branch is used to respectively perform standardization and weighting processing on the circuit edge smoothness difference value, the width uniformity deviation, and the gray consistency fluctuation in the circuit texture continuity parameter, to generate a circuit feature vector set; In the element attention branch, the element feature weight calculation unit in the branch is used to respectively perform standardization and weighting processing on the pin spacing change amount, the alignment deviation value, and the bending angle fluctuation value in the element pin offset correlation parameter, to generate an element feature vector set; The solder feature vector set, the circuit feature vector set, and the element feature vector set are respectively subjected to feature dimension standardization processing, to obtain a standardized feature vector set; Based on the spatial correspondence relationship of the hierarchical defect correlation graph, the standardized feature vector set is subjected to cross-layer correlation fusion, to generate a cross-layer fusion feature vector. All cross-layer fusion feature vectors are integrated in the order of circuit board structure regions, to form a level-enhanced defect feature set, each level-enhanced defect feature in the level-enhanced defect feature set corresponding to the enhanced features of one associated defect region of the circuit board.

7. The deep learning-based circuit board defect detection method according to claim 6, characterized by, Based on the spatial correspondence relationship of the hierarchical defect correlation graph, the standardized feature vector set is subjected to cross-layer correlation fusion, to generate a cross-layer fusion feature vector, including: The identification information of all defect correlation pairs is extracted from the hierarchical defect correlation graph, each defect correlation pair containing a solder identification, a circuit identification, and an element identification, corresponding to the same defect region; According to the identification information, the three types of vectors corresponding to the same defect correlation pair are selected from the standardized feature vector set, to form an associated feature group; The three types of vectors in the associated feature group are subjected to feature connection operation, to obtain a connected cross-layer feature vector; The key feature elements in the connected cross-layer feature vector after standardization are extracted, the key feature elements corresponding to defect judgment key parameters; The key feature elements are subjected to feature enhancement processing, and the non-key feature elements remain unchanged, to obtain an adjusted cross-layer feature vector; The adjusted cross-layer feature vector is bound with the corresponding defect correlation pair identification information, to generate a cross-layer fusion feature vector, the cross-layer fusion feature vector containing the associated identification and the enhanced cross-layer feature information. 8.The deep learning-based circuit board defect detection method of claim 1, wherein, The calling model scene adaptation classification layer matches the hierarchical reinforced defect feature set with the circuit board exclusive defect type, outputs the solder joint, circuit and component defect results, combines the position mapping relationship of the hierarchical defect correlation graph, and generates a circuit board defect detection report containing the defect type, hierarchical position and correlation relationship, including: The hierarchical reinforced defect feature set is input into the feature splitting unit in the model scene adaptation classification layer, and is split into a solder joint-circuit correlation feature subset and a component-circuit correlation feature subset according to the defect correlation type corresponding to the cross-layer fusion feature vector; The solder joint-circuit correlation feature subset is input into the solder joint defect classification unit in the scene adaptation classification layer, and is classified by the solder joint defect classification subnetwork in the unit to output the solder joint defect type, including solder joint void, solder joint fake welding and solder joint tin overflow; The component-circuit correlation feature subset is input into the circuit defect classification unit and the component defect classification unit in the scene adaptation classification layer, the circuit defect classification unit classifies by the circuit defect classification subnetwork in the unit to output the circuit defect type, including circuit short circuit, circuit open circuit and circuit corrosion; the component defect classification unit classifies by the component defect classification subnetwork in the unit to output the component defect type, including component offset, pin deformation and component missing; The imaging layer coordinates of the solder joint imaging layer, the circuit imaging layer and the component imaging layer corresponding to each defect correlation pair in the hierarchical defect correlation graph are extracted, and the imaging layer coordinates are converted into the defect hierarchical position in the circuit board physical coordinate system, and each imaging layer coordinate corresponds to the defect hierarchical position of the solder joint, the circuit and the component; The solder joint defect type is associated with the solder joint defect hierarchical position to form the solder joint defect result, and the circuit defect result and the component defect result are formed in the same way; The defect correlation relationship in the hierarchical defect correlation graph is extracted, and the correlation pairs of the solder joint defect and the circuit defect and the component defect and the circuit defect are labeled, and the correlation relationship is integrated with the solder joint defect result, the circuit defect result and the component defect result; The integrated defect result and the correlation relationship are arranged as structured report content in the order of the circuit board structure area, the circuit board model identifier and the detection time information are added, and the circuit board defect detection report is generated. 9.The deep learning-based circuit board defect detection method of claim 8, wherein, The solder joint-circuit correlation feature subset is input into the input layer of the solder joint defect classification subnetwork, the cross-layer fusion feature vector is dimensionally compressed to obtain a compressed solder joint feature vector; The compressed solder joint feature vector is input into the hidden layer of the subnetwork, and the feature is transformed by the two-layer full connection network in the layer to generate a transformed solder joint feature vector, and the activation function of the full connection network is ReLU function; The transformed solder joint feature vector is input into the output layer of the subnetwork, and the probability values of the transformed solder joint feature vector corresponding to the solder joint void, the solder joint fake welding and the solder joint tin overflow are calculated by the Softmax activation function in the layer to obtain the solder joint defect probability distribution; ​ Extract the defect type with the maximum probability value in the probability distribution of the solder joint defect as the solder joint defect type corresponding to the cross-layer fusion feature vector; Statistical the solder joint defect types corresponding to all cross-layer fusion feature vectors, and record the number of each solder joint defect type and the corresponding defect association pair identifier; Bind the statistical solder joint defect type, the corresponding defect association pair identifier, and the defect hierarchical position to form a solder joint defect result.

10. A deep learning-based circuit board defect detection system, characterized by, The circuit board defect detection system based on deep learning includes a processor and a memory, the memory and the processor are connected, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to realize the circuit board defect detection method based on deep learning in any one of claims 1-9.

Images