Real-time detection and classification method for SMT component defects based on machine vision

Abstract

The application provides a machine vision-based real-time SMT component defect detection and classification method, and relates to the technical field of machine vision, comprising the following steps: acquiring a candidate defect region image, extracting multi-granularity semantic features, and constructing a visual semantic graph; detecting visual contradictions and encoding them into contradiction vectors by calculating the synergistic correlation and current correlation of semantic unit pairs; converting the contradiction vectors into embedding vectors, and classifying defects by using density adaptive clustering; and reconstructing the visual semantic graph when the boundary definition contradiction intensity exceeds a threshold value. The application realizes real-time detection and accurate classification of SMT component defects, and improves the detection accuracy and adaptive ability.

Description

Technical Field

[0001] This invention relates to machine vision technology, and more particularly to a method for real-time detection and classification of defects in SMT components based on machine vision. Background Technology

[0002] Surface mount technology (SMT) is an indispensable process in modern electronics manufacturing. It achieves miniaturization, weight reduction and high reliability of electronic products by directly mounting electronic components onto the surface of printed circuit boards.

[0003] Existing methods mainly rely on predefined rules or static models for defect judgment, lacking a deep understanding of semantic-level relationships and failing to effectively detect anomalies caused by visual-semantic contradictions. As a result, they have low accuracy when dealing with irregular defects in complex scenarios. Summary of the Invention

[0004] A first aspect of the present invention provides a machine vision-based method for real-time detection and classification of defects in SMT components, comprising: Obtain candidate defect region images of SMT components, extract multi-granularity semantic features from the candidate defect region images, and construct a visual semantic map from the multi-granularity semantic features. The semantic unit pairs in the visual semantic graph are traversed, and the co-correlation of each semantic unit pair in the preset normal sample set and the current correlation in the candidate defect region image are calculated. When the co-correlation and the current correlation have opposite signs, it is determined that there is a visual contradiction in the semantic unit pair, and the detected visual contradiction is encoded as a contradiction vector. The contradiction vector is input into the contradiction pattern encoder to obtain the contradiction pattern embedding vector, and the contradiction pattern encoder is trained through a self-supervised contrastive learning framework. In the embedding space formed by the contradictory pattern embedding vectors, a density-adaptive clustering algorithm is used for dynamic clustering, and defect classification is performed based on the positional relationship between the contradictory pattern embedding vectors and the cluster centers. When the contradiction intensity of semantic features related to boundary clarity in the contradiction vector exceeds a preset intensity threshold, the visual semantic map is reconstructed based on the boundary positioning deviation information.

[0005] The steps of acquiring candidate defect region images of SMT components, extracting multi-granularity semantic features from the candidate defect region images, and constructing a visual semantic map from the multi-granularity semantic features include: The candidate defect region image is decomposed into multiple scales, and semantic features are extracted at different scales to obtain multi-granular semantic features; Each semantic feature in the multi-granularity semantic features is mapped to a semantic unit of the visual semantic graph, and node attributes are assigned to each semantic unit based on the distribution characteristics of the semantic features in the feature space; The semantic correlation between semantic units is calculated. When the semantic correlation between two semantic units meets the association threshold, a connection edge is established in the visual semantic graph. The weight of the edge is determined according to the strength of the semantic correlation. The semantic correlation is obtained by measuring the feature similarity between the semantic features corresponding to the semantic units and the spatial position relationship of the semantic units in the candidate defect region image.

[0006] The steps of traversing the semantic unit pairs in the visual semantic graph, calculating the co-correlation of each semantic unit pair in a preset normal sample set and the current correlation in the candidate defect region image, determining that there is a visual contradiction in the semantic unit pair when the co-correlation and the current correlation have opposite signs, and encoding the detected visual contradiction into a contradiction vector include: Semantic unit pairs are extracted from the visual semantic map; the joint distribution of feature response values ​​of the semantic unit pairs in a preset normal sample set is statistically analyzed; the dominant change trend of the joint distribution is fitted to obtain a cooperative direction vector; and the sign and value of the cooperative correlation are determined based on the quadrant position of the cooperative direction vector. The semantic unit is extracted to form the feature response value in the candidate defect region image to construct the current response vector, and the sign and value of the current correlation are determined according to the quadrant position of the current response vector; When the sign of the co-correlation is opposite to the sign of the current correlation, it is determined that there is a visual contradiction in the semantic unit pair, and the absolute value of the difference between the co-correlation value and the current correlation value is calculated as the contradiction intensity. Semantic unit pairs with visual contradictions are encoded as contradiction vectors, where the dimension value of the contradiction vector corresponds to the contradiction intensity. At the same time, the topological position of the semantic unit pair in the visual semantic graph and the semantic granularity level to which it belongs are recorded as the structural information of the contradiction vector.

[0007] The steps of statistically analyzing the joint distribution of the feature response values ​​of the semantic units in a preset set of normal samples, fitting the dominant trend of the joint distribution to obtain a cooperative direction vector, and determining the sign and value of the cooperative correlation based on the quadrant position of the cooperative direction vector include: The feature response values ​​of the semantic unit pairs are extracted from a preset set of normal samples, and a two-dimensional response value pair composed of the feature response values ​​of the semantic unit pairs is constructed. The two-dimensional response value pair is represented as data points in a two-dimensional coordinate space, and the joint probability distribution of all data points is statistically analyzed to form the joint distribution of the semantic unit pairs. Principal component analysis is performed on the joint distribution to fit the dominant trend. The direction of the first principal component of the data point distribution is extracted as the cooperative direction vector, which corresponds to the projection direction with the largest variance in the joint distribution. The quadrant of the cooperative direction vector is determined according to its orientation angle in the two-dimensional coordinate space. The first and third quadrants indicate positive cooperative correlation, while the second and fourth quadrants indicate negative cooperative correlation. The degree of dispersion of the data points projected along the cooperative direction vector is used as the cooperative strength. The cooperative strength is then normalized to a preset numerical range and used as the value of the cooperative correlation.

[0008] The steps of inputting the contradiction vector into the contradiction pattern encoder to obtain the contradiction pattern embedding vector, and training the contradiction pattern encoder through a self-supervised contrastive learning framework, include: Based on the structural symmetry of SMT components, the topological positions in the contradiction vector are mirrored according to the symmetry axis of the SMT component to obtain a mirror contradiction vector while keeping the contradiction intensity unchanged. The original contradiction vector and the mirror contradiction vector are paired as positive sample pairs, and negative samples are selected from the contradiction vectors extracted from the candidate defect region images of different batches of SMT components to form training sample pairs for the self-supervised contrastive learning framework. The contradictory pattern encoder is trained using the training sample pairs; the constraint weight coefficients of the training sample pairs in the contrastive learning loss function are determined based on the contradictory intensity value; in the loss function optimization, the constraint weight coefficients serve as scaling factors for the distance adjustment magnitude of the sample pairs in the embedding space, realizing a monotonically increasing mapping relationship between the contradictory intensity value and the distance adjustment magnitude; The contradiction vector is input into the trained contradiction pattern encoder and mapped into a contradiction pattern embedding vector through multi-layer nonlinear transformation. The mapping process encodes the distribution pattern of the contradiction intensity, the topological positional relationship, and the equivalence of contradictions in symmetrical positions.

[0009] The steps of dynamically clustering in the embedding space formed by the contradictory pattern embedding vectors using a density-adaptive clustering algorithm, and classifying defects based on the positional relationship between the contradictory pattern embedding vectors and the cluster centers, include: The local density of each contradictory pattern embedding vector is calculated in the embedding space. The local density is determined by counting the number of other contradictory pattern embedding vectors within the neighborhood radius centered on the target contradictory pattern embedding vector. When contradictory pattern embedding vectors of a new batch of SMT components are added to the embedding space, the neighborhood radius is dynamically adjusted based on the overall distribution density of contradictory pattern embedding vectors in the embedding space. Cluster centers are identified based on the distance between the local density and the contradictory pattern embedding vectors; the density determination criteria for cluster center identification are dynamically adjusted according to the defect occurrence frequency of the SMT component production line, and the density determination criteria are reduced to form new cluster centers when the contradictory pattern embedding vectors of new types of defects continue to increase. Each contradictory pattern embedding vector in the embedding space is assigned to the nearest cluster center to form a dynamic clustering result; the defect is classified based on the dynamic clustering result.

[0010] When the contradiction intensity of semantic features related to boundary clarity in the contradiction vector exceeds a preset intensity threshold, the step of reconstructing the visual semantic map based on boundary positioning deviation information includes: Filter the contradictory terms in the contradictory vector that involve semantic features related to boundary clarity, and extract the contradiction intensity value corresponding to the contradictory term; when the contradiction intensity value exceeds a preset intensity threshold, backtrack to the boundary region in the candidate defect region image where the contradiction occurred; Subpixel-level boundary localization is performed in the boundary region. The detected actual boundary position is spatially registered with the expected position of the corresponding boundary in a preset normal sample set. The normal offset distance and tangential offset distance of the actual boundary position relative to the expected position are calculated to form the boundary localization deviation information. The set of semantic units that need to be corrected in the visual semantic graph is identified based on the boundary positioning deviation information. For each semantic unit in the set of semantic units, the boundary positioning deviation information is converted into a compensation vector of node attributes, and the original node attributes are compensated and corrected. Based on the corrected node attributes, the semantic correlation between semantic units is recalculated and the weight of the connecting edges is updated to complete the adaptive reconstruction of the visual semantic graph.

[0011] A second aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0012] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0013] This invention employs a method of visual contradiction detection and contradiction pattern encoding, which can effectively capture the difference features between defective regions and normal samples. By training the contradiction pattern encoder through a self-supervised contrastive learning framework, it achieves accurate classification of different types of defects, improves the generalization ability of defect recognition, and adapts to complex and ever-changing production environments. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the real-time detection and classification method for SMT component defects based on machine vision, according to an embodiment of the present invention. Figure 2 Flowchart for contradiction identification. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0017] Figure 1 This is a flowchart illustrating the real-time detection and classification method for SMT component defects based on machine vision, as described in an embodiment of the present invention. Figure 1 As shown, the method includes: Obtain candidate defect region images of SMT components, extract multi-granularity semantic features from the candidate defect region images, and construct a visual semantic map from the multi-granularity semantic features. The semantic unit pairs in the visual semantic graph are traversed, and the co-correlation of each semantic unit pair in the preset normal sample set and the current correlation in the candidate defect region image are calculated. When the co-correlation and the current correlation have opposite signs, it is determined that there is a visual contradiction in the semantic unit pair, and the detected visual contradiction is encoded as a contradiction vector. The contradiction vector is input into the contradiction pattern encoder to obtain the contradiction pattern embedding vector, and the contradiction pattern encoder is trained through a self-supervised contrastive learning framework. In the embedding space formed by the contradictory pattern embedding vectors, a density-adaptive clustering algorithm is used for dynamic clustering, and defect classification is performed based on the positional relationship between the contradictory pattern embedding vectors and the cluster centers. When the contradiction intensity of semantic features related to boundary clarity in the contradiction vector exceeds a preset intensity threshold, the visual semantic map is reconstructed based on the boundary positioning deviation information.

[0018] In one optional implementation, the steps of acquiring candidate defect region images of SMT components, extracting multi-granularity semantic features from the candidate defect region images, and constructing a visual semantic map from the multi-granularity semantic features include: The candidate defect region image is decomposed into multiple scales, and semantic features are extracted at different scales to obtain multi-granular semantic features; Each semantic feature in the multi-granularity semantic features is mapped to a semantic unit of the visual semantic graph, and node attributes are assigned to each semantic unit based on the distribution characteristics of the semantic features in the feature space; The semantic correlation between semantic units is calculated. When the semantic correlation between two semantic units meets the association threshold, a connection edge is established in the visual semantic graph. The weight of the edge is determined according to the strength of the semantic correlation. The semantic correlation is obtained by measuring the feature similarity between the semantic features corresponding to the semantic units and the spatial position relationship of the semantic units in the candidate defect region image.

[0019] For example, candidate defect region images of SMT components can be obtained by acquiring surface images of SMT components using a high-precision industrial camera. The acquired images have a resolution of 4096×3000 pixels, an acquisition distance of 15 cm, and illumination conditions controlled at 800-1000 lux. After acquisition, the original images are scanned using a sliding window method, with a window size of 224×224 pixels and a stride of 32 pixels. A pre-trained defect detection network is then used to initially filter out defective regions, forming a candidate defect region image set.

[0020] A pyramid structure is used to process candidate defect region images, specifically generating feature maps at three different scales. The original candidate defect region image is 224×224 pixels, and bilinear interpolation is used to generate reduced images of 112×112 pixels and 56×56 pixels, respectively. A feature extraction network is then used to process these three images at different scales. The feature extraction network employs a modified VGG structure, containing five convolutional blocks, each containing 2-3 convolutional layers and a max-pooling layer. For images at a scale of 224×224, the output features of the 1st, 3rd, and 5th convolutional blocks are extracted, with feature dimensions of 64×224×224, 256×56×56, and 512×28×28, respectively. For images at a scale of 112×112, the output features of the 3rd and 5th convolutional blocks are extracted, with feature dimensions of 256×28×28 and 512×14×14, respectively. For images at a scale of 56×56, the output features of the 5th convolutional block are extracted, with a feature dimension of 512×7×7. This yields a total of 6 semantic features at different granularities, capturing multi-level information from low-level texture features to high-level semantic features.

[0021] Adaptive region partitioning is performed on each feature map. Specifically, for large feature maps (e.g., 64×224×224), an 8×8 region partitioning method is used; for medium-sized feature maps (e.g., 256×56×56 and 256×28×28), a 4×4 region partitioning method is used; and for small feature maps (e.g., 512×28×28, 512×14×14, and 512×7×7), a 2×2 region partitioning method is used. The feature vectors within each region are merged into a single feature vector through average pooling, forming a semantic unit. In this way, a total of 8×8 + 4×4 + 4×4 + 2×2 + 2×2 + 2×2 = 92 semantic units are obtained.

[0022] The assignment of node attributes considers three aspects: feature strength, feature variance, and spatial location. Feature strength is calculated by averaging the values ​​of each dimension of the feature vector within the semantic unit's corresponding region, resulting in a scalar value normalized to [0,1]. Feature variance is calculated by averaging the standard deviations of each dimension of the feature vector within the semantic unit's corresponding region, reflecting the consistency of the region's features, with a value normalized to [0,1]. Spatial location is represented by the semantic unit's normalized coordinates (x,y) in the original image, both ranging from [0,1]. These three attributes together constitute the node attribute vector of the semantic unit.

[0023] The semantic relevance between semantic units comprehensively considers feature similarity and spatial relationship. Feature similarity is calculated using cosine similarity, with a value ranging from -1 to 1; a larger value indicates greater similarity in the features of the two semantic units. Specifically, for the feature vectors F_A and F_B of two semantic units A and B, their feature similarity is calculated as the inner product of the two vectors divided by the product of their respective norms. Spatial relationship is obtained by calculating the negative exponent of the Euclidean distance between the center points of the two semantic units in the normalized coordinate system, with a value ranging from 0 to 1; a larger value indicates closer spatial positions of the two units. The comprehensive semantic relevance is obtained through weighted summation, with a weight of 0.7 for spatial relationship and 0.3 for feature similarity.

[0024] When the combined semantic relevance of two semantic units exceeds the set association threshold of 0.6, a connection edge is established in the visual semantic graph. The weight of the edge directly adopts the combined semantic relevance value, reflecting the association strength between the two semantic units. For example, in practical applications, for a sample with a solder joint defect, the semantic relevance between the solder joint area and the surrounding normal area is usually between 0.35 and 0.55, which is significantly lower than the semantic relevance between different parts within the normal solder joint area (usually between 0.75 and 0.9). Therefore, by setting a reasonable threshold, the boundary between the defective area and the normal area can be effectively distinguished.

[0025] The visual semantic map of this invention not only preserves the local details of the image but also captures the semantic relationships between different regions, providing rich structured information for subsequent defect classification and localization, and significantly improving the accuracy of defect detection.

[0026] In one optional implementation, the steps of traversing semantic unit pairs in the visual semantic map, calculating the co-correlation of each semantic unit pair in a preset normal sample set and the current correlation in the candidate defect region image, determining that there is a visual contradiction in the semantic unit pair when the co-correlation is opposite to the current correlation, and encoding the detected visual contradiction into a contradiction vector include: Semantic unit pairs are extracted from the visual semantic map; the joint distribution of feature response values ​​of the semantic unit pairs in a preset normal sample set is statistically analyzed; the dominant change trend of the joint distribution is fitted to obtain a cooperative direction vector; and the sign and value of the cooperative correlation are determined based on the quadrant position of the cooperative direction vector. The semantic unit is extracted to form the feature response value in the candidate defect region image to construct the current response vector, and the sign and value of the current correlation are determined according to the quadrant position of the current response vector; When the sign of the co-correlation is opposite to the sign of the current correlation, it is determined that there is a visual contradiction in the semantic unit pair, and the absolute value of the difference between the co-correlation value and the current correlation value is calculated as the contradiction intensity. Semantic unit pairs with visual contradictions are encoded as contradiction vectors, where the dimension value of the contradiction vector corresponds to the contradiction intensity. At the same time, the topological position of the semantic unit pair in the visual semantic graph and the semantic granularity level to which it belongs are recorded as the structural information of the contradiction vector.

[0027] Combination Figure 2 The flowchart for contradiction identification is illustrated below. For example, in visual contradiction detection, the first step is to extract semantic unit pairs from the constructed visual semantic map. For instance, the visual semantic map contains 92 semantic units, which are distributed across 6 semantic levels of different granularities.

[0028] When extracting semantic unit pairs from a visual semantic map, two strategies are employed: first, extracting spatially adjacent semantic unit pairs within the same semantic granularity level; and second, extracting semantic unit pairs corresponding to spatial locations across different semantic granularity levels. For extraction within the same level, taking a 4×4 region partition corresponding to a 256×56×56 feature layer as an example, a semantic unit at location (2,3) in a certain region can form a semantic unit pair with semantic units at adjacent locations (2,4), (3,3), etc. For cross-level extraction, taking a semantic unit at location (2,3) in a 256×56×56 feature layer as an example, it can form a cross-level semantic unit pair with a semantic unit at the corresponding spatial location region (1,1) in a 512×28×28 feature layer. This cross-level pairing can capture abnormal associations between different levels of semantic abstraction. By traversing all semantic unit combinations in the visual semantic map, theoretically 92×91 / 2=4186 semantic unit pairs can be extracted for a visual semantic map containing 92 semantic units. However, in practical applications, by setting spatial distance thresholds and hierarchical association rules for filtering, usually 200-500 effective semantic unit pairs are extracted from each candidate defect region image for subsequent contradiction detection.

[0029] After extracting semantic unit pairs, it is necessary to statistically analyze the joint distribution of the feature response values ​​of these unit pairs in a preset normal sample set. The preset normal sample set contains 1000 defect-free SMT component images, all of which have been constructed with corresponding visual semantic maps through the same multi-granularity semantic feature extraction process. For each semantic unit pair (A, B), its feature response values ​​in all normal samples are extracted. The feature response value is defined as the feature intensity component in the semantic unit node attribute vector, which has been normalized to the [0,1] interval in the aforementioned implementation. For the semantic unit pair (A, B), the feature response values ​​extracted from the 1000 normal samples form a two-dimensional data point set {(a1,b1),(a2,b2),...,(a...b1)}. 1000 ,b 1000 )}, where aᵢ represents the feature intensity of semantic unit A in the i-th normal sample, and bᵢ represents the feature intensity of semantic unit B in the i-th normal sample. The distribution of these two-dimensional data points in the feature response value coordinate system reflects the cooperative change pattern of the two semantic units under normal conditions.

[0030] To fit the dominant trend of the joint distribution, principal component analysis was performed on the two-dimensional data point set. First, the mean vector of the data point set was calculated. Then, a decentralized data matrix was constructed, and its covariance matrix was calculated. The covariance matrix is ​​a 2×2 matrix. By solving for the eigenvalues ​​and eigenvectors of this matrix, two principal component directions were obtained. The first principal component direction corresponds to the largest eigenvalue, which is the co-directional vector, representing the main trend of data change. For example, for a semantic unit pair consisting of the center and edge regions of a solder joint, the co-directional vector in normal samples is (0.82, 0.57), indicating that the feature strengths of the two regions tend to increase or decrease simultaneously in normal samples, exhibiting a positively correlated co-directional pattern.

[0031] The co-direction vector can be represented in the form (v1, v2). The quadrant position is determined by the sign combination of v1 and v2. When v1 > 0 and v2 > 0, the vector is in the first quadrant, indicating a positive correlation with a positive sign. When v1 < 0 and v2 < 0, the vector is in the third quadrant, also indicating a positive correlation with a positive sign. When v1 > 0 and v2 < 0, the vector is in the fourth quadrant, indicating a negative correlation with a negative sign. When v1 < 0 and v2 > 0, the vector is in the second quadrant, also indicating a negative correlation with a negative sign. The co-correlation value is obtained by calculating the product of v1 and v2. For the co-direction vector (0.82, 0.57), its co-correlation value is 0.82 × 0.57 = 0.47, with a positive sign, indicating that the two semantic units exhibit a strong positive correlation co-correlation pattern in normal samples.

[0032] For a candidate defect region image to be detected, the feature response values ​​of the same semantic unit pair (A, B) in the corresponding visual semantic map of the image are extracted to form the current response vector. The current response vector consists of two components: (a_current, b_current), where a_current is the feature intensity of semantic unit A in the current image, and b_current is the feature intensity of semantic unit B in the current image. For example, for the semantic unit pair consisting of the center region and the edge region of the solder joint, in a candidate defect image suspected of having a cold solder joint, its current response vector is (0.88, -0.52), indicating that the feature intensity of the center region of the solder joint is enhanced to 0.88, while the feature intensity of the edge region is abnormally reduced to the negative value region -0.52.

[0033] The sign and value of the current correlation are determined based on the quadrant position of the current response vector. The method is the same as for the co-direction vector, determining the quadrant by analyzing the sign combination of a_current and b_current. For the current response vector (0.88, -0.52), since a_current > 0 and b_current < 0, this vector is located in the fourth quadrant, indicating a negative correlation, with a negative sign. The current correlation value is calculated as 0.88 × (-0.52) = -0.46. Comparing the current correlation (negative sign, value -0.46) with the co-correlation of normal samples (positive sign, value 0.47), the signs are found to be opposite—one is positive and the other is negative. Therefore, it is determined that there is a visual contradiction in this semantic unit pair, indicating an abnormal deviation in the co-correlation relationship between the solder joint center and the edge.

[0034] The absolute value of the difference between the co-correlation value and the current correlation value is calculated as the inconsistency strength. In the above case of poor soldering, the inconsistency strength is |0.47-(-0.46)|=0.93. Theoretically, the range of inconsistency strength is [0,2], where 0 represents no inconsistency and 2 represents an extreme inconsistency of complete opposite. The greater the inconsistency strength, the more serious the deviation of the currently observed semantic unit co-correlation relationship from the normal situation, and the higher the probability of the existence of a defect. In practical applications, an inconsistency strength exceeding 0.6 is generally considered a significant inconsistency and requires special attention.

[0035] Semantic unit pairs exhibiting visual contradictions are encoded as contradiction vectors. A contradiction vector is a structured data representation containing three parts: contradiction intensity, topological location, and semantic granularity level. The contradiction intensity, as the principal dimension value of the contradiction vector, directly records the calculated value, such as 0.93. The topological location information records the specific position of the semantic unit pair in the visual semantic map. Taking a 4×4 region division corresponding to a 256×56×56 feature layer as an example, if semantic unit A is located in region (2,3) and semantic unit B is located in region (2,4), then the topological location is recorded as [(level identifier: L3, region coordinates: (2,3)), (level identifier: L3, region coordinates: (2,4))], where L3 represents the number of this semantic level among the six semantic granularity levels. The semantic granularity level information records the degree of abstraction of the semantic unit to its corresponding feature, which can be divided into three categories: "fine-grained," "medium-grained," and "coarse-grained," corresponding to low-level texture features, mid-level structural features, and high-level semantic features, respectively. For cross-level semantic unit pairs, the granularity level of each semantic unit is recorded. This structural information, together with the contradiction intensity, constitutes a complete contradiction vector representation, in the form of: {intensity: 0.93, position: [(L3,2,3), (L3,2,4)], granularity: "medium granularity"}, which provides rich feature descriptions for subsequent contradiction pattern encoding and clustering classification.

[0036] In practical applications, for a candidate region image suspected of having a defect, by traversing 200-500 semantic unit pairs extracted from the visual semantic map, 10-30 semantic unit pairs with visual inconsistencies are detected. For example, in an image of a cold solder joint defect in an SMT component, the semantic unit pair with a contradiction intensity of 0.93 is found between the center and edge regions of the solder joint, the semantic unit pair with a contradiction intensity of 0.87 is found at the junction of the solder joint and the pad, and the semantic unit pair with a contradiction intensity of 0.76 is found in different regions within the solder joint. By comprehensively analyzing the spatial distribution pattern, contradiction intensity distribution, and semantic granularity correlation of these contradictory vectors, the core region of the defect can be accurately located, providing a basis for subsequent defect type determination.

[0037] This invention effectively detects abnormal patterns in candidate defect region images by quantitatively analyzing changes in the collaborative relationships between visual semantic units. The advantage of this method is that it does not require a large number of defect samples for training, making it suitable for the common problems of scarce and imbalanced defect samples in industrial visual inspection.

[0038] In one optional implementation, the steps of statistically analyzing the joint distribution of the semantic unit's feature response values ​​in a preset normal sample set, fitting the dominant trend of the joint distribution to obtain a cooperative direction vector, and determining the sign and value of the cooperative correlation based on the quadrant position of the cooperative direction vector include: The feature response values ​​of the semantic unit pairs are extracted from a preset set of normal samples, and a two-dimensional response value pair composed of the feature response values ​​of the semantic unit pairs is constructed. The two-dimensional response value pair is represented as data points in a two-dimensional coordinate space, and the joint probability distribution of all data points is statistically analyzed to form the joint distribution of the semantic unit pairs. Principal component analysis is performed on the joint distribution to fit the dominant trend. The direction of the first principal component of the data point distribution is extracted as the cooperative direction vector, which corresponds to the projection direction with the largest variance in the joint distribution. The quadrant of the cooperative direction vector is determined according to its orientation angle in the two-dimensional coordinate space. The first and third quadrants indicate positive cooperative correlation, while the second and fourth quadrants indicate negative cooperative correlation. The degree of dispersion of the data points projected along the cooperative direction vector is used as the cooperative strength. The cooperative strength is then normalized to a preset numerical range and used as the value of the cooperative correlation.

[0039] For example, the feature response value is defined as the feature intensity component in the semantic unit node attribute vector, which has been normalized to the [0,1] interval. For a given semantic unit pair, the feature intensity value of each sample pair of the two semantic units is extracted from a preset sample set containing 1000 normal samples. Specifically, for the i-th normal sample, its visual semantic map is first constructed according to the aforementioned implementation method, and then the corresponding nodes of semantic units A and B in the visual semantic map are located. The feature intensity attribute values ​​of these two nodes are extracted and recorded as the response value of the sample to semantic unit A and the response value to semantic unit B, respectively, thus forming a two-dimensional response value pair. For example, for the semantic unit pair consisting of the center region and the edge region of the solder joint, the response value pair extracted in the first normal sample is (0.78, 0.82), in the second normal sample it is (0.81, 0.85), in the third normal sample it is (0.76, 0.79), and so on. By traversing all 1000 normal samples, 1000 such two-dimensional response value pairs can be obtained, all of which fall within a unit square region of [0,1]×[0,1].

[0040] These two-dimensional response pairs are represented as data points in a two-dimensional coordinate space. In this space, the horizontal axis represents the feature response value of semantic unit A, and the vertical axis represents the feature response value of semantic unit B. Each two-dimensional response pair corresponds to a point on the coordinate plane. Specifically, a grid partitioning method can be used for joint probability density estimation. The region [0,1]×[0,1] is divided into grids. For example, both the horizontal and vertical axes are divided into 20 equally spaced intervals, forming 20×20 grid cells, totaling 400 grid cells. Each grid cell has a side length of 0.05. The number of data points falling into each grid cell is then counted and divided by the total sample size of 1000 to obtain the frequency, which is used as the joint probability density estimate for that grid region. For example, if there are 35 data points in a grid cell with coordinates [0.75, 0.80)×[0.80, 0.85), the joint probability density estimate for that region is 35 / 1000 = 0.035. A smoother distribution estimate can also be obtained using kernel density estimation. Specifically, taking a Gaussian kernel as an example, a two-dimensional Gaussian distribution function is assigned to each data point. This function reaches its maximum value at that data point and decays towards the surrounding area, with the decay rate controlled by a bandwidth parameter. The two-dimensional Gaussian distribution functions corresponding to all 1000 data points are superimposed to obtain the overall joint probability distribution estimate. For the semantic unit pairs at the center and edge of the solder joint, if the data points are mainly concentrated in the diagonal band region from (0.75, 0.80) to (0.85, 0.90), it indicates that these two semantic units tend to simultaneously take higher or lower values ​​in normal samples, exhibiting a positively correlated joint distribution pattern.

[0041] Principal component analysis (PCA) was performed on the joint distribution to fit the dominant trend. The specific steps of PCA are as follows: First, the mean vector of 1000 two-dimensional response value pairs was calculated. For the response value sequence of semantic unit A, the sum of all sample response values ​​was calculated and divided by 1000 to obtain the mean. The same mean was calculated for the response value sequence of semantic unit B, resulting in the two components of the mean vector. Second, the mean vector was subtracted from each data point to obtain the centered data. For the i-th data point, the centering result was the two coordinate values ​​of that point minus the corresponding mean. Third, the 2×2 covariance matrix of the centered data was calculated. This matrix has four elements: the element in the first row and first column is equal to the sum of the squares of the centered values ​​of all samples in the direction of semantic unit A divided by 1000; the element in the first row and second column is equal to the sum of the products of the centered values ​​of all samples in the directions of semantic unit A and semantic unit B divided by 1000; the element in the second row and first column is equal to the element in the first row and second column; and the element in the second row and second column is equal to the sum of the squares of the centered values ​​of all samples in the direction of semantic unit B divided by 1000. The fourth step involves eigenvalue decomposition of the covariance matrix, solving the characteristic equation to obtain two eigenvalues. One eigenvalue is larger and designated as the largest eigenvalue, while the other is smaller. For each eigenvalue, the corresponding system of linear equations is solved to obtain the corresponding eigenvector. The fifth step selects the eigenvector corresponding to the largest eigenvalue as the first principal component direction, i.e., the co-directional direction vector. This vector contains two components corresponding to the directions of semantic units A and B, respectively.

[0042] For example, for semantic unit pairs of solder joint centers and edges, assuming the four elements of the calculated covariance matrix are 0.0082, 0.0071, 0.0071, and 0.0063, the largest eigenvalue is 0.0141 and the smallest eigenvalue is 0.0004, obtained through eigenvalue decomposition. The two components of the eigenvector corresponding to the largest eigenvalue are 0.82 and 0.57, respectively. Therefore, the cooperative direction vector has a first component of 0.82 and a second component of 0.57. The cooperative direction vector indicates that the projection dispersion of data points is greatest when observed along this direction, i.e., this direction is the dominant direction of data variation. In actual calculations, the specific values ​​of the two components of the eigenvector depend on the calculation result of the covariance matrix. Usually, it is not necessary to normalize the eigenvector; the original eigenvector components can be used directly.

[0043] Let the first component of the cooperative direction vector correspond to the direction of semantic unit A, and the second component correspond to the direction of semantic unit B. The determination method is as follows: when the first component > 0 and the second component > 0, the vector is located in the first quadrant, indicating that both semantic units are increasing simultaneously, and the cooperative correlation sign is positive; when the first component < 0 and the second component < 0, the vector is located in the third quadrant, indicating that both semantic units are decreasing simultaneously, essentially still changing in the same direction, and the cooperative correlation sign is positive; when the first component > 0 and the second component < 0, the vector is located in the fourth quadrant, indicating that semantic unit A increases while B decreases, and the cooperative correlation sign is negative; when the first component < 0 and the second component > 0, the vector is located in the second quadrant, indicating that semantic unit A decreases while B increases, and the cooperative correlation sign is negative. For the cooperative direction vector of the semantic unit pair between the solder joint center and the edge, the first component is 0.82 > 0, and the second component is 0.57 > 0. This vector is located in the first quadrant, therefore the cooperative correlation sign is positive. For a pair of semantic units that have a complementary relationship, if the first component of their cooperative direction vector is 0.75 and the second component is -0.66, since the first component is >0 but the second component is <0, the vector is located in the fourth quadrant. Therefore, the cooperative correlation sign is negative, indicating that the two semantic units exhibit a negative cooperative pattern in normal samples.

[0044] For the first and second components of the cooperative direction vector, the product of these two components is calculated as the original cooperative strength value. The sign of this product is consistent with the sign determination result of the aforementioned cooperative correlation: the product is positive when the two components have the same sign, and negative when the two components have opposite signs. The absolute value of this product represents the magnitude of the cooperative strength; the larger the value, the stronger the cooperative relationship between the two semantic units in the dominant direction of change. For the semantic unit pair between the center and edge of the solder joint, the first component of the cooperative direction vector is 0.82 and the second component is 0.57, and its cooperative correlation value is 0.82 × 0.57 = 0.467. Since the numerical range of the cooperative direction vectors of different semantic unit pairs varies, the cooperative correlation value can be normalized for easier subsequent comparison. In practical applications, to maintain the interpretability of the values, the original component product form of the cooperative correlation value is usually retained; for example, the cooperative correlation value of the solder joint case above is 0.467.

[0045] To further quantify the strength of collaboration, the dispersion of data points projected along the collaboration direction vector is used as a supplementary metric. Specifically, for each centered data point, its projected length along the collaboration direction vector is calculated. The projection length is calculated by multiplying the two centered coordinate values ​​of the data point by their corresponding components of the collaboration direction vector, summing the results, and then dividing by the magnitude of the collaboration direction vector for normalization. Here, the magnitude of the collaboration direction vector is equal to the square root of the sum of the squares of its two components. By calculating the projection length values ​​of 1000 data points, the distribution of data points along the dominant direction of change can be obtained. Then, the standard deviation of these projection length values ​​is calculated: first, the mean of the projection length values ​​is calculated; then, the square of the difference between each projection length value and the mean is calculated; all squares are summed, divided by 1000, and finally, the square root of the result is taken to obtain the standard deviation. The standard deviation reflects the dispersion of data points along the collaboration direction; a larger value indicates a greater amplitude of data change in that direction, and a stronger collaboration strength.

[0046] In fact, according to the properties of principal component analysis, the variance of the projected length values ​​is exactly equal to the corresponding largest eigenvalue; therefore, the standard deviation of the projected length is equal to the square root of the largest eigenvalue. For the example of the semantic unit pair between the solder joint center and the edge, if the largest eigenvalue is 0.0141, then the standard deviation of the projected length is equal to the square root of 0.0141, which is 0.119. This value can serve as another measure of cooperative strength, complementing the aforementioned component product method, to jointly describe the cooperative pattern characteristics of semantic unit pairs in normal samples.

[0047] Through the above steps, the joint distribution of semantic unit response values ​​in a pre-defined normal sample set can be systematically statistically analyzed, and the dominant trend can be fitted using principal component analysis to obtain a cooperative direction vector. The sign of the cooperative correlation is determined based on the quadrant position of this vector, the numerical value of the cooperative correlation is determined based on the product of the vector components, and the cooperative strength is quantified by the standard deviation of the projection length. This provides an accurate benchmark for normal cooperative patterns for subsequent visual contradiction detection and defect identification. This method fully utilizes the statistical correlation patterns between semantic units in normal samples, transforming complex visual semantic relationships into quantifiable numerical features through mathematical modeling, enabling abnormal patterns to be effectively identified based on their deviation from normal cooperative patterns.

[0048] In one optional implementation, the contradictory vector is input into the contradictory pattern encoder to obtain the contradictory pattern embedding vector. The contradictory pattern encoder is trained using a self-supervised contrastive learning framework, which includes the following steps: Based on the structural symmetry of SMT components, the topological positions in the contradiction vector are mirrored according to the symmetry axis of the SMT component to obtain a mirror contradiction vector while keeping the contradiction intensity unchanged. The original contradiction vector and the mirror contradiction vector are paired as positive sample pairs, and negative samples are selected from the contradiction vectors extracted from the candidate defect region images of different batches of SMT components to form training sample pairs for the self-supervised contrastive learning framework. The contradictory pattern encoder is trained using the training sample pairs; the constraint weight coefficients of the training sample pairs in the contrastive learning loss function are determined based on the contradictory intensity value; in the loss function optimization, the constraint weight coefficients serve as scaling factors for the distance adjustment magnitude of the sample pairs in the embedding space, realizing a monotonically increasing mapping relationship between the contradictory intensity value and the distance adjustment magnitude; The contradiction vector is input into the trained contradiction pattern encoder and mapped into a contradiction pattern embedding vector through multi-layer nonlinear transformation. The mapping process encodes the distribution pattern of the contradiction intensity, the topological positional relationship, and the equivalence of contradictions in symmetrical positions.

[0049] For example, the training of the contradictory pattern encoder is based on a self-supervised contrastive learning framework. The core idea is to utilize the structural symmetry of SMT components to construct positive sample pairs. Through contrastive learning, the encoder can identify the essential features of contradictory patterns while ignoring positional changes. SMT components typically have a clear axis of symmetry, such as the center line of the major axis of a rectangular surface mount resistor, the two diagonals of a square integrated circuit chip, or any diameter line of a circular capacitor. For a candidate defect region image of a rectangular surface mount resistor, its visual semantic map contains 92 semantic units. Assuming the detected contradictory vector is {intensity: 0.93, position: [(L3,2,3),(L3,2,4)], granularity: "medium granularity"}, it indicates that the contradictory occurs between the region coordinates (2,3) and (2,4) at the third semantic level. If the axis of symmetry of the chip resistor is the vertical center line, corresponding to the 2.5th column position in the 4×4 region partition, then when mirroring the topological position (2,3), its horizontal distance relative to the axis of symmetry is calculated as 3-2.5=0.5 region units, and the mirrored position is 2.5-0.5=2, i.e., region coordinates (2,2). Similarly, the mirrored position of position (2,4) is calculated as 2.5-(4-2.5)=1, i.e., region coordinates (2,1). The mirrored contradictory vector maintains the contradictory strength of 0.93, only modifying the topological position to [(L3,2,2),(L3,2,1)], and the granularity level remains "medium granular". The original contradictory vector and the mirrored contradictory vector constitute a positive sample pair, and they should be mapped to similar positions in the embedding space, with an expected Euclidean distance of less than 0.1.

[0050] The negative sample selection strategy extracts inconsistency vectors from candidate defect region images of SMT components from different batches. Different batches refer to image batches collected at different times during the production line, or image sets from different defect types. Specifically, an inconsistency vector library is maintained, containing all inconsistency vectors detected in the past 24 hours, indexed by collection timestamp and defect type label. For the current positive sample pair, inconsistency vectors with an inconsistency intensity difference greater than 0.3 are randomly sampled from the inconsistency vector library as negative samples. For example, if the inconsistency intensity of the current positive sample pair is 0.93, then inconsistency vectors with an inconsistency intensity less than 0.63 or greater than 1.23 are selected as negative samples. If the granularity of the current positive sample pair is "medium-grained," inconsistency vectors with a granularity of "fine-grained" or "coarse-grained" are preferentially selected as negative samples to increase the discriminative power of the sample pairs. Each positive sample pair is paired with 5 negative samples, forming a training combination of 1 positive sample pair and 5 negative samples. The contradiction vector library is updated every 6 hours, and historical contradiction vectors older than 48 hours are deleted, keeping the library size between 10,000 and 50,000 contradiction vectors.

[0051] The contradiction pattern encoder employs a multilayer perceptron structure containing three fully connected layers. The input layer receives the numerical representation of the contradiction vector, encoding the contradiction strength, the two coordinate components of the topological location, the semantic level number, and the granularity category into a fixed-dimensional input vector. Specifically, the contradiction strength is directly used as a scalar feature, with a numerical range of [0,2]. The two coordinate components of the topological location are normalized to the [0,1] interval; for example, the region coordinates (2,3) are normalized to (2 / 4,3 / 4)=(0.5,0.75) under a 4×4 partition. The semantic level number uses one-hot encoding, with six semantic levels corresponding to a six-dimensional vector; the third level is encoded as [0,0,1,0,0,0]. The granularity category uses three-dimensional one-hot encoding: "fine-grained" is encoded as [1,0,0], "medium-grained" as [0,1,0], and "coarse-grained" as [0,0,1]. These features are concatenated to form a 13-dimensional input vector, which is then fed into the first fully connected layer. The first layer contains 64 neurons, using ReLU activation, with a weight matrix of dimension 13×64. The second layer contains 32 neurons, also using ReLU activation, with a weight matrix of dimension 64×32. The third layer is the output layer, containing 16 neurons, with no activation function, directly outputting a 16-dimensional contradictory pattern embedding vector, with a weight matrix of dimension 32×16.

[0052] The contrastive learning loss function is calculated based on cosine similarity scaled to a temperature parameter. For a training batch, containing 32 positive sample pairs and their corresponding 160 negative samples (a total of 192 samples), 192 16-dimensional embedding vectors are generated by the encoder. For the i-th sample and its positive sample pair j, the cosine similarity between their embedding vectors is calculated, i.e., the inner product of the two vectors divided by their respective L2 norm products. This similarity value is scaled by a temperature parameter of 0.07, and then an exponential function is applied to obtain the similarity score for the positive sample pair. For the i-th sample and all its negative samples k, the similarity is calculated and similarly scaled to a temperature parameter and an exponential function is applied to obtain the similarity score for the negative samples. The loss function calculates the positive sample similarity score of the i-th sample divided by the sum of all positive and negative sample similarity scores, and the negative logarithm is taken as the loss value for that sample. The batch loss is obtained by averaging the loss values ​​of all samples within the batch.

[0053] The constraint weight coefficients are dynamically determined based on the inconsistency intensity value. For the inconsistency intensity range [0,2], it is converted into a constraint weight coefficient range [0.5,2.0] through piecewise linear mapping. The specific mapping rule is as follows: when the inconsistency intensity is less than 0.5, the constraint weight coefficient is fixed at 0.5, indicating that the inconsistency contributes little to the model training; when the inconsistency intensity is between 0.5 and 1.5, the constraint weight coefficient increases linearly, calculated as 0.5 + (inconsistency intensity - 0.5) × 1.0. For example, when the inconsistency intensity is 0.93, the constraint weight coefficient is 0.5 + (0.93 - 0.5) × 1.0 = 0.93; when the inconsistency intensity is greater than 1.5, the constraint weight coefficient is fixed at 2.0 to avoid extreme inconsistency samples having an excessive impact on the loss function. The constraint weight coefficient acts as a multiplier of the sample's loss value. In the loss function calculation, the negative logarithmic loss of each sample is multiplied by its corresponding constraint weight coefficient, realizing a monotonically increasing mapping between inconsistency intensity and training importance. The sample pair with a conflict intensity of 0.93 has a weight of 0.93 in the loss calculation, while the sample pair with a conflict intensity of 0.3 has a weight of only 0.5. The former has about 1.86 times the influence on the encoder parameter update.

[0054] The training process uses the Adam optimizer with an initial learning rate of 0.001 and a batch size of 32. Training data is sampled evenly from a contradiction vector library according to defect type to ensure that contradiction vectors for various defects such as cold solder joints, component misalignment, and missing pins have a similar proportion in the training set. Training is iterated 10,000 times, with model performance evaluated on a validation set every 1,000 iterations. The validation set contains 500 manually labeled contradiction vector pairs, where contradiction vectors at symmetrical positions are labeled as equivalent, and contradiction vectors of different defect types are labeled as non-equivalent. The validation metric is the ratio of the average distance between equivalent contradiction vector pairs and the average distance between non-equivalent contradiction vector pairs in the embedding space, with a desired ratio less than 0.2. After training, the encoder weight parameters are saved as a model file, approximately 150KB in size.

[0055] During the inference phase, the contradiction vectors detected in the candidate defect region images are converted into 13-dimensional input vectors using the same numerical conversion process and then input into the trained encoder. Through forward propagation of a three-layer fully connected network, the first layer outputs a 64-dimensional feature vector, the second layer outputs a 32-dimensional feature vector, and the third layer outputs a 16-dimensional contradiction pattern embedding vector. The encoder achieves three encoding functions through hierarchical feature extraction. The contradiction intensity distribution pattern encoding is reflected in the nonlinear transformation of the input contradiction intensity values ​​by the first fully connected layer, mapping the contradiction intensity in the [0,2] interval to a distribution in the 64-dimensional feature space. Contradiction vectors with similar intensities are less than 0.3 apart in this feature space. Topological positional relationship encoding extracts the spatial proximity of position coordinates through the second layer. Contradiction vectors in adjacent regions maintain topological continuity in the 32-dimensional feature space, with the Euclidean distance linearly correlated with the original region distance, and a scaling factor of approximately 0.15. The equivalence encoding of symmetrical positional contradictions is achieved through a contrastive learning loss function constraint. During training, the distance between symmetrical contradiction vectors in the 16-dimensional embedding space is forced to be less than 0.1, enabling the encoder to learn invariant features under positional mirror transformations. The values ​​of each dimension of the embedding vector typically range from [-3, 3], with the specific numerical distribution depending on the inconsistency strength and topological location. For example, an inconsistency vector with an inconsistency strength of 0.93, a location between (2, 3) and (2, 4), a level of L3, and a granularity of "medium granularity" has an embedding vector of [1.2, -0.8, 0.5, 1.7, -1.3, 0.2, 0.9, -0.4, 1.1, 0.6, -1.0, 0.3, 1.5, -0.7, 0.8, 1.0]. Its mirror image inconsistency vector has an embedding vector of [1.1, -0.9, 0.6, 1.6, -1.2, 0.3, 0.8, -0.5, 1.0, 0.7, -0.9, 0.4, 1.4, -0.8, 0.9, 0.9]. The Euclidean distance between the two is 0.08, satisfying the symmetric equivalence requirement. The encoding process transforms the discrete topological location information of the contradiction vectors into coordinates in a continuous embedding space, enabling semantically similar contradiction patterns to cluster in the space and contradiction patterns of different defect types to separate in the space, providing a good feature representation for subsequent clustering and classification.

[0056] The contradictory pattern encoder of this invention fully utilizes the prior knowledge of the symmetry of SMT components through self-supervised contrastive learning. It can learn the essential feature representation of contradictory patterns without a large amount of manual annotation, achieving robustness to positional changes and sensitivity to the intensity of contradictions, and significantly improving the accuracy and generalization ability of defect classification.

[0057] In one optional implementation, a density-adaptive clustering algorithm is used for dynamic clustering in the embedding space formed by the contradictory pattern embedding vectors. The step of classifying defects based on the positional relationship between the contradictory pattern embedding vectors and the cluster centers includes: The local density of each contradictory pattern embedding vector is calculated in the embedding space. The local density is determined by counting the number of other contradictory pattern embedding vectors within the neighborhood radius centered on the target contradictory pattern embedding vector. When contradictory pattern embedding vectors of a new batch of SMT components are added to the embedding space, the neighborhood radius is dynamically adjusted based on the overall distribution density of contradictory pattern embedding vectors in the embedding space. Cluster centers are identified based on the distance between the local density and the contradictory pattern embedding vectors; the density determination criteria for cluster center identification are dynamically adjusted according to the defect occurrence frequency of the SMT component production line, and the density determination criteria are reduced to form new cluster centers when the contradictory pattern embedding vectors of new types of defects continue to increase. Each contradictory pattern embedding vector in the embedding space is assigned to the nearest cluster center to form a dynamic clustering result; the defect is classified based on the dynamic clustering result.

[0058] For example, the density-adaptive clustering algorithm is executed in a 16-dimensional embedding space composed of contradictory pattern embedding vectors, which consists of all contradictory pattern embedding vectors generated by the aforementioned contradictory pattern encoder. The embedding space uses Euclidean distance as a metric; the distance between two 16-dimensional vectors is obtained by taking the square root of the sum of the squares of the differences in their corresponding dimensions. For instance, vector A is [1.2,-0.8,0.5,1.7,-1.3,0.2,0.9,-0.4,1.1,0.6,-1.0,0.3,1.5,-0.7,0.8,1.0], and vector B is [1.1,-0.9,0.6,1.6,-1.2,0.3,0.8,-0.5,1.0,0.7,-0.9]. The distance between the two is calculated as the square root of the sum of the squares of the differences in each dimension, which is approximately 0.01. The embedding vectors of all contradictory patterns in the embedding space are stored in a vector library. The vector library uses a KD-tree index structure to accelerate neighborhood queries and supports range search operations with logarithmic time complexity.

[0059] Local density calculation is based on vector counting within a neighborhood radius. For a target contradictory pattern embedding vector in the embedding space, an initial neighborhood radius of 0.5 is set, and all other contradictory pattern embedding vectors are searched within this radius. A KD-tree index is used to query all vectors whose Euclidean distance from the target vector is less than 0.5, and the number of query results is used as the local density value of the target vector. For example, if a contradictory pattern embedding vector finds 23 other vectors within a radius of 0.5, its local density value is 23. A larger local density value indicates a denser distribution of contradictory pattern embedding vectors in the region where the vector is located, corresponding to common defect types that frequently occur in the production line. The vector library maintains a local density field for each contradictory pattern embedding vector, which is dynamically updated when a new vector is added.

[0060] The dynamic adjustment of the neighborhood radius is based on the overall distribution density of contradictory pattern embedding vectors in the embedding space. The overall distribution density is defined as the total number of contradictory pattern embedding vectors in the vector library divided by the effective volume estimate of the embedding space. The effective volume of the embedding space is obtained by multiplying the ranges of all vectors in each dimension. For example, the minimum value of the first dimension is -2.5, the maximum value is 2.8, and the range is 5.3. Multiplying the range values ​​of all 16 dimensions yields the volume estimate. When contradictory pattern embedding vectors of a new batch of SMT components are added to the vector library, the overall distribution density is recalculated. If the overall distribution density increases by more than 20%, it indicates that the embedding space has become denser, and the neighborhood radius is reduced proportionally to maintain the relative stability of the local density calculation. The specific adjustment strategy is as follows: calculate the ratio of the new overall distribution density to the old density, and divide the neighborhood radius by the cube root of this ratio. For example, if the overall distribution density increases from 800 to 1200, the ratio is 1.5, and the neighborhood radius is adjusted from 0.5 to 0.5 divided by the cube root of 1.5, i.e., 0.5 ÷ 1.145 ≈ 0.437. If the overall distribution density decreases by more than 20%, the neighborhood radius is increased accordingly to ensure that the neighborhood always contains a sufficient number of vectors for density estimation. The neighborhood radius is limited to a range of 0.2 to 1.0 to avoid instability in local density calculations due to a value that is too small or reduced density discrimination due to a value that is too large.

[0061] Cluster center identification combines two metrics: local density and inter-vector distance. Candidate cluster centers must simultaneously satisfy the conditions of high local density and large distances from other high-density vectors. The specific decision-making process is as follows: all contradictory pattern embedding vectors in the vector library are sorted in descending order of their local density values. Vectors with local density values ​​greater than a density threshold are selected as the high-density vector set. The density threshold is initially set to the 75th percentile of the local density values ​​of all vectors in the vector library. For example, if the vector library contains 1000 vectors, and the density value of the 250th vector after sorting by local density is 18, then the density threshold is 18. For each vector in the high-density vector set, the minimum distance to all vectors with higher local density values ​​is calculated, denoted as the separation distance of that vector. If the separation distance of a high-density vector is greater than the separation threshold of 1.5, it is identified as a cluster center. The separation threshold of 1.5 ensures that different cluster centers have significant spatial separation in the embedding space.

[0062] The density determination criteria are dynamically adjusted in response to changes in the frequency of defects on the SMT component production line. The production line maintenance defect frequency statistics module records the number of times each known defect type is detected over the past 24 hours. When the number of detections for a defect type corresponding to a certain cluster increases by more than 30% in the last 6 hours, it indicates an increase in the frequency of that defect type. The density determination criteria for that cluster are then increased accordingly. Specifically, the density threshold is multiplied by 1.2, requiring newly formed cluster centers to have higher local density to be identified, thus avoiding redundant cluster centers near existing clusters. When the embedding vector of contradictory patterns of new defect types continuously increases, a density determination criteria reduction mechanism is triggered. The criterion for continuous increase is: within a continuous 4-hour sliding window, the cumulative number of new vectors more than 2.0 away from all existing cluster centers reaches 15 or more. At this point, the density threshold is reduced to the density value corresponding to the 60th percentile, for example, from 18 to 12, allowing areas with relatively low local density to form new cluster centers and capturing newly emerging defect patterns. After a new cluster center is formed, the system records the creation timestamp and initial vector members of the cluster for subsequent defect type labeling and statistical analysis.

[0063] Clustering assignment for contradictory pattern embedding vectors employs a nearest neighbor strategy. For each contradictory pattern embedding vector in the vector library, its Euclidean distance to all identified cluster centers is calculated, and the cluster center with the smallest distance is selected as the vector's assigned cluster. If the distance of a vector to its nearest cluster center exceeds a distance threshold of 3.0, the vector is marked as an outlier and does not belong to any cluster, awaiting subsequent density determination conditions to form a new cluster or be reassigned. The clustering assignment results are stored as a cluster identifier field in each contradictory pattern embedding vector record in the vector library. The cluster identifier uses an integer number, starting from 1 and incrementing, with outliers having a cluster identifier of 0. After the dynamic clustering results are formed, the number of contradictory pattern embedding vectors contained in each cluster, the coordinates of the cluster center, and the cluster radius are statistically analyzed. The cluster radius is defined as the maximum distance from all vectors within the cluster to the cluster center, reflecting the compactness of the cluster.

[0064] Defect classification is based on the mapping relationship established by clustering results. For known defect types, specific clusters are associated with defect categories through manual annotation or historical data. For example, cluster 1 corresponds to solder joint defects, cluster 2 to component misalignment defects, and cluster 3 to pin missing defects. When a new candidate defect region image undergoes contradiction detection and encoding to generate a contradiction pattern embedding vector, this vector is assigned to a certain cluster, and its defect category is the defect type corresponding to that cluster. If the vector is assigned to an unlabeled cluster, the system marks it as a pending category, prompting manual review to confirm the defect type, and the review result is updated in the cluster annotation mapping table. For outlier vectors, an abnormal defect alarm is triggered, requiring immediate manual intervention for inspection, because outliers correspond to rare or unseen novel defect patterns.

[0065] This invention can dynamically adjust clustering parameters according to the actual defect distribution on the production line, automatically discover new defect patterns and form corresponding clusters, without the need to preset the number of defect categories, adapt to the diversity and time-varying nature of defect types in industrial sites, and significantly improve the robustness and real-time response capability of defect classification.

[0066] In an optional implementation, when the contradiction intensity of semantic features related to boundary clarity in the contradiction vector exceeds a preset intensity threshold, the step of reconstructing the visual semantic map based on boundary positioning deviation information includes: Filter the contradictory terms in the contradictory vector that involve semantic features related to boundary clarity, and extract the contradiction intensity value corresponding to the contradictory term; when the contradiction intensity value exceeds a preset intensity threshold, backtrack to the boundary region in the candidate defect region image where the contradiction occurred; Subpixel-level boundary localization is performed in the boundary region. The detected actual boundary position is spatially registered with the expected position of the corresponding boundary in a preset normal sample set. The normal offset distance and tangential offset distance of the actual boundary position relative to the expected position are calculated to form the boundary localization deviation information. The set of semantic units that need to be corrected in the visual semantic graph is identified based on the boundary positioning deviation information. For each semantic unit in the set of semantic units, the boundary positioning deviation information is converted into a compensation vector of node attributes, and the original node attributes are compensated and corrected. Based on the corrected node attributes, the semantic correlation between semantic units is recalculated and the weight of the connecting edges is updated to complete the adaptive reconstruction of the visual semantic graph.

[0067] For example, the screening of contradictory terms related to boundary clarity semantic features is based on the parsing of the semantic feature identifier field in the contradictory vector. The contradictory vector data structure contains a semantic feature identifier field, which is encoded in 8 bits. The 3rd to 5th bits are used to identify the semantic feature type: coded value 001 indicates a semantic feature related to boundary clarity, coded value 010 indicates a semantic feature related to texture consistency, and coded value 011 indicates a semantic feature related to color distribution. The screening module iterates through all contradictory vectors, extracts the 3rd to 5th bits of the semantic feature identifier field through bitwise operations, and performs a bitwise AND operation with the target coded value 001. If the result is non-zero, the contradictory term is determined to involve semantic features related to boundary clarity. The screened contradictory terms constitute a subset of boundary contradictions. Each contradictory term in the subset retains a complete contradictory vector structure, including the contradictory strength value, topological location coordinates, hierarchical identifier, and granularity identifier.

[0068] The extraction of the conflict intensity value directly reads the conflict intensity field from the conflict vector. This field is a floating-point number with a value ranging from 0 to 2. The preset intensity threshold is set to 1.2, indicating that the semantic conflict caused by the boundary positioning deviation has reached the level requiring intervention. The filtering module performs a threshold comparison on the conflict intensity value of each conflict item in the boundary conflict subset. When the conflict intensity value is greater than 1.2, the boundary positioning deviation analysis process is triggered. The determination result of the trigger condition is stored in the trigger flag field of the conflict item, with a flag value of 1 indicating triggering and a flag value of 0 indicating no triggering. The preset intensity threshold can be adjusted according to the defect detection accuracy requirements of the actual production line, with a value ranging from 0.8 to 1.5. The lower the threshold, the higher the sensitivity to boundary deviation, increasing the reconstruction frequency and computational overhead.

[0069] The backtracking of the boundary region relies on the topological position coordinate field in the contradiction vector. This field records the start and end positions of the contradiction term in the candidate defect region image, represented in pixel coordinates. For example, the start position is x-coordinate 120, y-coordinate 85, and the end position is x-coordinate 145, y-coordinate 110. The boundary region localization module calculates a rectangular bounding box based on the start and end positions. The top-left corner of the bounding box is the start position, and the bottom-right corner is the end position. The image area within the bounding box is the boundary region where the contradiction occurred. The candidate defect region image is stored in grayscale format with a resolution of 512 pixels by 512 pixels, and pixel values ​​ranging from 0 to 255. The boundary region is extracted from the complete image by cropping, retaining all pixel data within the bounding box to form a boundary region sub-image. The size of the boundary region sub-image is equal to the width and height of the bounding box. For example, if the x-coordinate span is 25 pixels and the y-coordinate span is 25 pixels, then the size of the boundary region sub-image is 25 pixels by 25 pixels.

[0070] Subpixel-level boundary localization performs gradient analysis on the boundary region sub-image. The Sobel operator is used to calculate the lateral and vertical gradients for each pixel position in the boundary region sub-image. The lateral gradient is obtained by multiplying the pixel value to the left by -1, the pixel value to the right by +1, and then summing the results with weighted combinations of the pixels above and below it. The vertical gradient is obtained by multiplying the pixel value above it by -1, the pixel value below it by +1, and then summing the results with weighted combinations of the pixels to the left and right it. The gradient magnitude is defined as the square root of the sum of the squares of the lateral and vertical gradients, and the gradient direction is defined as the vertical gradient divided by the arctangent of the lateral gradient. Pixels with gradient magnitudes greater than a gradient threshold of 30 are marked as boundary candidate points. The gradient threshold of 30 is dynamically adjusted based on the contrast of the boundary region sub-image; it decreases to 20 when the contrast is below 50 and increases to 40 when the contrast is above 150. Boundary candidate points are sorted in descending order of gradient magnitude, and the top 10 candidate points with the largest gradient magnitudes are selected for subpixel localization optimization. Subpixel localization employs a parabolic interpolation method, fitting a two-dimensional parabola to the gradient magnitude of candidate points and their eight neighboring pixels. The x and y coordinates of the peak position of the parabola represent the subpixel-level boundary position, achieving a positional accuracy of 0.1 pixels. The subpixel-level boundary position is represented by floating-point coordinates, for example, x-coordinate 132.7 and y-coordinate 97.3.

[0071] A pre-defined set of normal samples is stored in a sample database, containing 500 SMT component images that have been manually reviewed and confirmed as qualified products. Each normal sample image is pre-annotated with the expected positions of key boundaries. These boundary positions are stored as a sequence of control points, each containing an x-coordinate, y-coordinate, and boundary direction angle. Spatial registration employs a feature point matching strategy, extracting corner features from the boundary region sub-image and the normal sample image. Corner detection uses the Harris corner detection algorithm; pixels with a corner response value greater than a response threshold of 0.01 are identified as corners. An affine transformation matrix is ​​calculated between the boundary region sub-image and the normal sample image based on the corner correspondence. This affine transformation matrix includes rotation, translation, and scaling parameters. The affine transformation matrix is ​​applied to the control point sequence of the expected boundary positions in the normal sample image, transforming the expected positions to the coordinate system of the boundary region sub-image, thus completing the spatial registration. The difference between the registered expected positions and the sub-pixel level boundary positions constitutes the boundary positioning deviation.

[0072] Boundary positioning deviation information is decomposed into two components: normal offset distance and tangential offset distance. The normal offset distance is defined as the perpendicular distance from the actual boundary position to the desired position. It is calculated by projecting the vector connecting the actual and desired boundary positions onto the boundary normal direction vector; the projection length is the normal offset distance. The boundary normal direction vector is determined by the boundary gradient direction. Adding 90 degrees to the gradient direction angle yields the normal direction angle. The x-coordinate of the normal direction vector is the cosine of the normal direction angle, and the y-coordinate is the sine of the normal direction angle. The tangential offset distance is defined as the parallel distance from the actual boundary position to the desired position. It is calculated by projecting the vector connecting the actual and desired boundary positions onto the boundary tangential direction vector; the projection length is the tangential offset distance. The boundary tangential direction vector is orthogonal to the normal direction vector. The x-coordinate of the tangential direction vector is the negative sine of the normal direction angle, and the y-coordinate is the cosine of the normal direction angle. The normal and tangential offset distances are measured in pixels, ranging from -10 to +10 pixels. Positive values ​​indicate that the actual boundary position is offset outward relative to the desired position, while negative values ​​indicate an inward offset. The boundary positioning deviation information is stored in a structured data format, including a normal offset distance field and a tangential offset distance field, with data precision retained to 0.01 pixels.

[0073] The identification of the semantic unit set requiring correction is based on topological proximity analysis of the visual semantic graph. Each semantic unit in the visual semantic graph corresponds to an image patch in the candidate defect region image, with a patch size of 32 pixels × 32 pixels. Semantic units exist in the visual semantic graph as nodes, and node attributes contain a 64-dimensional feature vector. The boundary region sub-image containing the boundary localization deviation information is located to the node set in the visual semantic graph through coordinate mapping. The mapping rule is that the row and column index of the starting node is obtained by dividing the coordinate of the top left corner of the boundary region sub-image by 32, and the row and column index of the ending node is obtained by dividing the coordinate of the bottom right corner by 32, with the index value rounded down. Nodes within the range of all row and column indices between the starting and ending nodes constitute the initial correction candidate set. For each node in the initial correction candidate set, the overlap area between its image patch and the boundary region sub-image is calculated. Nodes with an overlap area greater than 50% of the image patch area are included in the semantic unit set requiring correction. The number of nodes in the semantic unit set is usually between 4 and 16, depending on the spatial range of the boundary region.

[0074] The construction of the node attribute compensation vector maps boundary positioning deviation information to a 64-dimensional feature space. The compensation vector generator receives normal and tangential offset distances as input and generates a 64-dimensional compensation vector through a linear transformation. The linear transformation matrix is ​​a 2-row, 64-column weight matrix, with the first row corresponding to the weight coefficients of the normal offset distance and the second row corresponding to the weight coefficients of the tangential offset distance. The weight matrix is ​​obtained through offline training. The training dataset contains 1000 pairs of boundary positioning deviations and their corresponding node attribute corrections. The training objective is to minimize the mean squared error between the compensated node attributes and the correctly labeled node attributes. The compensation vector is calculated by multiplying the normal offset distance by the first row of the weight matrix and then adding the tangential offset distance by the second row of the weight matrix. The result is a 64-dimensional vector, with each dimension ranging from -0.5 to +0.5. The original node attributes and the compensation vector are added element-wise to obtain the corrected node attributes. The corrected node attributes replace the original attribute values ​​of the corresponding nodes in the visual semantic map. The node attribute update operation uses an atomic write mechanism to ensure data consistency during concurrent access.

[0075] For any two nodes in the semantic unit set, extract their corrected 64-dimensional node attribute vectors, calculate the dot product of the two vectors, divide by the product of their magnitudes, and obtain the cosine similarity value, which ranges from -1 to +1. A cosine similarity value greater than 0.6 indicates that the two nodes are highly semantically related, a similarity value between 0.3 and 0.6 indicates moderately related, and a similarity value less than 0.3 indicates weakly related or unrelated. The rule for updating the edge weights is as follows: if the cosine similarity is greater than 0.6 and the original edge does not exist, create a new edge and set its weight to the cosine similarity value; if the cosine similarity is greater than 0.6 and the original edge already exists, update the edge weight to the cosine similarity value; if the cosine similarity is less than 0.3 and the original edge exists, delete the edge. The edge weight ranges from 0.3 to 1.0, with higher weights indicating stronger semantic connections between nodes. The edge set of the visual semantic graph is reindexed after the weight is updated, and the adjacency list data structure is used to store the outgoing edge list of each node, supporting efficient graph traversal and query operations.

[0076] The boundary positioning deviation compensation mechanism of this invention dynamically optimizes the topological structure and semantic association of the visual semantic graph through sub-pixel level boundary analysis and node attribute correction, effectively eliminating contradictions and conflicts caused by boundary ambiguity.

[0077] A second aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0078] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0079] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine vision-based method for real-time detection and classification of defects in SMT components, characterized in that, include: Obtain candidate defect region images of SMT components, extract multi-granularity semantic features from the candidate defect region images, and construct a visual semantic map from the multi-granularity semantic features. The semantic unit pairs in the visual semantic graph are traversed, and the co-correlation of each semantic unit pair in the preset normal sample set and the current correlation in the candidate defect region image are calculated. When the co-correlation and the current correlation have opposite signs, it is determined that there is a visual contradiction in the semantic unit pair, and the detected visual contradiction is encoded as a contradiction vector. The contradiction vector is input into the contradiction pattern encoder to obtain the contradiction pattern embedding vector, and the contradiction pattern encoder is trained through a self-supervised contrastive learning framework. In the embedding space formed by the contradictory pattern embedding vectors, a density-adaptive clustering algorithm is used for dynamic clustering, and defect classification is performed based on the positional relationship between the contradictory pattern embedding vectors and the cluster centers. When the contradiction intensity of semantic features related to boundary clarity in the contradiction vector exceeds a preset intensity threshold, the visual semantic map is reconstructed based on the boundary positioning deviation information. The steps of obtaining candidate defect region images of SMT components, extracting multi-granularity semantic features from the candidate defect region images, and constructing a visual semantic map from the multi-granularity semantic features include: The candidate defect region image is decomposed into multiple scales, and semantic features are extracted at different scales to obtain multi-granular semantic features; Each semantic feature in the multi-granularity semantic features is mapped to a semantic unit of the visual semantic graph, and node attributes are assigned to each semantic unit based on the distribution characteristics of the semantic features in the feature space; Calculate the semantic correlation between semantic units. When the semantic correlation between two semantic units meets the association threshold, establish a connection edge in the visual semantic graph. The weight of the edge is determined according to the strength of the semantic correlation. The semantic relevance is obtained by measuring the feature similarity between semantic features corresponding to semantic units and the spatial position relationship of semantic units in the candidate defect region image.

2. The method according to claim 1, characterized in that, The steps of traversing the semantic unit pairs in the visual semantic graph, calculating the co-correlation of each semantic unit pair in a preset normal sample set and the current correlation in the candidate defect region image, determining that there is a visual contradiction in the semantic unit pair when the co-correlation and the current correlation have opposite signs, and encoding the detected visual contradiction into a contradiction vector include: Extract semantic unit pairs from the visual semantic graph; The joint distribution of the feature response values ​​of the semantic units in a preset normal sample set is statistically analyzed. The dominant trend of the joint distribution is fitted to obtain the cooperative direction vector. The sign and value of the cooperative correlation are determined based on the quadrant position of the cooperative direction vector. The semantic unit is extracted to form the feature response value in the candidate defect region image to construct the current response vector, and the sign and value of the current correlation are determined according to the quadrant position of the current response vector; When the sign of the co-correlation is opposite to the sign of the current correlation, it is determined that there is a visual contradiction in the semantic unit pair, and the absolute value of the difference between the co-correlation value and the current correlation value is calculated as the contradiction intensity. Semantic unit pairs with visual contradictions are encoded as contradiction vectors, where the dimension value of the contradiction vector corresponds to the contradiction intensity. At the same time, the topological position of the semantic unit pair in the visual semantic graph and the semantic granularity level to which it belongs are recorded as the structural information of the contradiction vector.

3. The method according to claim 2, characterized in that, The steps of statistically analyzing the joint distribution of the feature response values ​​of the semantic units in a preset set of normal samples, fitting the dominant trend of the joint distribution to obtain a cooperative direction vector, and determining the sign and value of the cooperative correlation based on the quadrant position of the cooperative direction vector include: Extract the feature response values ​​of the semantic unit pairs from a preset set of normal samples, and construct a two-dimensional response value pair composed of the feature response values ​​of the semantic unit pairs; The two-dimensional response value pairs are represented as data points in a two-dimensional coordinate space, and the joint probability distribution of all data points is statistically analyzed to form the joint distribution of semantic unit pairs; Principal component analysis is performed on the joint distribution to fit the dominant trend of change. The direction of the first principal component of the data point distribution is extracted as the co-direction vector. The direction of the first principal component corresponds to the projection direction with the largest variance in the joint distribution. The quadrant in which the cooperative direction vector is located is determined by the direction angle of the cooperative direction vector in the two-dimensional coordinate space. The first and third quadrants indicate that the cooperative correlation is positive, while the second and fourth quadrants indicate that the cooperative correlation is negative. The degree of dispersion of the data points projected along the cooperative direction vector is used as the cooperative strength. The cooperative strength is then normalized to a preset numerical range and used as the value of the cooperative correlation.

4. The method according to claim 2, characterized in that, The steps of inputting the contradiction vector into the contradiction pattern encoder to obtain the contradiction pattern embedding vector, and training the contradiction pattern encoder through a self-supervised contrastive learning framework, include: Based on the structural symmetry of SMT components, the topological positions in the contradiction vector are mirrored along the symmetry axis of the SMT component to obtain a mirror contradiction vector, while keeping the contradiction strength unchanged; The original contradiction vector and the mirror contradiction vector are paired as positive sample pairs, and negative samples are selected from the contradiction vectors extracted from the candidate defect area images of SMT components from different batches to form the training sample pairs of the self-supervised contrastive learning framework. The contradictory pattern encoder is trained using the training samples. The constraint weight coefficients of the training sample pairs in the contrastive learning loss function are determined based on the contradiction intensity value. In the loss function optimization, the constraint weight coefficients serve as scaling factors for the distance adjustment magnitude of sample pairs in the embedding space, thereby realizing a monotonically increasing mapping relationship between the contradiction intensity value and the distance adjustment magnitude. The contradiction vector is input into the trained contradiction pattern encoder and mapped into a contradiction pattern embedding vector through multi-layer nonlinear transformation. The mapping process encodes the distribution pattern of the contradiction intensity, the topological positional relationship, and the equivalence of contradictions in symmetrical positions.

5. The method according to claim 1, characterized in that, The steps of dynamically clustering in the embedding space formed by the contradictory pattern embedding vectors using a density-adaptive clustering algorithm, and classifying defects based on the positional relationship between the contradictory pattern embedding vectors and the cluster centers, include: The local density of each contradictory pattern embedding vector is calculated in the embedding space. The local density is determined by counting the number of other contradictory pattern embedding vectors within the neighborhood radius centered on the target contradictory pattern embedding vector. When contradictory mode embedding vectors of a new batch of SMT components are added to the embedding space, the neighborhood radius is dynamically adjusted based on the overall distribution density of contradictory mode embedding vectors in the embedding space. Cluster centers are identified based on the distance between the local density and the contradictory pattern embedding vectors. The density determination criteria for cluster center identification are dynamically adjusted based on the frequency of defects occurring in the SMT component production line. When the embedding vector of contradictory patterns of new types of defects continues to increase, the density determination criteria are reduced to form new cluster centers. Each contradictory pattern embedding vector in the embedding space is assigned to the nearest cluster center to form a dynamic clustering result; Defects are classified based on the dynamic clustering results.

6. The method according to claim 1, characterized in that, When the contradiction intensity of semantic features related to boundary clarity in the contradiction vector exceeds a preset intensity threshold, the step of reconstructing the visual semantic map based on boundary positioning deviation information includes: Filter out contradictory terms in the contradictory vector that involve semantic features related to boundary clarity, and extract the contradiction intensity value corresponding to the contradictory terms; When the inconsistency intensity value exceeds a preset intensity threshold, the process is traced back to the boundary region in the candidate defect region image where the inconsistency occurred; Subpixel-level boundary localization is performed in the boundary region. The detected actual boundary position is spatially registered with the expected position of the corresponding boundary in a preset normal sample set. The normal offset distance and tangential offset distance of the actual boundary position relative to the expected position are calculated to form the boundary localization deviation information. The set of semantic units that need to be corrected in the visual semantic map is identified based on the boundary positioning deviation information. For each semantic unit in the set of semantic units, the boundary positioning deviation information is converted into a compensation vector of node attributes, and the original node attributes are compensated and corrected. Based on the corrected node attributes, the semantic correlation between semantic units is recalculated and the weights of the connecting edges are updated to complete the adaptive reconstruction of the visual semantic graph.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images