Defect detection method and device based on multi-source data

Abstract

The invention discloses a defect detection method and device based on multi-source data. The method comprises the following steps: acquiring data to be subjected to defect detection and source field sample data with defect labels; based on a pre-constructed mapping model, performing alignment mapping on the to-be-defect detection data and the source field sample data to obtain defect detection features and source field sample features; performing distance analysis on each defect detection feature and all source field sample features to determine a first pseudo tag; based on a pre-constructed label classification model, performing label updating on the first pseudo label and the source field sample data with the defect label to obtain a second pseudo label; and updating the mapping model and the label classification model based on the first pseudo-label and the second pseudo-label, repeating alignment mapping and label updating until a predetermined condition is met, and giving a defect label corresponding to the data to be subjected to defect detection. Through high-robustness defect detection models of two mechanisms, the precision of generating defect labels is improved.

Description

Technical Field

[0001] This invention belongs to the field of defect detection technology, specifically relating to a defect detection method and apparatus based on multi-source data. Background Technology

[0002] In fields such as industrial defect detection using models, transfer learning methods are widely used to address the scarcity of labeled samples in target application scenarios. This involves generating pseudo-labels for unlabeled samples in the target application scenario and using these pseudo-labels for iterative model training. However, the accuracy of the pseudo-labels directly determines the model's final detection performance; therefore, improving the quality of pseudo-labels and utilizing them effectively is a key challenge in this field.

[0003] To address the aforementioned issues, existing technologies have proposed several transfer learning methods based on selective pseudo-label learning (TLPLS), with representative methods including selective pseudo-label learning based on structured prediction (SPL). These methods typically rely on a single information source for pseudo-label learning. For example, they determine pseudo-labels and confidence levels by calculating the distance between target domain samples and the class centers of the source or target domains. Subsequently, according to a preset ratio, samples with higher confidence levels and their pseudo-labels from each class are selected for iterative training of the model. The selection ratio is gradually increased with the number of iterations until all target samples are included.

[0004] However, the aforementioned existing technologies have significant drawbacks. First, their pseudo-label learning process relies solely on single-dimensional information such as class centers or source domain samples, neglecting the combined effect of multi-source information. When sample features are mixed across categories, relying solely on distance from class centers can easily lead to erroneous pseudo-labels, resulting in overly general information representation and the loss of crucial category information. Second, existing technologies employ a strategy of selecting high-confidence samples at a fixed ratio. This approach has inherent drawbacks, as it cannot prevent the introduction of some pseudo-labels with high confidence but actual errors into the training process, while simultaneously discarding some pseudo-labels with low confidence but actual correctness. The introduction of these errors or the loss of information interfere with the iterative optimization of the model, leading to poor performance or even training instability. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, this invention provides a defect detection method and apparatus based on multi-source data, which can generate high-precision defect labels.

[0006] In a first aspect, the present invention provides a defect detection method based on multi-source data, comprising: Acquire the defect detection data and the source domain sample data with defect labels; Based on a pre-built mapping model, the defect detection data and the source domain sample data are aligned and mapped to obtain the defect detection features and the source domain sample features, respectively. Each defect detection feature is compared with the features of all source domain samples to determine the target defect detection features, target defect detection data and corresponding first pseudo-label. Based on the pre-built label classification model, the target defect detection data and its corresponding first pseudo-label, as well as the source domain sample data with defect labels, are updated to obtain the second pseudo-label corresponding to the target defect detection data. Based on the first pseudo-label and the second pseudo-label, the mapping model and the label classification model are updated, and the alignment mapping and label update are repeated until the predetermined conditions are met, and the defect label corresponding to the defect detection data is given.

[0007] Furthermore, based on a pre-built mapping model, the defect detection data and source domain sample data are aligned and mapped to obtain defect detection features and source domain sample features, respectively, including: Align and map the defect detection data and the source domain sample data to the same data space; An optimization objective function is constructed with the goal of minimizing intra-class distribution differences and maximizing intra-class compactness and inter-class separation within the data space. The Lagrange algorithm is used to solve the objective function and determine the shared feature matrix of the data space. By aligning and mapping the defect detection data and source domain sample data using a shared feature matrix, the defect detection features and source domain sample features are given.

[0008] Furthermore, the objective function is optimized, specifically expressed as: Where min is the minimization function. To optimize the constraints of the objective, W represents the squared Euclidean norm of the shared characteristic matrix W. T The shared feature matrix W is the transpose, where α1 and α2 represent the weights of intra-class compactness and inter-class separability, respectively, and λ represents the regularization parameter. s Let H be the cluster center matrix of all classes in the source domain sample data, and let I be a diagonal matrix. k Let represent the k-dimensional identity matrix, MCCD represent the maximum cluster center dissimilarity, DSC represent the intra-cluster compactness, DCMC represent the compactness of multiple clusters within the same class, DMCM represent the inter-cluster separation, DCNC represent the separation of neighboring clusters, (DSC+DCMC) represents the intra-cluster compactness, and (DMCM+DCNC) represents the inter-cluster separation.

[0009] Furthermore, the objective function is solved using the Lagrange algorithm to determine the shared feature matrix of the data space, including: After matrixing the objective function, construct the Lagrange function; Set the partial derivatives to 0, and solve for all eigenvalues ​​and eigenvectors that satisfy the requirements; The obtained eigenvalues ​​are sorted, and the k smallest eigenvalues ​​are selected to form Lagrange multipliers. The k corresponding eigenvectors form a shared feature matrix.

[0010] Furthermore, distance analysis is performed on each defect detection feature and all source domain sample features to determine the target defect detection features, target defect detection data, and corresponding first pseudo-labels for each defect, including: Calculate the Euclidean distance between each defect detection feature and all source neighborhood sample features; Arrange the Euclidean distances between each defect detection feature and all source domain sample features in ascending order, and select a predetermined number of source domain sample features that are closest to the defect detection feature. Based on the defect labels of a predetermined number of source domain sample features, determine the initial pseudo-label matrix of defect detection features; A consistency judgment is performed on the initial pseudo-label matrix to determine the target defect detection features, the target defect detection data, and the corresponding first pseudo-label.

[0011] Furthermore, based on the pre-built label classification model, the target defect detection data and its corresponding first pseudo-label, as well as the source domain sample data with defect labels, are labeled to obtain the second pseudo-label corresponding to the target defect detection data, specifically including: The source domain sample data, source domain sample features and defect labels, target defect detection features and the first pseudo label are input into the label classification model; In the label classification model, the optimization objective of the classifier is constructed with the goal of minimizing the difference in classification results between samples of the same class and maximizing the difference in classification results between samples of different classes, and the explicit solution of the optimization objective of the classifier is calculated. Based on the explicit solution, the second pseudo-label corresponding to the target defect detection data is obtained.

[0012] Furthermore, the optimization objectives for constructing the classifier include: Based on the differences in classification results between the cluster centers of the source domain and the defect detection domain, the differences in classification results between samples of the same cluster and their cluster centers, the differences in classification results between cluster centers of the same cluster, and the differences in classification results between neighboring main cluster centers, the optimization objective of the classifier is constructed.

[0013] Furthermore, based on the explicit solution, the second pseudo-label corresponding to the target defect detection data is obtained, including: Calculate the product of the explicit solution and the kernel function matrix of the target defect detection features; Based on the calculation results, a second pseudo-label is obtained, where the label value of the second pseudo-label is the row where the maximum value of each column vector is located.

[0014] Furthermore, based on the first and second pseudo-labels, the mapping model and label classification model are updated, and the alignment mapping and label update are repeated until predetermined conditions are met, thus providing the defect labels corresponding to the defect detection data, including: For each target defect detection data, compare its first pseudo-label and second pseudo-label. If they match, determine the target defect detection data and its pseudo-label, and use them as the updated defect detection domain sample data. The updated defect detection domain sample data is input into the mapping model and the label classification model to update the model parameters. The updated model parameters include updating the shared feature matrix of the mapping model and updating the classifier coefficient matrix of the label classification model. Repeatedly perform alignment mapping and label updates, and iterate and optimize until the predetermined number of iterations is reached. Once the iteration ends, output the defect label corresponding to the defect detection data.

[0015] Secondly, the present invention also provides a defect detection device based on multi-source data, employing the above-mentioned defect detection method based on multi-source data, the device comprising: The data acquisition module is used to acquire the defect detection data and the source domain sample data with defect labels; The feature mapping module is used to align and map the defect detection data and source domain sample data based on a pre-built mapping model, so as to obtain defect detection features and source domain sample features respectively. The initial label determination module is used to perform distance analysis between each defect detection feature and all source domain sample features to determine each target defect detection feature, target defect detection data and corresponding first pseudo-label. The label update module is used to update the labels of the target defect detection data and the corresponding first pseudo-label, as well as the source domain sample data with defect labels, based on a pre-built label classification model, so as to obtain the second pseudo-label corresponding to the target defect detection data. The final label determination module is used to update the mapping model and label classification model based on the first pseudo-label and the second pseudo-label, and repeatedly align the mapping and label update until the predetermined conditions are met, and give the defect label corresponding to the defect detection data.

[0016] The present invention provides a defect detection method and apparatus based on multi-source data, which has at least the following beneficial effects: (1) Two different mechanisms are used to generate defect pseudo-labels for the defect detection data. The consistency of the two different sources of defect pseudo-labels is judged. The high reliability samples that pass the consistency judgment can participate in the next round of model training, which effectively improves the accuracy of defect pseudo-labels. This avoids the problem of low pseudo-label accuracy caused by relying on a single information source. It also avoids the problem of introducing erroneous pseudo-labels by selecting high confidence samples in a fixed proportion to interfere with the model iteration process. This ensures the stability and convergence of the iteration process, thereby obtaining a more robust defect detection model and ensuring the quality of defect pseudo-labels.

[0017] (2) Based on the source domain sample data, generate pseudo labels from the first source for the target domain samples; by checking the consistency of the labels of the nearest neighbor samples, effectively filter out low-quality pseudo labels located at the category boundary or in the feature ambiguity region, and initially improve the purity of the label data used for subsequent steps.

[0018] (3) Based on the cluster center, pseudo-labels from a second source are obtained. By cross-validating the defect pseudo-labels from two independent sources, the reliability of the pseudo-labels is "double-confirmed," which greatly reduces the risk of erroneous pseudo-labels being introduced into the iteration process and ensures that only the most reliable samples are used to optimize the model. Through iterative loops, the model continuously optimizes itself using higher-quality pseudo-label data, gradually improving the performance of feature representation and classifier, and finally obtaining a high-precision and robust defect detection model. Attached Figure Description

[0019] Figure 1 A flowchart of a defect detection method based on multi-source data provided by the present invention; Figure 2 A flowchart illustrating the process of obtaining defect detection features and source domain sample features according to a certain embodiment of the present invention; Figure 3 A flowchart for determining the first pseudo-tag according to one embodiment of the present invention; Figure 4 A flowchart for determining the second pseudo-tag according to one embodiment of the present invention; Figure 5 A flowchart illustrating the defect labels corresponding to the defect detection data provided in one embodiment of the present invention. Figure 6 The present invention provides a structural block diagram of a defect detection device based on multi-source data. Detailed Implementation

[0020] To better understand the above technical solutions, a detailed description of the solutions will be provided below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0022] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0023] like Figure 1 As shown, the present invention provides a defect detection method based on multi-source data, comprising: Acquire the defect detection data and the source domain sample data with defect labels; Based on a pre-built mapping model, the defect detection data and the source domain sample data are aligned and mapped to obtain the defect detection features and the source domain sample features, respectively. Each defect detection feature is compared with the features of all source domain samples to determine the target defect detection features, target defect detection data and corresponding first pseudo-label. Based on the pre-built label classification model, the target defect detection features and their corresponding first pseudo-labels, as well as the source domain sample data with defect labels, are updated to obtain the second pseudo-labels corresponding to the target defect detection data. Based on the first pseudo-label and the second pseudo-label, the mapping model and the label classification model are updated, and the alignment mapping and label update are repeated until the predetermined conditions are met, and the defect label corresponding to the defect detection data is given.

[0024] This invention generates defect pseudo-labels for defect detection data through two different mechanisms, and determines the consistency of these two different sources of defect pseudo-labels. High-reliability samples that pass the consistency determination can participate in the next round of model training, effectively improving the accuracy of defect pseudo-labels. This avoids the problem of low pseudo-label accuracy caused by relying on a single information source, and also avoids the problem of introducing erroneous pseudo-labels by selecting high-confidence samples in a fixed proportion, which would interfere with the model iteration process. This ensures the stability and convergence of the iteration process, thereby obtaining a more robust defect detection model and guaranteeing the quality of defect pseudo-labels.

[0025] like Figure 2 As shown, based on a pre-built mapping model, the defect detection data and source domain sample data are aligned and mapped to obtain defect detection sample features and source domain sample features, respectively, including: Align and map the defect detection data and the source domain sample data to the same data space; An optimization objective function is constructed with the goal of minimizing intra-class distribution differences and maximizing intra-class compactness and inter-class separation within the data space. The Lagrange algorithm is used to solve the objective function and determine the shared feature matrix of the data space. By aligning and mapping the defect detection data and source domain sample data using a shared feature matrix, the features of the defect detection samples and the source domain samples are given.

[0026] In this embodiment, by mapping the data space, the data distribution of the source domain and the target domain is aligned, and subsequent pseudo-label learning is performed in a unified feature dimension, thereby achieving effective knowledge transfer.

[0027] The mapping model can be a Discriminative Feature Transfer Learning (DFTCC) model based on cluster centers. To efficiently measure the compactness among samples within the same cluster, it uses the cluster center as a marker, measuring intra-cluster compactness based on the distance of each sample to its cluster center. Furthermore, to measure the compactness among multiple clusters within the same class, the distance between the cluster center and the main cluster center (the cluster with the largest sample size) is added to the intra-cluster compactness measurement. Additionally, to efficiently measure the separability between samples from different classes, the distance between the main cluster center and the mean of all samples is used.

[0028] Specifically, in one embodiment, the shared feature matrix It can map the m-dimensional source domain data features and the defect detection data features to a k-dimensional data space, so that the intra-class distribution difference in the data space is as small as possible, and the intra-class compactness and inter-class separation are as large as possible. Thus, the transfer learning optimization objective (i.e. optimization objective function) of DFTCC is constructed.

[0029] The objective function is optimized, specifically expressed as: Where min is the minimization function. To optimize the constraints of the objective, structural information of the source domain cluster center features after stabilization mapping is used. W represents the squared Euclidean norm of the shared characteristic matrix W. T For the transpose of the shared characteristic matrix W, U s H represents the cluster center matrix of all classes in the source domain sample data, where H is a diagonal matrix and I is a cluster center matrix of all classes. k Let represent the k-dimensional identity matrix, MCCD represent the maximum cluster centroid dissimilarity, DSC represent intra-cluster compactness, DCMC represent intra-cluster compactness, DMCM represent inter-cluster separation, DCNC represent neighboring cluster separation, (DSC+DCMC) represent intra-cluster compactness, and (DMCM+DCNC) represent inter-cluster separation. α1 and α2 represent the weights of intra-cluster compactness and inter-cluster separation, respectively, and λ represents the regularization parameter used to control... To prevent W from becoming too complex, the weights α1, α2, and λ all need to be set in advance.

[0030] Understandably, MCCD is a measure of the distributional difference of cluster centers. By minimizing MCCD, the cluster centers of the source neighborhood and the defect detection domain can be aligned. For the domain centers, since they reflect global information, the sample mean can be used for calculation. Specifically, first, the squared Euclidean distance between the mean of all samples in the source neighborhood and the defect detection domain is calculated as the global distributional difference; then, for each class, the sum of the squared Euclidean distances of all its cluster centers in the source neighborhood and the defect detection domain is calculated, and the average is taken as the local cluster difference; the global distributional difference and the local cluster difference are weighted and combined (e.g., the weights are 1 and 1 / 2 respectively) to obtain the MCCD.

[0031] Distance Scaling (DSC) measures the distance between samples within the same cluster and their cluster centers, reflecting the compactness of samples within a cluster. For a single sample in both the source neighborhood and the defect detection domain, its cluster is recorded during clustering, from which the DSC can be calculated. Specifically, for the source neighborhood cluster, the sum of the squared Euclidean distances between all samples in that cluster and their cluster centers is calculated, and the average is taken as the intra-cluster distance within the source neighborhood cluster. For the defect detection domain cluster, the sum of the squared Euclidean distances between all samples in that cluster and their cluster centers is calculated, and the average is taken as the intra-cluster distance within the defect detection domain cluster. All clusters are merged, and the intra-cluster distances of the source neighborhood and the defect detection domain are added together and divided by the number of clusters to obtain the DSC. Minimizing the DSC enhances the compactness of intra-cluster clusters and avoids the dispersion of samples within the same cluster.

[0032] DCMC (Distributed Distributed Catch-up) is based on the center of the class's principal cluster (the cluster with the largest sample size). It calculates the distance between the center of a class cluster and the center of its principal cluster, reflecting the compactness of multiple clusters within the same class. Specifically, for the source domain class, the sum of the squared Euclidean distances between all cluster centers and the principal cluster center of that class is calculated, and the average is taken as the distance between multiple clusters within the same class in the source domain. For the defect detection domain class, the sum of the squared Euclidean distances between all cluster centers and the principal cluster center of that class is calculated, and the average is taken as the distance between multiple clusters within the same class in the defect detection domain. All classes are merged, and the distances between multiple clusters within the same class in the source domain and the defect detection domain are added together and divided by the number of classes to obtain the DCMC. Minimizing the DCMC can enhance the compactness of multiple clusters within the same class and avoid the dispersion of clusters within the same class.

[0033] DMCM (Discretionary Dispersion Principle) measures the distance between the cluster centers of classes and the mean of all samples in the entire domain, reflecting the inter-class separability. Specifically, for a source domain class, the squared Euclidean distance between its cluster center and the mean of all samples in the source domain is calculated as the inter-class separability distance. For a defect detection domain class, the squared Euclidean distance between its cluster center and the mean of all samples in the source domain is calculated as the inter-class separability distance. After merging all classes, the inter-class separability distances of the source domain and the defect detection domain are added together and divided by the number of classes to obtain the DMCM. Maximizing the DMCM enhances inter-class separability and avoids confusion between different classes.

[0034] DCNC measures the distance between the center of the primary cluster of a class and the centers of the primary clusters of neighboring classes, reflecting the separability of neighboring classes. Specifically, for a source domain class, the squared Euclidean distance between its primary cluster center and the centers of the primary clusters of neighboring classes is calculated as the source domain neighboring class separation distance; for a domain class to be detected, the squared Euclidean distance between its primary cluster center and the centers of the primary clusters of neighboring classes is calculated as the domain neighboring class separation distance; all classes are merged, and the source domain neighboring class separation distance and the domain neighboring class separation distance are added together and divided by the number of classes to obtain the DCNC. Maximizing the DCNC can enhance the separability of neighboring classes and solve the classification problem of easily confused classes.

[0035] These metrics collectively constitute the optimization objective of cross-domain transfer learning, which ultimately achieves highly robust defect detection by aligning inter-domain distributions, enhancing intra-class compactness, and improving inter-class separation.

[0036] The Lagrange algorithm is used to solve the objective function and determine the shared feature matrix of the data space, specifically including: After matrixing the objective function, construct the Lagrange function; Set the partial derivatives to 0, and solve for all eigenvalues ​​and eigenvectors that satisfy the requirements; The obtained eigenvalues ​​are sorted, and the k smallest eigenvalues ​​are selected to form Lagrange multipliers. The k corresponding eigenvectors form a shared feature matrix.

[0037] In this embodiment, a discriminative feature transfer learning model is used to learn from existing labeled source domain sample data and unlabeled defect detection data to solve for the shared feature matrix W. Through this shared feature matrix W, the high-dimensional original data (i.e., defect detection data and source domain sample data) are mapped to a shared, low-dimensional feature space, resulting in mapped source domain sample features and defect detection features.

[0038] To efficiently solve the above objective function, we first matrix-divide the objective function. Matrix-divide the objective function specifically includes: Construct the source domain fundamental matrix, which includes the source domain sample feature matrix, the source domain cluster center matrix, the source domain principal cluster center matrix, and the source domain principal cluster center extended matrix; Construct the basic matrix of the defect detection domain, which includes the sample feature matrix of the defect detection domain, the cluster center matrix of the defect detection domain, the main cluster center matrix of the defect detection domain, and the main cluster center expansion matrix of the defect detection domain. Based on the source domain fundamental matrix and the defect detection domain fundamental matrix, each metric is matrixed, including MCCD, DSC, DCMC, DMCM and DCNC. Substitute the matrix results of each metric into the objective function to give the matrix results of the objective function.

[0039] It is understandable that the columns of the source domain sample feature matrix correspond to the features of each sample in the source domain, the columns of the source domain cluster center matrix correspond to the centers of each cluster in the source domain, and the columns of the source domain principal cluster center matrix correspond to the principal cluster centers of each class in the source domain. Expanding the source domain principal cluster center matrix to have the same dimension as the source domain cluster center matrix yields the expanded source domain principal cluster center matrix. The construction process for each basic matrix in the defect detection domain is similar. Then, based on the basic matrices, metrics such as MCCD, DSC, DCMC, DMCM, and DCNC are calculated to obtain the matrixed results of each metric. Finally, substituting the matrixed results of each metric into the optimization objective function simplifies the objective function while keeping the constraints unchanged.

[0040] To solve the constrained optimization problem, this embodiment uses the Lagrange multiplier method to incorporate the constraints into the objective function, transforming it into an unconstrained optimization problem, and then solving the generalized eigenvalue problem.

[0041] Specifically, based on the matrix transformation results, Lagrange multipliers are introduced, and constraints are added to the objective function to construct the Lagrange function. The partial derivatives of the Lagrange function with respect to the shared eigenvalue matrix are calculated and set to zero, transforming the problem into a generalized eigenvalue problem. The shared eigenvalue matrix becomes an effective solution to the objective function. The generalized eigenvalue problem is solved to obtain all eigenvalues ​​and their corresponding eigenvectors. Numerical computation tools such as the Python package scipy.linalg can be used. Since the objective function is to find the minimum value, the obtained eigenvalues ​​are sorted, and the k smallest eigenvalues ​​are selected to form the Lagrange multipliers. The k corresponding eigenvectors form the shared eigenvalue matrix W.

[0042] In one embodiment, such as Figure 3 As shown, distance analysis is performed on each defect detection feature and all source domain sample features to determine the target defect detection features, target defect detection data, and corresponding first pseudo-labels for each defect, including: Calculate the Euclidean distance between each defect detection feature and all source neighborhood sample features; Arrange the Euclidean distances between each defect detection feature and all source domain sample features in ascending order, and select a predetermined number of source domain sample features that are closest to the defect detection feature. Based on the defect labels of a predetermined number of source domain sample features, determine the initial pseudo-label matrix of defect detection features; A consistency judgment is performed on the initial pseudo-label matrix to determine the target defect detection features, the target defect detection data, and the corresponding first pseudo-label.

[0043] In the scheme for determining the first pseudo-label, pseudo-label learning based on source domain sample data is first performed: in the shared feature space, the Euclidean distance between each defect detection feature and all source domain sample features is calculated. For each defect detection feature, based on the distance, its nearest *a* source domain sample features are found, and the defect labels of these *a* source domain sample features are used as the initial pseudo-label matrix (including *a* initial pseudo-labels) for that defect detection feature. Then, the consistency judgment of the first pseudo-label is performed: the generated *a* initial pseudo-labels are judged. If the *a* initial pseudo-labels are completely identical, the initial pseudo-label matrix is ​​determined to pass the consistency test and can be considered reliable. The defect detection data mapped to this defect detection feature is determined as the target defect detection data, and the corresponding first pseudo-label is given. Conversely, if the *a* initial pseudo-labels are inconsistent, the defect detection data mapped to the defect detection feature is considered to be in a mixed category region, and its initial pseudo-label is unreliable and is filtered out. Thus, target defect detection data with consistent first pseudo-labels are selected.

[0044] In this embodiment, the reasonableness of label transmission is ensured by using the source domain sample information most similar in the data space and based on the similarity after feature alignment. The unlabeled defect detection data is initially labeled by local similarity transmission (source domain samples most similar in the feature space) to generate the first pseudo label. Furthermore, by checking the consistency of the labels of neighboring samples, low-quality pseudo labels located at category boundaries or feature ambiguity areas are effectively filtered out, thus initially improving the purity of the label data used for subsequent steps.

[0045] In one embodiment, such as Figure 4 As shown, based on a pre-built label classification model, the target defect detection data and its corresponding first pseudo-label, as well as the source domain sample data with defect labels, are labeled to obtain the second pseudo-label corresponding to the target defect detection data. Specifically, this includes: The source domain sample data, source domain sample features and defect labels, target defect detection features and the first pseudo label are input into the label classification model; In the label classification model, the optimization objective of the classifier is constructed with the goal of minimizing the difference in classification results between samples of the same class and maximizing the difference in classification results between samples of different classes, and the explicit solution of the optimization objective of the classifier is calculated. Based on the explicit solution, the second pseudo-label corresponding to the target defect detection data is obtained.

[0046] When obtaining the first pseudo-label, the original data is mapped to a low-dimensional shared space using a defined shared feature matrix in the data space, eliminating the distribution differences between the source domain and the defect detection data. When obtaining the second pseudo-label, a label classification model is used to set the optimization objective of the classifier, preserving information from the source domain sample data while adapting to the feature distribution of the defect detection data, thus generating the second pseudo-label.

[0047] The label classification model is a cluster-center-based adaptive classifier (CACC) model. It directly uses the discriminative feature transfer learning based on the aforementioned cluster center to learn features and obtain highly discriminative features, which are used as input to CACC. The maximum cluster center difference and the intra-class compactness and inter-class separation measures based on the cluster center are embedded into the CACC framework. The distribution difference measure and the intra-class compactness and inter-class separation measure are added to the optimization objective of the classifier to minimize the difference in classification results between samples of the same class and maximize the difference in classification results between samples of different classes. A shared classifier is used to solve the problem and improve the classification robustness.

[0048] Furthermore, the optimization objective for constructing the classifier specifically includes: Based on the differences in classification results between the cluster centers of the source domain and the defect detection domain, the differences in classification results between samples of the same cluster and their cluster centers, the differences in classification results between cluster centers of the same cluster, and the differences in classification results between neighboring main cluster centers, the optimization objective of the classifier is constructed.

[0049] Furthermore, based on the explicit solution, the second pseudo-label corresponding to the target defect detection data is obtained, including: Calculate the product of the explicit solution and the kernel function matrix of the target defect detection features; Based on the calculation results, a second pseudo-label is obtained, where the label value of the second pseudo-label is the row where the maximum value of each column vector is located.

[0050] Using the learned shared feature matrix The m-dimensional source domain data features and the defect detection domain data features are mapped to a k-dimensional data space. In this space, a classifier f(x) is solved that can accurately identify the source domain samples and simultaneously minimize the difference in classification results between the corresponding cluster centers of the two domains (i.e., the source domain and the defect detection domain), minimize the difference in classification results between samples of the same cluster and their cluster centers, minimize the difference in classification results between cluster centers of the same cluster, and maximize the difference in classification results between the cluster centers of neighboring classes.

[0051] To explicitly solve for the classifier f(x), we first construct its expression: Where f(x) is the classifier output, x is the target defect detection feature, and 2CK is the total number of cluster centers in the source neighborhood and the defect detection domain. yes The kernel function value of x is a specific value, and the radial basis function kernel is used as the kernel function. , Cluster center features of all classes in the source domain sample features and defect detection features standard deviation This represents the cluster center matrix obtained by clustering using source domain sample features and defect detection features, respectively. express The Column is the feature vector of the i-th cluster center. Here, represents the classifier coefficients, and represents the coefficient vector of the i-th cluster center, which determines its weighted contribution to the classification. The classifier coefficient matrix, express The label matrix of each cluster center, and its column vector This represents a vectorized label, where β is a C x 2CK matrix used to match the label matrix of 2CK cluster centers. Its column vector This represents a vectorized label.

[0052] Under the above classifier expression, based on the principle of minimizing structural risk, and with the objectives of simultaneously minimizing the difference in classification results between the centers of two domain clusters, minimizing the difference in classification results between samples of the same cluster and their cluster centers, minimizing the difference in classification results among centers of the same cluster, and maximizing the difference in classification results among the centers of neighboring main clusters, the adaptive optimization objective of the classifier is constructed as follows: Where β is the classifier coefficient matrix, i.e., the core parameter to be solved, γ1 and γ2 are weighting parameters used to adjust the weights of the intra-class compactness term and the inter-class separation term, and L cls For the classification error of cluster centers, L intra For intra-class compactness, L inter For inter-class separability, λ is a regularization parameter to prevent overfitting. It represents the square of the Euclidean norm.

[0053] It is understandable that the classification error of the cluster center is determined by the kernel matrix and label matrix of the cluster center; the intra-class compactness is jointly determined by the kernel matrix of the source domain sample and the source domain cluster center, the kernel matrix of the source domain sample and its own cluster center, the kernel matrix of the sample to be detected and the cluster center of the defect detection domain, and the kernel matrix of the sample to be detected and its own cluster center; the inter-class separability is jointly determined by the kernel matrix of the same type of cluster center in the source domain, the kernel matrix of the neighboring cluster centers in the source domain, the kernel matrix of the same type of cluster center in the defect detection domain, and the kernel matrix of the neighboring cluster centers in the defect detection domain.

[0054] To efficiently solve for the adaptive optimization of the classifier, we can first expand the kernel matrix (substituting the cluster centers and sample kernel functions) to rewrite the adaptive optimization objective of the classifier into a quadratic form. Then, we take the partial derivative of the adaptive optimization objective with respect to the classifier coefficient matrix β, set the partial derivative to 0, and thus obtain the explicit solution β*. This gives the second pseudo-label. The specific label value is the row where the maximum value of each column vector is located.

[0055] Understandably, the input to CACC is: the mapped, labeled source domain data learned by DFTCC. And the mapped defect detection domain with the first pseudo-label data K in the K-means algorithm; weight parameters γ1 and γ2; balance parameter λ. The output of CACC is: the second pseudo-label of the target defect detection data.

[0056] In this embodiment, pseudo-label learning is performed based on cluster centers. The target defect detection data with consistent pseudo-labels is merged with all source domain sample data. Based on this higher-quality dataset, a classifier adaptive method (CACC) is run to learn and determine the cluster centers of each category in both the source domain and the defect detection domain. Then, using the learned cluster center information, all samples in the defect detection domain are classified to obtain a second pseudo-label based on the cluster centers. Cluster centers represent the average features of samples of the same category, making them more representative than single-source samples. Through "global cluster information," more representative pseudo-labels are provided for the defect detection data, addressing the issue of "local defects transmitted from single-source samples." The adaptive optimization objective also ensures that the pseudo-labels are not affected by "local noise" (such as "abnormal features" of a sample in the source domain).

[0057] In one embodiment, such as Figure 5 As shown, based on the first and second pseudo-labels, the mapping model and label classification model are updated, and the alignment mapping and label update are repeated until a predetermined condition is met, giving the defect label corresponding to the defect detection data, including: For each target defect detection data, compare its first pseudo-label and second pseudo-label. If they match, determine the target defect detection data and its pseudo-label, and use them as the updated defect detection domain sample data. The updated defect detection domain sample data is input into the mapping model and the label classification model to update the model parameters. The updated model parameters include updating the shared feature matrix of the mapping model and updating the classifier coefficient matrix of the label classification model. Repeatedly perform alignment mapping and label updates, and iterate and optimize until the predetermined number of iterations is reached. Once the iteration ends, output the defect label corresponding to the defect detection data.

[0058] For each target defect detection data point, its first pseudo-label based on source domain sample data and its second pseudo-label based on cluster center are compared. Only when the pseudo-labels from the two sources are completely identical is the defect detection data point finally confirmed as a high-reliability sample, and its pseudo-label is adopted for the next iteration of the model. The selected high-reliability updated defect detection domain sample data is fed back into the mapping model and label classification model to update the shared feature matrix of the mapping model and the classifier coefficient matrix of the label classification model. Then, the alignment mapping and label update are repeated in a loop until the predetermined number of iterations is reached, at which point the iteration ends. The mapping model and label classification model after the iteration are used as the final applicable models to analyze the defect detection data, and the final given second pseudo-label is used as the final prediction result for the defect detection data.

[0059] The first pseudo-label represents "local similarity," and the second pseudo-label represents "global clustering," forming a complementary pair. This embodiment achieves "double verification" of the reliability of pseudo-labels by cross-validating defect pseudo-labels from two independent sources. This significantly reduces the risk of erroneous pseudo-labels being introduced into the iterative process, ensuring that only the most reliable samples are used to optimize the model. The iterative loop driven by highly reliable samples allows the model to gradually learn more accurate feature representations and progressively improve the classifier's performance, ultimately resulting in a high-precision, highly robust defect detection model.

[0060] The following section uses defect detection in images from UAV inspections of power transmission lines as an application scenario to elaborate on the specific implementation process of this invention.

[0061] In automated inspections of power transmission lines, a large number of labeled defect images from a specific region or equipment (such as Type A towers) are typically accumulated, including images of "damaged insulators," "bird nests," and "strand breaks." These constitute the source domain sample data. When drones inspect new regions or Type B towers, they acquire a large number of new, unlabeled images. Due to changes in shooting angle, lighting conditions, and background environment, these images exhibit distributional differences from the source domain sample data, thus forming the defect detection data.

[0062] The specific steps for processing the defect detection data are as follows: Step 1: Feature Space Mapping of Transmission Line Images First, labeled defect images (source domain sample data) from the source domain (type A tower) and unlabeled images (defect detection data) from the target domain (type B tower) are input into a pre-trained deep convolutional neural network (such as ResNet) to extract their respective high-dimensional visual features. Then, a discriminative feature transfer learning model (DFTCC) is used to learn a shared feature matrix W. This matrix W maps the visual features of the source and target domains into a unified, low-dimensional shared feature space, resulting in aligned source and target domain features. The purpose of this step is to eliminate the data distribution gap caused by differences in tower type, lighting, background, etc., making the representation of the same defect (such as "insulator damage") in different domains more similar in the feature space.

[0063] Step 2: Defect pseudo-label learning based on source domain images In the aligned shared feature space, for each unlabeled image of the target domain (Type B tower), calculate its feature Euclidean distance to all source domain (Type A tower) images. Sort the images by distance from smallest to largest, and find the 'a' (e.g., a=5) source neighborhood images that are most similar in features for each target image. Then, use the true defect labels of these 5 source neighborhood images (e.g., "insulator damaged", "insulator damaged", "insulator damaged", "bird's nest", "insulator damaged") as the initial pseudo-label set for the target image.

[0064] Step 3: First pseudo-label consistency determination The five pseudo-labels generated for each target image in step two are then subjected to a consistency check. For example, for a target image, if all five pseudo-labels are "damaged insulator," the pseudo-labels are considered to have passed the consistency check, and the target image is preliminarily and reliably labeled as "damaged insulator." If the pseudo-labels are four "damaged insulator" and one "bird's nest," they are considered inconsistent, and the image is believed to be in an area with ambiguous defect features; therefore, the pseudo-labels are temporarily rejected. Through this step, a batch of target domain images with high confidence and consistent pseudo-labels are selected.

[0065] Step 4: Defect pseudo-label learning based on cluster center The target images with consistent pseudo-labels selected in step three are merged with all source domain images to form a higher-quality, cross-domain training set. Based on this training set, a classifier is run to learn the cluster centers of each defect category in the source and target domains within the shared feature space. Then, using these learned, more representative cluster centers, all target domain images are classified, generating a pseudo-label based on the cluster centers for each target image. This constitutes the second source of pseudo-labels.

[0066] Step 5: Second (Multi-Source) Pseudo-Label Consistency Determination For each target image, its pseudo-labels from two sources are compared: one is the pseudo-label based on the source neighborhood samples that passed the consistency check in step three, and the other is the pseudo-label based on the cluster centers generated in step four. Only when the pseudo-labels from the two sources are completely consistent is the target image finally confirmed as a high-reliability sample, and its label is adopted for the next round of model optimization. This "double confirmation" mechanism greatly ensures the accuracy of the pseudo-labels used.

[0067] Step Six: Iterative Optimization and Defect Identification The final high-reliability target image sample set selected in step five is fed back into the model to update and optimize the shared feature representation matrix W in step one and the cluster centers in step four. Then, steps one through six are repeated iteratively. In each iteration, the model's ability to align features between the two domains and its ability to distinguish defect categories are improved due to the use of higher-quality pseudo-labels. After reaching the preset number of iterations, model training is complete. Finally, the trained model is used to identify defects in transmission line images across all target domains, outputting the final defect category.

[0068] This embodiment generates pseudo-labels for target domain samples using two different mechanisms: one is label transfer based on nearest neighbor samples from the source domain, and the other is based on learned cross-domain cluster centers. A two-stage consistency determination mechanism is employed: the first stage judges the consistency of nearest neighbor sample labels for initial screening, and the second stage judges the consistency of multi-source pseudo-labels, ensuring the quality of the pseudo-labels. Only high-reliability samples that pass both-stage consistency determinations are allowed to participate in the next round of model training iterations. This effectively avoids interference from erroneous pseudo-labels in the model iteration process, ensuring the stability and convergence of the learning process, thereby obtaining a more robust final model. Through iterative optimization, the two types of pseudo-labels—one based on source domain samples and the other based on cluster centers—are integrated, and the consistency determination mechanism is used to screen high-reliability samples, thereby improving model performance.

[0069] like Figure 6 As shown, the present invention also provides a defect detection device based on multi-source data, employing the above-mentioned defect detection method based on multi-source data, the device comprising: The data acquisition module is used to acquire the defect detection data and the source domain sample data with defect labels; The feature mapping module is used to align and map the defect detection data and source domain sample data based on a pre-built mapping model, so as to obtain defect detection features and source domain sample features respectively. The initial label determination module is used to perform distance analysis between each defect detection feature and all source domain sample features to determine each target defect detection feature, target defect detection data and corresponding first pseudo-label. The label update module is used to update the labels of the target defect detection data and the corresponding first pseudo-label, as well as the source domain sample data with defect labels, based on a pre-built label classification model, so as to obtain the second pseudo-label corresponding to the target defect detection data. The final label determination module is used to update the mapping model and label classification model based on the first pseudo-label and the second pseudo-label, and repeatedly align the mapping and label update until the predetermined conditions are met, and give the defect label corresponding to the defect detection data.

[0070] This invention considers the need for robust defect detection to suppress the influence of label noise. Addressing the problem of low accuracy in pseudo-label learning methods due to insufficient label information, this invention first analyzes the types and sources of potential label noise and constructs a dataset with noisy labels. Next, based on representation theorems, multi-source pseudo-label learning is performed by integrating class, cluster, and sample information. Using multi-source pseudo-labels and sample labels, a pseudo-label consistency judgment criterion based on multi-source data is designed to obtain the final pseudo-labels, achieving unification of multi-source label information. Finally, the judgment criterion is used to re-judge the labels of all samples in the source domain, achieving label denoising. This allows for the allocation of unified pseudo-labels to the target domain, identifying defects in the target domain, and accurately locating defects based on a defect segmentation deep network. This completes defect detection based on pseudo-label learning, providing decision support for the final defect detection.

[0071] The defect detection method and apparatus based on multi-source data provided by this invention significantly improves the initial accuracy of pseudo-labels by fusing two information sources: cluster centers and source domain samples. The determination strategy based on the consistency of multi-source pseudo-labels does not simply screen according to the confidence ratio, but selects truly reliable samples for training by judging whether pseudo-labels from different sources are consistent. This effectively avoids the interference of erroneous pseudo-labels, ensures the stability and convergence of the iterative process, and ultimately obtains a more robust defect detection model.

[0072] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A defect detection method based on multi-source data, characterized in that, include: Acquire the defect detection data and the source domain sample data with defect labels; Based on a pre-built mapping model, the defect detection data and the source domain sample data are aligned and mapped to obtain the defect detection features and the source domain sample features, respectively. Each defect detection feature is compared with the features of all source domain samples to determine the target defect detection features, target defect detection data and corresponding first pseudo-label. Based on the pre-built label classification model, the target defect detection data and its corresponding first pseudo-label, as well as the source domain sample data with defect labels, are updated to obtain the second pseudo-label corresponding to the target defect detection data. Based on the first pseudo-label and the second pseudo-label, the mapping model and the label classification model are updated, and the alignment mapping and label update are repeated until the predetermined conditions are met, and the defect label corresponding to the defect detection data is given.

2. The defect detection method as described in claim 1, characterized in that, Based on a pre-built mapping model, the defect detection data and source domain sample data are aligned and mapped to obtain defect detection features and source domain sample features, respectively, including: Align and map the defect detection data and the source domain sample data to the same data space; An optimization objective function is constructed with the goal of minimizing intra-class distribution differences and maximizing intra-class compactness and inter-class separation within the data space. The Lagrange algorithm is used to solve the objective function and determine the shared feature matrix of the data space. By aligning and mapping the defect detection data and source domain sample data using a shared feature matrix, the defect detection features and source domain sample features are given.

3. The defect detection method as described in claim 2, characterized in that, The objective function is optimized, specifically expressed as: ； Where min is the minimization function. To optimize the constraints of the objective, W represents the squared Euclidean norm of the shared characteristic matrix W. T The shared feature matrix W is the transpose, where α1 and α2 represent the weights of intra-class compactness and inter-class separability, respectively, and λ represents the regularization parameter. s Let H be the cluster center matrix of all classes in the source domain sample data, and let I be a diagonal matrix. k Let represent the k-dimensional identity matrix, MCCD represent the maximum cluster center dissimilarity, DSC represent the intra-cluster compactness, DCMC represent the compactness of multiple clusters within the same class, DMCM represent the inter-cluster separation, DCNC represent the separation of neighboring clusters, (DSC+DCMC) represents the intra-cluster compactness, and (DMCM+DCNC) represents the inter-cluster separation.

4. The defect detection method as described in claim 3, characterized in that, The Lagrange algorithm is used to solve the objective function and determine the shared feature matrix of the data space, including: After matrixing the objective function, construct the Lagrange function; Set the partial derivatives to 0, and solve for all eigenvalues ​​and eigenvectors that satisfy the requirements; The obtained eigenvalues ​​are sorted, and the k smallest eigenvalues ​​are selected to form Lagrange multipliers. The k corresponding eigenvectors form a shared feature matrix, where k is a natural number.

5. The defect detection method as described in claim 1, characterized in that, Distance analysis is performed on each defect detection feature and all source domain sample features to determine the target defect detection features, target defect detection data, and corresponding first pseudo-labels for each target defect, including: Calculate the Euclidean distance between each defect detection feature and all source neighborhood sample features; Arrange the Euclidean distances between each defect detection feature and all source domain sample features in ascending order, and select a predetermined number of source domain sample features that are closest to the defect detection feature. Based on the defect labels of a predetermined number of source domain sample features, determine the initial pseudo-label matrix of defect detection features; A consistency judgment is performed on the initial pseudo-label matrix to determine the target defect detection features, the target defect detection data, and the corresponding first pseudo-label.

6. The defect detection method as described in claim 1, characterized in that, Based on a pre-built label classification model, the target defect detection data and its corresponding first pseudo-label, as well as the source domain sample data with defect labels, are updated to obtain the second pseudo-label corresponding to the target defect detection data, specifically including: The source domain sample data, source domain sample features and defect labels, target defect detection features and the first pseudo label are input into the label classification model; In the label classification model, the optimization objective of the classifier is constructed with the goal of minimizing the difference in classification results between samples of the same class and maximizing the difference in classification results between samples of different classes, and the explicit solution of the optimization objective of the classifier is calculated. Based on the explicit solution, the second pseudo-label corresponding to the target defect detection data is obtained.

7. The defect detection method as described in claim 6, characterized in that, The optimization objectives for constructing the classifier include: Based on the differences in classification results between the cluster centers of the source domain and the defect detection domain, the differences in classification results between samples of the same cluster and their cluster centers, the differences in classification results between cluster centers of the same cluster, and the differences in classification results between neighboring main cluster centers, the optimization objective of the classifier is constructed.

8. The defect detection method as described in claim 6, characterized in that, Based on the explicit solution, the second pseudo-label corresponding to the target defect detection data is obtained, including: Calculate the product of the explicit solution and the kernel function matrix of the target defect detection features; Based on the calculation results, a second pseudo-label is obtained, where the label value of the second pseudo-label is the row where the maximum value of each column vector is located.

9. The defect detection method as described in claim 1, characterized in that, Based on the first and second pseudo-labels, the mapping model and label classification model are updated, and the alignment mapping and label updates are repeated until predetermined conditions are met, thus providing the defect labels corresponding to the defect detection data, including: For each target defect detection data, compare its first pseudo-label and second pseudo-label. If they match, determine the target defect detection data and its pseudo-label, and use them as the updated defect detection domain sample data. The updated defect detection domain sample data is input into the mapping model and the label classification model to update the model parameters. The updated model parameters include updating the shared feature matrix of the mapping model and updating the classifier coefficient matrix of the label classification model. Repeatedly perform alignment mapping and label updates, and iterate and optimize until the predetermined number of iterations is reached. Once the iteration ends, output the defect label corresponding to the defect detection data.

10. A defect detection device based on multi-source data, characterized in that, The apparatus employing the defect detection method based on multi-source data as described in any one of claims 1 to 9 includes: The data acquisition module is used to acquire the defect detection data and the source domain sample data with defect labels; The feature mapping module is used to align and map the defect detection data and source domain sample data based on a pre-built mapping model, so as to obtain defect detection features and source domain sample features respectively. The initial label determination module is used to perform distance analysis between each defect detection feature and all source domain sample features to determine each target defect detection feature, target defect detection data and corresponding first pseudo-label. The label update module is used to update the labels of the target defect detection data and the corresponding first pseudo-label, as well as the source domain sample data with defect labels, based on a pre-built label classification model, so as to obtain the second pseudo-label corresponding to the target defect detection data. The final label determination module is used to update the mapping model and label classification model based on the first pseudo-label and the second pseudo-label, and repeatedly align the mapping and label update until the predetermined conditions are met, and give the defect label corresponding to the defect detection data.

Images