Method and apparatus for estimating number of industrial defect classes based on prototype vector correlation
Patent Information
- Application Number
- CN202611110083.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-24
- Publication Date
- 2026-10-30
AI Technical Summary
[0005]本发明要解决的技术问题是现有技术由于无法预先确定产线上可能出现的所有的缺陷的类别数量，导致缺陷识别的准确率和可用性低的问题
本发明通过引入已知类别数的有标签图像集合作为外部参照，利用其类别原型向量之间的相关性为无标签图像集合设定相关性阈值，将相关性阈值的计算与无标签图像集合解耦，避免了对无标签图像集合中数据分布的依赖，进而能够利用相关性阈值对无标签原型向量矩阵进行循环去重，仅对无标签原型向量矩阵进行操作，与无标签图像集合中图像的总量无关，从而在不依赖预先确定实际缺陷类别数量的前提下，实现精准估计无标签图像集合的缺陷类别数量，从而提高自动化缺陷识别的准确率和可用性。
Smart Images
Figure CN123015409A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a method and apparatus for estimating the number of industrial defect categories based on prototype vector correlation. Background Technology
[0002] In the field of intelligent manufacturing, accurately identifying and classifying various defects that may exist during the production process is a key technical step in achieving closed-loop quality control and production line optimization. The core objective of identifying defects in the production process is to categorize them according to their morphology, location, and cause, thereby enabling batch processing and targeted repair of products with the same type of defects in subsequent steps. More importantly, it provides crucial evidence for defect tracing and process improvement in the production process, thus systematically improving product yield and production efficiency.
[0003] Currently, with the popularization of artificial intelligence technology, automatic defect classification schemes based on computer vision have become mainstream. However, existing industrial defect classification schemes require prior knowledge of the specific number of defect categories in order to design the output dimension of the classifier or define the number of clusters. But in real-world applications, even with the help of process experts, it is often difficult to accurately predict all possible defect forms and their exact number on the production line. Therefore, the unknown number of defect categories has become a key bottleneck restricting the practical application of existing unsupervised defect classification methods on actual production lines.
[0004] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention
[0005] The technical problem to be solved by the present invention is that the existing technology has low accuracy and usability in defect identification because it cannot predict the number of all types of defects that may occur on the production line.
[0006] Firstly, a method for estimating the number of industrial defect categories based on prototype vector correlation is provided, including: Prepare a pre-trained feature extraction network to extract features from a set of unlabeled images. and a collection of labeled images Features of all images in the dataset; The extracted features are processed separately using a defect classifier during training. After the round, the weight matrix in the defect classifier is extracted as the prototype vector matrix to obtain the labeled prototype vector matrix. and unlabeled prototype vector matrix ; According to the labeled prototype vector matrix The correlation threshold is obtained from the correlation between prototype vectors. ; Using the aforementioned correlation threshold For the unlabeled prototype vector matrix Perform a loop to remove duplicates, resulting in a set of selected vectors. ; select the set of vectors The number of prototype vectors in the unlabeled image set is determined as follows. Defect category quantity estimate .
[0007] Furthermore, in the process of using a defect classifier to process the extracted features separately during training... After the round, the weight matrix in the defect classifier is extracted as the prototype vector matrix to obtain the labeled prototype vector matrix. and unlabeled prototype vector matrix Previously, it also included: Predefined maximum number of categories As a collection of unlabeled images An initial value for the number of defect categories; using a set of labeled images. Number of defect categories and the maximum number of categories The output dimensions of the labeled and unlabeled parts of the corresponding defect classifiers are initialized separately to obtain the weight matrices of the labeled parts. Weight matrix of unlabeled part ,in, For feature dimensions.
[0008] Furthermore, the step of using the labeled prototype vector matrix... The correlation threshold is obtained from the correlation between prototype vectors. include: Calculate the labeled prototype vector matrix Correlation matrix between prototype vectors ; For the correlation matrix The correlation threshold is obtained by averaging all elements in the equation. .
[0009] Furthermore, the method also includes: The unlabeled image set The feature clustering of all images is the maximum number of categories. For each of the given clusters, the features are averaged to obtain the central feature of that cluster. The central features of all clusters are then concatenated to obtain an unlabeled prototype vector matrix. ;in, For feature dimensions; The labeled image set The feature clustering of all images is the number of defect categories. The system identifies several clusters. The central feature of each cluster is obtained by averaging the features within that cluster. The central features of all clusters are then concatenated to obtain a labeled prototype vector matrix. .
[0010] Furthermore, the use of the correlation threshold For the unlabeled prototype vector matrix Perform a loop to remove duplicates, resulting in a set of selected vectors. include: Retain the unlabeled prototype vector matrix The correlation between prototype vectors is lower than the correlation threshold. The prototype vector, excluding the unlabeled prototype vector matrix. The correlation between prototype vectors is higher than or equal to the correlation threshold. The prototype vectors are processed until the stopping condition is met, thus obtaining the set of selected vectors. .
[0011] Furthermore, the method also includes: In the first iteration, from the unlabeled prototype vector matrix Randomly select a prototype vector as the initial seed, and add the initial seed to the set of selected vectors. In the process, the remaining unselected prototype vectors are added to the prototype vector set. Calculate the set of prototype vectors Each prototype vector Maximum correlation with the initial seed; Find the prototype vector with the lowest maximum relevance. If the maximum relevance of the prototype vector is less than the relevance threshold Then the prototype vector with the lowest maximum correlation will be selected. Add to the set of selected vectors In, and from the set of prototype vectors Remove from middle; Starting from the second iteration, the prototype vector set is calculated. Each prototype vector and the set of selected vectors The maximum correlation of each prototype vector is used to progressively update the selected vector set. until the maximum correlation is greater than or equal to the correlation threshold. or prototype vector set If the middle element is empty, the iteration stops.
[0012] Furthermore, the expression for the maximum correlation is: ; in, Represents the set of prototype vectors The prototype vector in This represents the set of selected vectors.
[0013] Furthermore, in the selection of the vector set The number of prototype vectors in the unlabeled image set is determined as follows. Defect category quantity estimate Subsequently, the method further includes: The weight matrix of the defect classifier Not included in the set of selected vectors The prototype vectors in the model are pruned to obtain the weight matrix of the defect classifier. Using the weight matrix Continue with the subsequent training process; in, Indicates the maximum number of categories. Represents a set of labeled images The number of defect categories, This represents an estimated number of defect categories. For feature dimensions.
[0014] Secondly, an industrial defect category quantity estimation device based on prototype vector correlation is provided, the industrial defect category quantity estimation device based on prototype vector correlation includes: a processor and a memory for storing processor-executable instructions; The processor is configured to execute the industrial defect category quantity estimation method based on prototype vector correlation.
[0015] Thirdly, a non-volatile computer storage medium is provided, the computer storage medium storing computer-executable instructions, which are executed by one or more processors to perform the industrial defect category quantity estimation method based on prototype vector correlation described in the first aspect.
[0016] Fourthly, a computer program product containing instructions is provided that, when executed on a computer or processor, causes the computer or processor to perform the industrial defect category quantity estimation method based on prototype vector correlation as described in the first aspect.
[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention introduces a set of labeled images with a known number of categories as an external reference. It uses the correlation between the prototype vectors of these categories to set a correlation threshold for the unlabeled image set, decoupling the calculation of the correlation threshold from the unlabeled image set. This avoids dependence on the data distribution in the unlabeled image set, and allows for iterative deduplication of the unlabeled prototype vector matrix using the correlation threshold. The operation is performed only on the unlabeled prototype vector matrix and is independent of the total number of images in the unlabeled image set. Thus, without relying on a pre-determined number of actual defect categories, it can accurately estimate the number of defect categories in the unlabeled image set, thereby improving the accuracy and usability of automated defect identification. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating a method for estimating the number of industrial defect categories based on prototype vector correlation, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating a specific example of an industrial defect category quantity estimation method based on prototype vector correlation provided in an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating a specific example of a cyclic deduplication process provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an industrial defect category quantity estimation device based on prototype vector correlation provided in an embodiment of the present invention. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0021] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms for reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.
[0022] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.
[0023] In describing some embodiments, the terms "coupled," "coupled," and "connected," and their derivative expressions, may be used. For example, the term "connected" may be used in describing some embodiments to indicate that two or more components have direct physical or electrical contact with each other. Similarly, the term "coupled" may be used in describing some embodiments to indicate that two or more components have direct physical or electrical contact. However, the terms "connected" or "coupled" may also refer to two or more components that do not have direct contact with each other but still cooperate or interact with each other, such as "optical coupling," "wireless connection," etc. The embodiments disclosed herein are not necessarily limited to the scope of this invention.
[0024] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0025] In existing computer vision-based automatic defect classification schemes, one type of defect classification method relies on supervised learning. That is, a large number of samples covering all possible defect types are collected and labeled manually in advance to train a classification model with a fixed network architecture. However, in real industrial production lines, the occurrence of defects is sporadic and diverse, and comprehensively collecting and labeling samples of all possible defects requires extremely high manpower and time costs.
[0026] To address this issue, some manufacturing companies have adopted clustering-based methods, relying on hierarchical clustering or spectral clustering algorithms, to classify different defect categories without the need for manual labeling.
[0027] In addition, some defect classification methods are based on deep neural networks. To make the features extracted by deep neural networks more applicable to industrial scenarios, a new category discovery-based approach uses unlabeled data to perform self-supervised training on the feature extractor and classifier.
[0028] All of the above industrial defect classification schemes require prior knowledge of the specific categories and number of defects in order to design the output dimension of the classifier or define the number of clusters.
[0029] However, process experts often find it difficult to accurately predict all possible defect types and their exact numbers on the production line.
[0030] When using existing technical solutions, setting too many pre-defined defect categories can lead to overfitting of the AI model or the generation of meaningless dummy categories. On one hand, too many categories result in a surge in model parameters, increasing the hardware and software costs for model training and inference. On the other hand, in practical use, since dummy categories are essentially random noise regions in the feature space, when the model incorrectly activates a normal product (or a good product with slight noise) to a dummy category, the normal product will be automatically sorted as a defective product and scrapped, resulting in artificially low production line yield data, increased unnecessary material waste, or a significant increase in manual review workload, negating the purpose of automated defect identification. In this case, the defect identification data cannot be used to optimize subsequent production processes for defect tracing and process improvement. Conversely, if the number of categories is insufficient, it is impossible to effectively distinguish all real defect types, leading to the incorrect merging of multiple defect types, impairing the granularity and usability of the classification. On the one hand, the model is unable to uncover the subcategories of defects hidden within the larger categories, and it cannot continuously evolve based on newly generated data from the production line. Once the model is deployed, its performance becomes fixed, and its usability gradually decreases over time. On the other hand, in actual use, the quality statistics reports generated based on the model's output are severely distorted. Process engineers cannot accurately identify the root causes of frequently occurring defects on the corresponding production line, making it impossible to rectify and optimize defects in a timely manner. Furthermore, the physical repair methods for different defects are completely different, and the formation stages and severity levels of various defects also vary. Due to the coarse granularity of the classification, it is impossible to formulate and implement differentiated repair strategies based on the physical causes and severity levels of different defects, and it is also impossible to prioritize the rectification of defects that cause serious harm.
[0031] Therefore, the unknown number of defect categories has become a key bottleneck restricting the practical application of existing unsupervised defect classification methods in actual production lines.
[0032] To address this quantity estimation problem, most existing strategies require training additional prediction networks or rely on the elbow rule. However, since different types of defects have different distribution densities in the feature space, these strategies cannot accurately estimate the number of defects. Furthermore, regardless of the strategy, repeated training or clustering of all samples is required, resulting in a lengthy estimation process with a time complexity greater than O(n), severely limiting its applicability in real-world scenarios.
[0033] To address the problems of existing technologies, this embodiment proposes a method for estimating the number of industrial defect categories based on prototype vector correlation. In one embodiment, such as... Figure 1 As shown, it includes: Step 10: Prepare a pre-trained feature extraction network to extract unlabeled image sets. and a collection of labeled images Features of all images in the dataset.
[0034] Among them, the unlabeled image set The set of unlabeled target images for which the number of defect categories needs to be estimated. The set of labeled images. This is a set of labeled images with a known number of defect categories, whose defect categories do not overlap with the target image set. In this embodiment of the invention, it is used to provide a reference scale, compared to an unlabeled image set. It provides assistance in defect category identification.
[0035] Among them, extracting the unlabeled image set and a collection of labeled images The specific method for feature extraction of all images is determined by those skilled in the art based on the specific use case. In one embodiment, the feature extractor is a pre-trained feature extraction network responsible for mapping the input image into a high-dimensional feature vector. Extracting an unlabeled image set. and a collection of labeled images Features of all images in the dataset refer to the extraction of unlabeled image sets using feature extractors with the same structure and shared parameters. Features of all images in the set of labeled images Features of all images in the dataset.
[0036] like Figure 2 As shown, two types of input data are obtained (i.e., a set of unlabeled images). and a collection of labeled images The system utilizes a shared pre-trained feature extraction network to convert each extracted image into a numerical feature vector. Then, the feature vector is mapped to a high-dimensional feature vector, thereby extracting the features of the image through the feature extractor. The extracted features are the output of the corresponding image after passing through the feature extractor, and are the input of the subsequent defect classifier.
[0037] Step 20: Process the extracted features separately using a defect classifier during training. After the round, the weight matrix in the defect classifier is extracted as the prototype vector matrix to obtain the labeled prototype vector matrix. and unlabeled prototype vector matrix .
[0038] Among them, the specific network structure and number of training rounds of the preset defect classifier. The specific use case will be determined by those skilled in the art. During training, the unlabeled image set... and a collection of labeled images They share the same pre-defined defect classifier structure and parameters. In one embodiment, the defect classifier can be implemented using a linear classifier. Figure 2The specific example shown is illustrated using a linear classifier.
[0039] Unlabeled prototype vector matrix For the corresponding unlabeled portion extracted from the defect classifier (i.e., the set of unlabeled images) The set of weight vectors. It contains a label prototype vector matrix. For the corresponding labeled parts extracted from the defect classifier (i.e., the set of labeled images) The set of weight vectors is a reference scale used in this embodiment of the invention to generate the correlation threshold in step 30 below.
[0040] The weight parameters of the defect classifier are the trainable parameter matrix within the defect classifier, during training. Information on the classification of rear-wheel bearing defects, such as Figure 2 As shown, in this embodiment of the invention, two sets of features (i.e., the set of unlabeled images extracted in step 10) are used. Features and labeled image sets (Features) are input into a preset defect classifier for training. After the round, the trainable weight parameters in the defect classifier are frozen and extracted, and used as the class prototype center of the corresponding dataset.
[0041] Step 30: Based on the labeled prototype vector matrix The correlation threshold is obtained from the correlation between prototype vectors. .
[0042] This invention embodiment only utilizes the labeled prototype vector matrix The correlation between internal vectors is used to determine a fixed merging threshold as the correlation threshold. A specific example will be provided below, and will not be elaborated further here.
[0043] Step 40: Use the aforementioned correlation threshold For the unlabeled prototype vector matrix Perform a loop to remove duplicates, resulting in a set of selected vectors. ; select the set of vectors The number of prototype vectors in the unlabeled image set is determined as follows. Defect category quantity estimate .
[0044] Among them, the selected vector set This is the set of prototype vectors accumulated gradually during the iterative deduplication process, representing the centers of the defect categories that are determined to be truly existing. For example... Figure 2 As shown, the estimated number of defect categories To count the total number of prototype vectors in the selected vector set, i.e., the final output set of unlabeled images. The number of defect categories. The specific method for performing iterative deduplication shall be determined by those skilled in the art based on the specific use case.
[0045] In one embodiment, a greedy forward strategy can be used, utilizing a correlation threshold. For unlabeled prototype vector matrix Perform iterative filtering; by selecting vectors from the already selected set at each step. The prototype vector that is least similar to the prototype vector in the model is considered if its similarity is below the relevance threshold. If the original prototype is retained, continue; otherwise, stop. The number of retained prototypes is the estimated number of defect categories. This will be explained further below, and will not be repeated here.
[0046] This invention introduces a set of labeled images with a known number of categories as an external reference. It uses the correlation between the prototype vectors of these categories to set a correlation threshold for the unlabeled image set, decoupling the calculation of the correlation threshold from the unlabeled image set. This avoids dependence on the data distribution in the unlabeled image set, and allows for iterative deduplication of the unlabeled prototype vector matrix using the correlation threshold. The operation is performed only on the unlabeled prototype vector matrix and is independent of the total number of images in the unlabeled image set. Thus, without relying on a pre-determined number of actual defect categories, it can accurately estimate the number of defect categories in the unlabeled image set, thereby improving the accuracy and usability of automated defect identification.
[0047] The following is a further explanation of the industrial defect category quantity estimation method based on prototype vector correlation in this invention: Unlabeled image collection in a real production line The number of defect categories is not always available, so this invention provides a defect category number estimation strategy based on prototype vector correlation.
[0048] First, a set of unlabeled images from a target production line is given. ,in, For the first Zhang Image A collection of unlabeled images The total number of images in the batch. The ultimate goal of the method in this embodiment of the invention is to accurately predict the number of defects in this batch of images. .
[0049] Another set of labeled images is given. ,in, For the first Zhang Image The category label corresponding to this image. For this set of labeled images The total number. A collection of labeled images. Unlabeled image set used to assist in target prediction The number of defect categories in the set of labeled images Contains known Different types of defects, and with unlabeled image sets. The defect types in the two cases do not overlap.
[0050] like Figure 2 As shown, the specific steps are as follows: In one embodiment, in step 10, a pre-trained feature extraction network is first prepared to extract a set of unlabeled images. and a collection of labeled images Features of all images in the dataset. In an alternative embodiment, the ViT-B-8 model, pre-trained on the ImageNet dataset using a label-free distillation (DINO) strategy, can be used as the feature extractor.
[0051] The collection of unlabeled images extracted by the feature extractor Images in The characteristics are Extract the set of labeled images Images in The characteristics are ,in, For feature dimensions.
[0052] To improve the compatibility of the method in this embodiment of the invention, enabling the industrial defect category quantity estimation method based on prototype vector correlation in this embodiment of the invention to be combined with existing industrial defect classification methods, obtain prototype vectors of different defect categories from existing methods, and subsequently directly operate on the prototype vectors, in one embodiment, before step 20, the method further includes: Step 201: Predefine the maximum number of categories As a collection of unlabeled images The initial value for the number of defect categories.
[0053] Step 202: Use a set of labeled images Number of defect categories and the maximum number of categories The output dimensions of the labeled and unlabeled parts of the corresponding defect classifiers are initialized separately to obtain the weight matrices of the labeled parts. Weight matrix of unlabeled part ,in, For feature dimensions.
[0054] The embodiments of the present invention employ the number of defect categories in the labeled image set. and the maximum number of categories The output dimensions of the labeled and unlabeled parts of the defect classifier are initialized separately to obtain the weight matrices of the labeled parts. Weight matrix of unlabeled part Then during training After each round, the weight matrix of each defect classifier is extracted as the corresponding prototype vector matrix.
[0055] Among them, the largest number of categories and the preset maximum number of categories The specific methods are determined by those skilled in the art based on the specific use case, and are not limited here.
[0056] In one embodiment, firstly, a maximum number of categories is predefined. As a collection of unlabeled images The initial value for the number of defect categories. Wherein, Represents a set of labeled images. The number of defect categories included.
[0057] Among them, the largest number of categories Larger than the set of unlabeled images The actual number of categories, i.e., the maximum number of categories. It is definitely greater than the estimated number of defect categories. Maximum number of categories This indicates the maximum number of possible defect categories.
[0058] It should be noted that the maximum number of categories This invention provides an upper limit to the order of magnitude of defects, based on common sense (e.g., number of processes, historical experience) from production line experts, rather than a precisely calculated value. In practical applications, while experts cannot predict the specific number of categories, they have absolute control over the order of magnitude (e.g., the number of defect categories in a mobile phone screen production line cannot exceed 100). Therefore, the method of this invention ensures the maximum number of categories. Larger than the set of unlabeled images The actual number of categories; even the preset maximum number of categories. The relevance threshold can be automatically passed through the screening mechanism of this invention (i.e., the iterative deduplication process in step 40). All redundant virtual classes are removed. (Due to the maximum number of classes...) The value is determined based on the physical limits of the production line. Under rigorous engineering logic, as long as the maximum number of categories is preset... The limit is set based on the production line's maximum number of processes (for example, if the production line has a maximum of 20 processes, and each process can produce a maximum of 1 to 2 types of defects, then the limit is 40), so it cannot be less than the actual value; however, if extreme cases occur, the preset maximum number of categories... If the number of real prototypes is still less than the actual number of categories, then the screening process in step 40 will not be able to traverse all real categories because there are simply not enough real prototypes in the candidate pool. In actual operation, this situation is a violation of the input data, rather than a problem with the method of this embodiment of the invention. The solution should be determined by those skilled in the art based on the specific use case.
[0059] This invention iteratively determines the set of unlabeled images by automatically compressing them to the true number after giving a loose upper limit. Defect category quantity estimate .
[0060] In another embodiment, the labeled prototype vector matrix can also be obtained by the following method. and unlabeled prototype vector matrix The method further includes: The unlabeled image set The feature clustering of all images is the maximum number of categories. For each of the given clusters, the features are averaged to obtain the central feature of that cluster. The central features of all clusters are then concatenated to obtain an unlabeled prototype vector matrix. ,in, For feature dimensions.
[0061] The labeled image set The feature clustering of all auxiliary images is the number of defect categories. The system identifies several clusters. The central feature of each cluster is obtained by averaging the features within that cluster. The central features of all clusters are then concatenated to obtain a labeled prototype vector matrix. .
[0062] In one embodiment, for a clustering-based method, the unlabeled image set... All image features are clustered as There are 3 clusters, whose cluster centers are sets of unlabeled images. Unlabeled prototype vector matrix For a set of labeled images All image features are clustered as There are 3 clusters, whose cluster centers are sets of labeled images. Labeled prototype vector matrix .
[0063] In this embodiment of the invention, the cluster center vectors are arranged in rows to form the prototype vector matrix of the set, where each row represents the feature center of a defect category.
[0064] To illustrate the process of determining the relevance threshold, in one embodiment, step 30 includes: Step 301: Calculate the labeled prototype vector matrix Correlation matrix between prototype vectors .
[0065] In one embodiment, the tag prototype vector matrix is calculated respectively. Correlation matrix between prototype vectors and unlabeled prototype vector matrix Correlation matrix between prototype vectors .
[0066] Step 302: For the correlation matrix The correlation threshold is obtained by averaging all elements in the equation. .
[0067] The present invention subsequently uses a correlation threshold Implement the use of labeled prototype vector matrices The correlation between prototype vectors in the matrix is used to roughly estimate the unlabeled prototype vector matrix. The correlation between prototype vectors in the matrix. In one embodiment, this can be achieved by analyzing the correlation matrix. The correlation threshold is obtained by averaging all elements in the equation. .
[0068] In order to obtain from the unlabeled prototype vector matrix The process involves selecting true prototype vectors and pruning redundant prototype vectors. In one embodiment, in step 40, the relevance threshold is used. For the unlabeled prototype vector matrix Perform a loop to remove duplicates, resulting in a set of selected vectors. include: Retain the unlabeled prototype vector matrix The correlation between prototype vectors is lower than the correlation threshold. The prototype vector, excluding the unlabeled prototype vector matrix. The correlation between prototype vectors is higher than or equal to the correlation threshold. The prototype vectors are processed until the stopping condition is met, thus obtaining the set of selected vectors. .
[0069] In one embodiment, such as Figure 3As shown, in step 40, the above steps include: Step 401: In the first iteration, from the unlabeled prototype vector matrix Randomly select a prototype vector as the initial seed, and add the initial seed to the set of selected vectors. In the process, the remaining unselected prototype vectors are added to the prototype vector set. Calculate the set of prototype vectors Each prototype vector The maximum correlation with the initial seed.
[0070] Then calculate the prototype vector set. Each prototype vector and the set of selected vectors Maximum correlation of the prototype vector (i.e., the initial seed) In one embodiment, the expression for the maximum correlation is: ; in, Represents the set of prototype vectors The prototype vector in This represents the set of selected vectors.
[0071] Step 402: Obtain the prototype vector with the minimum maximum correlation. If the maximum relevance of the prototype vector is less than the relevance threshold Then the prototype vector with the lowest maximum correlation will be selected. Add to the set of selected vectors In, and from the set of prototype vectors Remove from the middle.
[0072] In one embodiment, if the prototype vector set The highest correlation Minimum prototype vector Less than the correlation threshold Then the prototype vector Add to the set of selected vectors In, and from the set of prototype vectors Remove from the middle.
[0073] Step 403: Starting from the second iteration, calculate the prototype vector set. Each prototype vector and the set of selected vectors The maximum correlation of each prototype vector is used to progressively update the selected vector set. until the maximum correlation is greater than or equal to the correlation threshold. or prototype vector set If the middle element is empty, the iteration stops.
[0074] The stopping condition is determined by those skilled in the art based on the specific application scenario. The stopping condition can be: a set of prototype vectors. The smallest prototype vector is greater than the correlation threshold. Or, the set of prototype vectors The elements in the middle are empty. The set of prototype vectors is also included. and the set of selected vectors The set of prototype vectors changes dynamically in each iteration. The prototype vector with the lowest maximum correlation is selected from the set of prototype vectors. Remove from the set of selected vectors and add to the set of selected vectors. And in the next iteration, the set of prototype vectors is calculated. Each prototype vector in the set of selected vectors The maximum correlation of each prototype vector, and so on.
[0075] The final set of selected vectors The number of prototype vectors in the unlabeled image set Defect category quantity estimate .
[0076] In one embodiment, after step 40, the method further includes: adjusting the weight matrix of the defect classifier. Not included in the set of selected vectors The prototype vectors in the model are pruned to obtain the weight matrix of the defect classifier. Using the weight matrix Continue with the subsequent training process.
[0077] The industrial defect category quantity estimation method based on prototype vector correlation of the present invention, as a strategy, has a time complexity that depends only on the number of prototype vectors, and is less dependent on the unlabeled image set. The number of images in the dataset is independent. This reduces the time complexity from greater than or equal to O(n) to O(1), significantly improving the speed of defect category identification. Furthermore, this invention operates directly on the prototype vector, thus possessing greater universality and compatibility with various defect classification frameworks based on supervised learning, clustering, and new category discovery. In addition, this invention uses other labeled image sets from the industrial field as references to remove redundant defect categories from unlabeled image sets, reducing domain shift during pruning and greatly improving prediction accuracy.
[0078] In one embodiment, the industrial defect category number estimation method based on prototype vector correlation of the present invention can be combined with the clustering-based method in the prior art to automatically merge redundant categories and more accurately estimate the number of true defect categories.
[0079] In another embodiment, the industrial defect category number estimation method based on prototype vector correlation of the present invention can also be combined with methods based on supervised learning or new category discovery to automatically estimate the number of defect categories during the training process and prune redundant category dimensions in the classifier, thereby obtaining more accurate multi-class defect classification results.
[0080] The foregoing embodiments provided a method for estimating the number of industrial defect categories based on prototype vector correlation. In this embodiment, another device for estimating the number of industrial defect categories based on prototype vector correlation is proposed. The device includes a processor and a memory for storing processor-executable instructions. The processor is configured to execute the method for estimating the number of industrial defect categories based on prototype vector correlation described in the foregoing embodiments.
[0081] like Figure 4 As shown, the industrial defect category quantity estimation device based on prototype vector correlation includes a processor 21 and a memory 22, wherein the processor 21 and the memory 22 can be connected by a bus or other means.
[0082] Processor 21 can be a CPU. Processor 21 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.
[0083] Memory 22, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the industrial defect category quantity estimation method based on prototype vector correlation in the aforementioned embodiments. The processor executes various functional applications and training processes by running the non-transitory software programs, instructions, and modules stored in the memory.
[0084] The memory 22 may include a program storage area and a training storage area. The program storage area may store the operating system and applications required for at least one function; the training storage area may store training data, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 22 may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, enterprise intranets, local area networks, mobile communication networks, and combinations thereof. The one or more modules stored in the memory 22, when executed by the processor 21, perform the industrial defect category quantity estimation method based on prototype vector correlation as shown in the embodiments of the present invention. Specific details of the above-described industrial defect category quantity estimation method based on prototype vector correlation can be understood by referring to the corresponding descriptions and effects in the embodiments of the present invention, and will not be repeated here.
[0085] This embodiment also provides a computer storage medium storing a computer program that can be executed by a processor to perform the industrial defect category quantity estimation method based on prototype vector correlation described in the foregoing embodiments.
[0086] The computer storage medium stores computer-executable instructions, which can execute the industrial defect category quantity estimation method based on prototype vector correlation in any of the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.
[0087] The specific steps of the industrial defect category quantity estimation method based on prototype vector correlation are described in the foregoing embodiments and will not be repeated in this embodiment.
[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for estimating the number of industrial defect categories based on prototype vector correlation, characterized in that, include: Prepare a pre-trained feature extraction network to extract features from a set of unlabeled images. and a collection of labeled images Features of all images in the dataset; The extracted features are processed separately using a defect classifier during training. After the round, the weight matrix in the defect classifier is extracted as the prototype vector matrix to obtain the labeled prototype vector matrix. and unlabeled prototype vector matrix ; According to the labeled prototype vector matrix The correlation threshold is obtained from the correlation between prototype vectors. ; Using the aforementioned correlation threshold For the unlabeled prototype vector matrix Perform a loop to remove duplicates, resulting in a set of selected vectors. ; select the set of vectors The number of prototype vectors in the unlabeled image set is determined as follows. Defect category quantity estimate .
2. The method for estimating the number of industrial defect categories based on prototype vector correlation according to claim 1, characterized in that, The extracted features are processed separately using a defect classifier during training. After the round, the weight matrix in the defect classifier is extracted as the prototype vector matrix to obtain the labeled prototype vector matrix. and unlabeled prototype vector matrix Previously, it also included: Predefined maximum number of categories As a collection of unlabeled images The initial value for the number of defect categories; Using a set of labeled images Number of defect categories and the maximum number of categories The output dimensions of the labeled and unlabeled parts of the corresponding defect classifiers are initialized separately to obtain the weight matrices of the labeled parts. Weight matrix of unlabeled part ,in, For feature dimensions.
3. The method for estimating the number of industrial defect categories based on prototype vector correlation according to claim 2, characterized in that, The based on the labeled prototype vector matrix The correlation threshold is obtained from the correlation between prototype vectors. include: Calculate the labeled prototype vector matrix Correlation matrix between prototype vectors ; For the correlation matrix The correlation threshold is obtained by averaging all elements in the equation. .
4. The method for estimating the number of industrial defect categories based on prototype vector correlation according to claim 1, characterized in that, The method further includes: The unlabeled image set The feature clustering of all images is the maximum number of categories. For each of the given clusters, the features are averaged to obtain the central feature of that cluster. The central features of all clusters are then concatenated to obtain an unlabeled prototype vector matrix. ;in, For feature dimensions; The labeled image set The feature clustering of all images is the number of defect categories. The system identifies several clusters. The central feature of each cluster is obtained by averaging the features within that cluster. The central features of all clusters are then concatenated to obtain a labeled prototype vector matrix. .
5. The method for estimating the number of industrial defect categories based on prototype vector correlation according to claim 1, characterized in that, The use of the correlation threshold For the unlabeled prototype vector matrix Perform a loop to remove duplicates, resulting in a set of selected vectors. include: Retain the unlabeled prototype vector matrix The correlation between prototype vectors is lower than the correlation threshold. The prototype vector, excluding the unlabeled prototype vector matrix. The correlation between prototype vectors is higher than or equal to the correlation threshold. The prototype vectors are processed until the stopping condition is met, thus obtaining the set of selected vectors. .
6. The method for estimating the number of industrial defect categories based on prototype vector correlation according to claim 5, characterized in that, The method further includes: In the first iteration, from the unlabeled prototype vector matrix Randomly select a prototype vector as the initial seed, and add the initial seed to the set of selected vectors. In the process, the remaining unselected prototype vectors are added to the prototype vector set. Calculate the set of prototype vectors Each prototype vector Maximum correlation with the initial seed; Find the prototype vector with the highest correlation. If the maximum relevance of the prototype vector is less than the relevance threshold Then the prototype vector with the lowest maximum correlation will be selected. Add to the set of selected vectors In, and from the set of prototype vectors Remove from the middle; Starting from the second iteration, the prototype vector set is calculated. Each prototype vector and the set of selected vectors The maximum correlation of each prototype vector is used to progressively update the selected vector set. until the maximum correlation is greater than or equal to the correlation threshold. or prototype vector set If the middle element is empty, the iteration stops.
7. The method for estimating the number of industrial defect categories based on prototype vector correlation according to claim 6, characterized in that, The expression for the maximum correlation is: ； in, Represents the set of prototype vectors The prototype vector in This represents the set of selected vectors.
8. The method for estimating the number of industrial defect categories based on prototype vector correlation according to claim 6, characterized in that, The selected vector set The number of prototype vectors in the unlabeled image set is determined as follows. Defect category quantity estimate Subsequently, the method further includes: The weight matrix of the defect classifier Not included in the set of selected vectors The prototype vectors in the model are pruned to obtain the weight matrix of the defect classifier. Using the weight matrix Continue with the subsequent training process; in, Indicates the maximum number of categories. Represents a set of labeled images The number of defect categories, This represents an estimated number of defect categories. For feature dimensions.
9. A device for estimating the number of industrial defect categories based on prototype vector correlation, characterized in that, The industrial defect category quantity estimation device based on prototype vector correlation includes: a processor and a memory for storing processor-executable instructions; The processor is configured to execute the industrial defect category quantity estimation method based on prototype vector correlation as described in any one of claims 1 to 8.
10. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions that are executed by one or more processors to perform the industrial defect category quantity estimation method based on prototype vector correlation as described in any one of claims 1 to 8.