Industrial product defect automatic classification method and system

Abstract

The invention provides an industrial product defect automatic classification method and system, and the method comprises the steps: collecting original industrial product defect image data, and constructing a labeled image sample set and an unlabeled image sample set; constructing a training image sample set based on the labeled image sample set and the unlabeled image sample set in combination with a plurality of image synthesis strategies; based on the training image sample set, introducing a transfer learning strategy and fusing an attention mechanism, and constructing and optimizing an industrial product defect classification model; performing semi-supervised joint training and online learning based on the training image sample set and the real-time small-batch image sample set; and constructing an industrial product defect identification log based on the real-time image flow sample set, the classification model parameters and the corresponding classification prediction function. On the basis of multi-strategy image enhancement and semi-supervised training, transfer learning and a channel attention mechanism are fused, expansion of industrial product defect image samples and fine defect identification are achieved, and the method is suitable for an intelligent defect detection system in various industrial manufacturing fields.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing and industrial automation control technology, and in particular to a method and system for automatically classifying industrial product defects. Background Art

[0002] With the continuous advancement of manufacturing processes, various industrial products face higher requirements for quality control and defect detection during production. This is particularly true in the processing and manufacturing of high-precision industrial products (such as metal components, electronic devices, ceramic sheets, glass substrates, printed circuit boards, and lithium battery electrodes). Common defects are numerous and complex, with some even reaching nanometer sizes, significantly impacting product performance and the stability of the overall manufacturing process. Therefore, designing efficient and robust deep learning models to address the diversity and complexity of industrial product defects has become a research hotspot and a key challenge in the application of intelligent industrial detection technology.

[0003] A Chinese patent application with publication number CN118735885A provides a method for detecting surface defects of industrial products based on DeepLab-MSC. The method includes: acquiring a surface image of the industrial product to be inspected; inputting the image into a preset detection model to obtain the surface defects of the industrial product to be inspected, wherein the detection model is obtained based on training with a training set, the training set including a steel surface defect dataset and an industrial gear defect dataset, and the detection model is constructed through a DeepLab-MSC network.

[0004] However, existing technologies still face numerous challenges. In high-precision manufacturing processes, common industrial product defects are diverse and complex, often at nanometer scales, potentially negatively impacting a product's structural integrity, functional stability, and overall manufacturing yield. Accurate defect identification and classification are crucial in the early stages of production. However, in actual production line defect detection images, these defects often exhibit complex visual features such as sparse distribution, blurred edges, irregular shapes, and low contrast. This makes it difficult for traditional rule-based image recognition methods to accurately extract discriminative features and effectively classify them, leading to false or missed identifications. Especially during the large-scale mass production phase of industrial products, production lines automatically collect and generate tens of thousands of defect detection images daily, requiring massive amounts of labeled data to support deep learning model training. However, the current labeling process primarily relies on manual annotation, which is inefficient, costly, and leads to a significant imbalance in the types of labeled samples. Certain key defect types, such as bridging, metal residue, and localized structural detachment, are underrepresented in the overall sample size, far lower than those found in common defect samples. If traditional purely supervised learning methods are directly employed, the model is prone to overfitting the main class samples and neglecting its ability to learn and discriminate rare defects, leading to risks such as misjudgments and missed detections. Failure to promptly identify these rare but high-risk defects early in the production line can lead to widespread quality defects after the entire batch of industrial products has rolled off the production line, ultimately causing widespread quality defects. Once such defects are discovered after entering subsequent processes or the shipping environment, they inevitably trigger large-scale process backtracking, rework and repair, or even the scrapping of entire batches, resulting in severe economic losses and impacting subsequent delivery plans and supply chain stability. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method and system for automatic classification of industrial product defects. The method first combines image standardization and size alignment strategies to unify the size and distribution characteristics of images collected on the production line, ensuring the consistency of subsequent model inputs.

[0006] Secondly, to address the rarity of samples for key rare defect categories, such as bridges and metal residues, a multi-source fusion training image sample set was constructed. This training image sample set incorporates three data sources: first, labeled images; second, a set of synthetic defect image samples generated using the CutMix image hybrid enhancement algorithm and a conditional generative adversarial network (cGAN); and third, a set of pseudo-labeled samples generated from an unlabeled image sample set. This effectively expands the model's training data for rare categories and improves its robustness to underfitting.

[0007] Furthermore, a transfer learning strategy and a channel attention mechanism are introduced to enhance the network's ability to recognize local salient features such as metal afterimages and bridge contours in complex visual scenes such as blurred edges and low contrast, thereby improving the discrimination accuracy of the classification model.

[0008] In terms of model training mechanisms, a combination of semi-supervised joint training and online learning is proposed. On the one hand, pseudo-labeled data generated from unlabeled images is used in model training to construct a semi-supervised joint loss function, thereby utilizing unlabeled data to assist model training. On the other hand, an online learning mechanism is introduced to continuously optimize and update model parameters using real-time small batch image samples collected at fixed intervals. The trained and optimized industrial product defect classification model parameters and the corresponding classification prediction function are output, enabling the model to quickly adapt to changes in the distribution of production line images and reducing manual labeling costs.

[0009] Finally, to ensure the traceability and automated closed-loop management capabilities of the system in actual production line deployment, an industrial product defect identification log module is constructed to record key information such as the defect number, detection time, defect type and confidence level of industrial product defects. A closed-loop feedback system with self-learning capabilities is constructed to ensure that key defects are discovered in a timely manner in the early process stages, reducing the risk of industrial product defects spreading.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] A method for automatically classifying defects in industrial products, comprising:

[0012] Collect original industrial product defect image data , construct a labeled image sample set and unlabeled image sample sets ;

[0013] Based on the labeled image sample set and unlabeled image sample sets , combining multiple image synthesis strategies to construct a training image sample set ;

[0014] Based on the training image sample set , introduce transfer learning strategy and integrate attention mechanism to build and optimize industrial product defect classification model;

[0015] Based on the training image sample set With real-time mini-batch image sample set Perform semi-supervised joint training and online learning;

[0016] Based on real-time image stream sample set , classification model parameters And the corresponding classification prediction function , build an industrial product defect identification log.

[0017] Furthermore, the labeled image sample set and unlabeled image sample sets The construction methods include:

[0018] Based on each original industrial product defect image Perform standardization processing to obtain standardized industrial product defect images ;

[0019] Based on standardized industrial product defect images Perform size adjustment to obtain uniform resolution images of industrial product defects ;

[0020] For industrial product defect images with uniform resolution Perform sample set division processing to construct annotated image sample set and unlabeled image sample sets , where the labeled image sample set Contains annotated images with valid defect category labels, the unlabeled image sample set Unlabeled images contain unlabeled information.

[0021] Furthermore, the obtained industrial product defect image with uniform resolution The method includes: when the standardized industrial product defect image When the size is larger than the preset target size, the size alignment is performed by proportional scaling combined with center cropping. When the size is smaller than the preset target size, the image is expanded to the target size using edge symmetric filling.

[0022] Furthermore, the training image sample set The construction methods include:

[0023] Based on the labeled image sample set Filter rare category image sample sets ;

[0024] Based on rare category image sample sets , using CutMix image hybrid enhancement algorithm to construct a hybrid enhanced image sample set ;

[0025] Based on rare category image sample sets and mixed enhanced image sample set , using conditional generative adversarial networks to generate defect synthetic image sample sets ;

[0026] Based on unlabeled image sample set Generate pseudo-label sample set ;

[0027] Fuse multi-source image sample sets to construct training image sample sets ;

[0028] The multi-source image sample set includes: a labeled image sample set , defect synthetic image sample set and pseudo-label sample sets .

[0029] Furthermore, the screening of rare category image sample sets The methods include:

[0030] Traverse the labeled image sample set , statistics for each defect category The corresponding number of image samples ;

[0031] Based on each defect category The number of image samples , calculate the average number of samples for all defect categories ;

[0032] The average number of samples for all defect categories With adjustable hyperparameters Multiply and calculate the rareness threshold ;

[0033] Based on the rarity threshold , the number of image samples is lower than the rareness threshold The defect categories are considered as rare categories and the rare category set is constructed ;

[0034] From the labeled image sample set The defect category labels are selected to belong to the rare category set samples, building a rare category image sample set .

[0035] Furthermore, the pseudo-label sample set is generated The methods include:

[0036] Based on a trained deep learning model , for the unlabeled image sample set Each unlabeled image in Perform inference to obtain the corresponding category prediction probability distribution ;

[0037] Class prediction probability distribution based on each unlabeled image , extract the category with the maximum predicted probability as the pseudo label of the unlabeled image ;

[0038] The confidence of the pseudo label is based on the maximum predicted probability value of the unlabeled image ;

[0039] Setting pseudo-label confidence threshold , when the pseudo-label confidence of the unlabeled image Greater than or equal to the pseudo label confidence threshold When the unlabeled image and its pseudo label are constructed as a pseudo label sample set , otherwise the unlabeled image is discarded.

[0040] Furthermore, the steps of constructing and optimizing the industrial product defect classification model include:

[0041] The training image sample set Input into the pre-trained image classification network, perform transfer learning initialization, and obtain the intermediate feature tensor ;

[0042] Based on the intermediate feature tensor And channel attention mechanism SE, generate weighted feature representation ;

[0043] Based on weighted feature representation With the overall classification loss defined , train and optimize the industrial product defect classification model, and output the classification model parameters after training and optimization And the corresponding classification prediction function .

[0044] Furthermore, the steps of the semi-supervised joint training and online learning include:

[0045] Based on the overall classification loss and pseudo-supervised loss for pseudo-labeled images , construct a semi-supervised joint loss function ;

[0046] Based on real-time small batch image sample set , build an online incremental learning mechanism and output the parameters of the industrial product defect classification model after training and optimization And the corresponding classification prediction function ;

[0047] The real-time mini-batch image sample set Automatically collect the latest batch of image data from the production line every T hours.

[0048] Furthermore, the method for constructing the industrial product defect identification log includes:

[0049] Based on real-time image stream sample set , combined with the classification model parameters obtained by training optimization And the corresponding classification prediction function , generate defect category prediction labels and its corresponding defect category prediction confidence ;

[0050] The real-time image stream sample set It is a real-time image stream collected from an industrial production line, which serves as the input dataset for defect category prediction;

[0051] Prediction confidence based on defect category and review rules, building manual review and closed-loop feedback collection;

[0052] Based on real-time image streaming samples , defect category prediction label and defect category prediction confidence , build an industrial product defect identification log.

[0053] An automatic industrial product defect classification system, which is used to implement the above-mentioned automatic industrial product defect classification method, includes a data acquisition module, a training image sample set construction module, a module for constructing and optimizing an industrial product defect classification model, a semi-supervised joint training and online learning module, and an industrial product defect recognition log construction module;

[0054] The data acquisition module is used to collect original industrial product defect image data , construct a labeled image sample set and unlabeled image sample sets ;

[0055] The training image sample set construction module: based on the labeled image sample set and unlabeled image sample sets , combining multiple image synthesis strategies to construct a training image sample set ;

[0056] The construction and optimization of the industrial product defect classification model module: based on the training image sample set , introduce transfer learning strategy and integrate attention mechanism to build and optimize the industrial product defect classification model.

[0057] The semi-supervised joint training and online learning module: based on the training image sample set With real-time mini-batch image sample set Perform semi-supervised joint training and online learning;

[0058] The industrial product defect recognition log construction module: based on real-time image stream sample set , classification model parameters And the corresponding classification prediction function , build an industrial product defect identification log.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] The present invention provides an automatic classification method and system for industrial product defects. First, the original defect images collected in the industrial product inspection production line are pre-processed, aiming to construct an image sample set with a unified format and complete information, providing a data basis for the efficient training of subsequent deep learning models and real-time defect recognition. Specifically, the image is grayscale or color standardized to eliminate the differences in illumination and contrast caused by different batch acquisition environments and equipment states; combined with the production line operation speed and model calculation performance requirements, the image is adjusted to a preset fixed resolution using a method of proportional scaling combined with center cropping or edge filling, which not only ensures the integrity of the tiny structural features of the defect, but also meets the standardization of the model input size, avoiding feature information loss or distortion; and based on whether there is defect category information marked by quality inspectors, the image samples are divided into a labeled image sample set and an unlabeled image sample set, alleviating the data challenges brought about by insufficient labeled samples and the diversification of defect types.

[0061] To address the extremely uneven distribution of sample categories in industrial product defect detection, the system collects statistics from labeled industrial product defect image data, sets thresholds to extract low-frequency defect categories such as pinholes, scratches, and particle contamination, and constructs a sample set of rare category images. Based on this rare category image sample set, the system uses CutMix image hybrid enhancement technology to randomly fuse images of different defect types (such as pinholes, scratches, or particle contamination) to generate a hybrid enhanced image sample set containing multiple superimposed defects. This effectively enriches the diversity of defect morphology and alleviates the problem of monotonous morphology and insufficient number of single defect samples. For low-frequency defects that are difficult to collect and require high manual labeling costs, a high-quality defect synthetic image sample set is synthesized based on a conditional generative adversarial network. This defect synthetic image sample set is highly realistic in detail texture and defect features, can cover the complex defect morphologies encountered in actual production, and enhance the model's recognition ability for rare defects. In addition, for unlabeled industrial product surface inspection images, a trained defect detection model is used to automatically generate a high-confidence pseudo-label sample set and incorporate it into the training samples, reducing the workload of manual labeling, filling in labeling blind spots, and improving the comprehensiveness of the training data. Finally, the labeled image sample set, the defect-synthesized image sample set, and the pseudo-label sample set are fused to construct a balanced training image sample set covering all defect types. This improves the model's detection accuracy and recall for key rare defects like pinholes and scratches, reducing the risk of missed detections on the production line and effectively lowering the rework and scrap rate of industrial products caused by undetected defects, thereby ensuring stable quality and production efficiency during the generation process.

[0062] Furthermore, based on the ResNet50 network pre-trained on large-scale image datasets such as ImageNet, a transfer learning strategy is used to quickly adapt to the task of industrial product defect classification, solving the problem of insufficient training data due to the limited number of low-frequency defect samples such as pinholes and scratches, and improving the model's training efficiency and stability under small sample conditions. Subsequently, a channel attention mechanism (SE) is introduced to automatically enhance key feature channels related to tiny and low-contrast defects such as pinholes and scratches, while suppressing irrelevant background and noise information, thereby improving the model's ability to perceive complex defect details. Finally, the model is trained end-to-end using the overall classification loss to achieve accurate classification of multiple categories of industrial product defects. This process addresses the problem of the difficulty in obtaining samples of tiny and rare defects commonly found on production lines, significantly improving the model's detection sensitivity and discrimination ability for key defects, effectively reducing the missed detection rate, and reducing rework, scrap, and quality risks caused by undetected defects, thereby ensuring quality control and production efficiency in the industrial product manufacturing process.

[0063] In addition, based on the training image sample set and the real-time small-batch image sample set, a semi-supervised joint training mechanism is adopted. By combining the overall classification loss and the pseudo-supervised loss of the pseudo-labeled image, a semi-supervised joint loss function is constructed to achieve collaborative optimization of multi-source data. A large amount of unlabeled production line data is effectively utilized to improve the model's recognition ability for rare defects such as scratches, delamination, particle contamination and other rare defect categories; at the same time, by periodically and automatically collecting real-time small-batch image sample sets of the production line, an online incremental learning mechanism is constructed, and the model parameters are continuously updated using the stochastic gradient descent algorithm, so that the model can respond in a timely manner to the dynamic evolution of defect patterns and data distribution changes in the industrial product manufacturing environment, avoid the performance degradation of the model due to environmental drift, ensure that the missed detection and false detection rates in the defect detection process are significantly reduced, reduce the rework, scrap and quality loss caused by the failure to capture new defects in a timely manner, and further reduce the cost of manual repeated labeling and intervention.

[0064] Finally, based on the real-time image stream sample set collected from the industrial product production line, combined with the classification model parameters obtained through training optimization and the corresponding classification prediction function, defect category prediction labels and their corresponding defect category prediction confidence are generated. Based on the defect category prediction confidence and review rules, samples below the threshold or suspected errors are sent to quality inspectors for review. After the labels are corrected, they are included in the feedback data set and regularly integrated with the training samples to achieve continuous incremental updates of the model. At the same time, a structured industrial product defect identification log is generated based on real-time image stream samples, defect category prediction labels, and defect category prediction confidence. The log accurately records the defect number, detection time, defect type, and confidence, ensuring that the information is traceable and easy to manage, enabling timely and accurate identification of difficult defects, avoiding rework, scrapping, and production line stagnation caused by missed detection and misjudgment, accelerating defect processing responses, reducing the pressure of manual labeling, and ensuring stable operation and quality control of industrial product production lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 This is a principle flow chart of an automatic classification method for industrial product defects of the present invention;

[0067] Figure 2 This is a flow chart of a method for constructing a labeled image sample set and an unlabeled image sample set in an automatic classification method for industrial product defects of the present invention;

[0068] Figure 3This is a flow chart of a method for constructing a training image sample set in an automatic classification method for industrial product defects of the present invention;

[0069] Figure 4 This is a flow chart of a method for constructing and optimizing an industrial product defect classification model in an automatic industrial product defect classification method of the present invention;

[0070] Figure 5 This is a flow chart of a method for performing semi-supervised joint training and online learning in an automatic classification method for industrial product defects of the present invention;

[0071] Figure 6 This is a flow chart of a method for constructing an industrial product defect identification log in an automatic industrial product defect classification method of the present invention;

[0072] Figure 7 This is an example diagram of an industrial product defect identification log of an automatic industrial product defect classification method of the present invention;

[0073] Figure 8 This is a functional module diagram of an automatic classification system for industrial product defects of the present invention. DETAILED DESCRIPTION

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0075] Example 1

[0076] See also Figure 1 As shown, this embodiment provides a method for automatically classifying defects in industrial products, including:

[0077] Step S1000: Collecting original industrial product defect image data , construct a labeled image sample set and unlabeled image sample sets .

[0078] Specifically, this step aims to collect original industrial product defect image data and complete preprocessing operations to achieve the unification of image input format and the optimization of subsequent model training and reasoning effects. The original industrial product defect image data is collected by industrial inspection equipment and constitutes the original industrial product defect image set. , recorded as:

[0079]

[0080] in, Indicates the index table variable, indicating the samples, of which . Represents the total number of samples, each sample is represented by composition. Indicates the An original industrial product defect image, usually a two-dimensional grayscale image or color image, represents the original image data input. Indicates the The defect category label corresponding to the original industrial product defect image. If the image has been manually or automatically labeled, then express One of the different types of industrial product defects; if the image is not labeled, then , recorded as undefined state.

[0081] Furthermore, if Figure 2 As shown, step S1000 includes:

[0082] Step S1100: Based on each original industrial product defect image Perform standardization processing to obtain standardized industrial product defect images .

[0083] Specifically, this step aims to identify each original industrial product defect image. Perform standardization to eliminate the inconsistency of grayscale or color value distribution between images and improve the numerical stability and convergence speed during model training. The standardization includes pixel normalization of image pixel values, which converts each original industrial product defect image into a pixel normalization. Converted into standardized industrial product defect images , the specific process formula is as follows:

[0084]

[0085] in, Represents the mean value of all image pixels; Represents the standard deviation of all image pixels.

[0086] Step S1200: Based on the standardized industrial product defect image Perform size adjustment to obtain uniform resolution images of industrial product defects .

[0087] Specifically, this step aims to standardize the defect images of industrial products. Perform size alignment to generate image data of uniform input size, ensuring data dimension consistency during the training and inference phases of the neural network model, thereby improving the model's computational efficiency and feature alignment accuracy during batch processing.

[0088] In the specific implementation process, in order to ensure the adaptability of the input image size and the neural network model structure, this step converts the standardized industrial product defect image obtained in step S1100 into Resize to a preset fixed size , thereby obtaining uniform resolution industrial product defect images Considering that industrial product defect images often contain defect structures as small as sub-nanometer, in order to avoid loss of defect information due to scaling or cropping, this step is based on two types of size alignment strategies based on the relationship between the size of the industrial product defect image and the target size. The specific process is as follows:

[0089] Case 1: If the standardized industrial product defect image Size larger than target size When processing, the method of proportional scaling combined with center cropping is used to ensure that the image structure is scaled without distortion while retaining the spatial feature information of the defect main area as much as possible. The specific process formula is as follows:

[0090]

[0091]

[0092]

[0093] in, Indicates the A standardized image of industrial product defects The scaling factor is used to ensure that at least one side is scaled to no less than the target size. Indicates selecting the maximum value; and Represents standardized industrial product defect images height and width; The target image height and width are set respectively; Represents a standardized industrial product defect image After scaling images; Represents a standardized industrial product defect image Perform proportional scaling operations; and Respectively represent the height and width of the scaled image; Represents the final uniform resolution industrial product defect image; Express Performs a center cropping operation to crop the center of an image of any size to the target size.

[0094] Case 2: If the standardized industrial product defect image Size is smaller than target size In order to avoid structural distortion and information loss caused by image magnification, an edge filling alignment strategy is adopted, that is, the image content is kept unchanged and expanded to the target size by filling blank pixels above, below, left and right. The blank pixel value is, for example, 0 or the image mean. The specific process formula is as follows:

[0095]

[0096] in, Indicates an edge-symmetric filling operation based on the center of the image.

[0097] For example, if a standardized industrial product defect image Size , the preset target size is When: The system first calculates the adaptive scaling factor based on the proportional relationship between the target size and the original size , and accordingly scale the image to approximately On this basis, the center cropping strategy is adopted to cut out the image with a size of The image area is obtained with the same size and complete structural information, effectively retaining the key features of the defect area. On the contrary, if the input image size is , which is significantly lower than the preset target size, the original size is kept unchanged and the image is filled with 28 pixels above and below and 18 pixels on the left and right to make up to ,The padding value can be set to 0 or the image mean to avoid the risk of distortion caused by image magnification and to ensure the consistency of size uniformity and feature alignment.

[0098] Specifically, through the above strategy, the standardized alignment of industrial product defect image sizes can be completed without introducing significant information loss and distortion, providing a unified input basis for feature extraction and anomaly recognition of subsequent models.

[0099] Step S1300: uniform resolution industrial product defect images Perform sample set division processing to construct annotated image sample set and unlabeled image sample sets .

[0100] Specifically, this step aims to address practical issues such as the scarcity of key defect samples and the high cost of manual labeling in the process of industrial product defect detection, and provide data support for the subsequent semi-supervised learning strategy for small sample defects. Whether there is manually labeled defect type annotation information, the preprocessed image samples are systematically divided, and labeled image sample sets and unlabeled image sample sets are constructed respectively to achieve collaborative modeling of supervised and semi-supervised algorithms.

[0101] In the specific implementation process, a standardized and uniform-sized industrial product defect image sample set is set. for ,in Indicates the uniform resolution industrial product defect images, Indicates the The defect category labels corresponding to the defect images of industrial products with uniform resolution. The system classifies the images based on whether they have manually annotated defect type labels. The classification strategy is as follows:

[0102] For image pairs with manual labeling , construct a labeled image sample set , the specific process formula is as follows:

[0103]

[0104] in, Represents a set of labeled image samples, including all Image pair , mainly used in the subsequent supervised learning stage, to train defect type classification models or as a reference for accuracy verification, Represents the empty set.

[0105] For images without label information, construct an unlabeled image sample set , the specific process formula is as follows:

[0106]

[0107] represents the unlabeled image sample set, i.e. Not included in The samples in are used for subsequent semi-supervised learning, pseudo-label production or synthetic training; Represents the set difference operator, which removes elements from one set that belong to another set.

[0108] For example, taking a batch of industrial product defect image data as an example, assuming that the original data set contains 6 image samples, numbered to . Among them, the image to The defect type has been marked by the quality inspection engineer, corresponding to the label 、 、 , label number Representing three typical defect types, that is, constructing a labeled image sample set ; The rest of the images to No label has been marked yet, corresponding label , that is, construct an unlabeled image sample set .

[0109] The final output data is an image dataset in a unified format ,Right now .

[0110] Specifically, through the division processing in this step, the system can achieve structured management of defect images without destroying the consistency of image data, and provide high-quality data input support for subsequent semi-supervised modeling strategies for small sample defect types. To address common challenging issues in actual industrial product manufacturing scenarios, such as severe imbalance in the distribution of defect categories, lack of established labeling standards for newly emerging defects, and complex and diverse image detail structures, this step helps to systematically organize training resources, enabling the model to more effectively identify potential category information in unlabeled samples, enhancing the model's perception of key feature areas and the sophistication of category division.

[0111] Step S2000: Based on the labeled image sample set and unlabeled image sample sets , combining multiple image synthesis strategies to construct a training image sample set .

[0112] Specifically, this step aims to address modeling challenges in industrial product defect image classification, often caused by uneven sample distribution or a limited number of samples in key, rare categories. This allows for the development of an image enhancement mechanism suitable for small sample sizes. By identifying rare categories and combining various image synthesis strategies, such as CutMix image blending enhancement, conditional generative adversarial network (cGAN) synthetic sample generation, and pseudo-label generation, the training sample set is expanded, improving the model's ability to identify and discriminate low-frequency categories.

[0113] Furthermore, if Figure 3 As shown, step S2000 includes:

[0114] Step S2100: Based on the labeled image sample set Filter rare category image sample sets .

[0115] Specifically, this step aims to solve the problem of insufficient number of samples in some categories and imbalanced distribution of training samples in the task of industrial product defect classification. Perform statistical analysis, set rareness thresholds, automatically identify low-frequency defect categories, and construct rare category image sample sets , providing accurate input basis for the small sample data enhancement in the subsequent step S2200.

[0116] In the specific implementation process, first traverse the labeled image sample set constructed in step S1300 , calculate each category The corresponding sample size , the specific process formula is as follows:

[0117]

[0118] in, Represents a set of labeled image samples Total quantity; Ⅱ Represents an indicator function, which outputs 1 if the current bracket condition is true, otherwise 0; Indicates the Defect category labels for samples; Indicates the Identifier of the class category; For all categories Perform traversal calculation; Indicates the total number of defect categories.

[0119] Next, based on each category The corresponding sample size , calculate the average number of samples for all categories , which is used to measure the overall sample distribution of the current data set. The specific process formula is as follows:

[0120]

[0121] Then, set the rarity threshold , which is used to identify categories with significantly lower sample sizes than the mean. The specific process formula is as follows:

[0122]

[0123] in, It is an adjustable hyperparameter that controls the strictness of the rarity determination, and usually takes values ​​such as 0.5, 0.3, etc.

[0124] Based on the rarity threshold Constructing rare category sets , the specific process formula is as follows:

[0125]

[0126] Among them, the number of all samples is below the threshold The categories are considered as rare categories and used for subsequent small sample synthesis and enhancement.

[0127] Finally, all labels belong to the rare category set The samples are from the labeled image sample set Filter out and build a rare category image sample set , the specific process formula is as follows:

[0128]

[0129] For example, take a set of industrial product defect image datasets as an example. If there are multiple types of defects, such as surface scratches, edge cracks, foreign matter contamination, etc., if the "foreign matter contamination" category has only 20 images, and the average number of samples in each category is For 100 sheets, set , then the rarity threshold At this point, the "foreign body contamination" category will be identified as a rare category, and the corresponding image sample will be classified into the rare category image sample set .

[0130] Specifically, through the processing of this step, the system can dynamically identify low-frequency category samples with synthesis potential without human intervention, provide accurate and controllable data support for subsequent enhancement modules, and effectively fill the structural deficiencies of the training set.

[0131] Step S2200: Based on a rare category image sample set , using CutMix image hybrid enhancement algorithm to construct a hybrid enhanced image sample set .

[0132] Specifically, this step aims to increase the number of samples of rare categories and the recognition ability of the model for this category. The CutMix image hybrid enhancement algorithm is used to enhance the rare category image sample set constructed in step S2100. The images in the image are synthetically enhanced to construct a mixed enhanced image sample set .

[0133] In the specific implementation process, we first select the rare category image sample set Randomly select any two images from , and its corresponding category label .

[0134] Then, based on the CutMix image hybrid enhancement strategy, the Hybrid enhanced image , and its corresponding hybrid enhancement label , the specific process formula is as follows:

[0135]

[0136]

[0137] in, Indicates the first image enhancement strategy based on CutMix. A hybrid enhanced image; Represents a randomly generated binary mask matrix with the same size as the image Same, used to control the blending area of ​​the image; Represents the inverse area of ​​the mask matrix, that is, it does not belong to area, used to Fill the remaining part of the image; Represents pixel-by-pixel multiplication operation; Represents a mixing coefficient between 0 and 1, indicating the the proportion of the region; Indicates the image belongs to proportion of the region.

[0138] Finally, based on the hybrid enhanced image pair , construct hybrid enhanced image sample set , the number of samples included is , the specific process formula is as follows:

[0139]

[0140] Specifically, in the actual production environment of industrial product defect detection, this step uses the CutMix image mixing enhancement algorithm to target extremely rare defect types such as bridges, residues, and levons that have a serious impact on product quality. By randomly selecting rare category images for regional mixing, it generates synthetic image samples that incorporate diverse features, significantly increasing the number of samples and morphological diversity, and compensating for the problem of single training data due to insufficient samples. On industrial product manufacturing production lines, this step achieves early and accurate identification of critical rare defects, avoiding large-scale quality problems and rework and repair caused by missed defects, thereby ensuring production yield and the electrical stability of the chip, and improving overall manufacturing efficiency and product reliability.

[0141] Step S2300: Based on a rare category image sample set and mixed enhanced image sample set , using the conditional generative adversarial network cGAN to generate defect synthetic image sample sets .

[0142] Specifically, this step aims to target rare but critical defect categories in the manufacturing process of industrial products, such as local connection anomalies and foreign matter residues, and use the conditional generative adversarial network technology cGAN to generate defect synthetic image sample sets. , in order to expand the training data and enhance the model's recognition ability and generalization performance for this type of defects.

[0143] In the specific implementation process, first, in order to ensure the diversity and randomness of the generated image, a random noise vector is obtained by independent sampling from a multidimensional standard normal distribution. , set the dimension of the noise vector to ,but ,in, represents a random noise vector, Represents a multidimensional standard normal distribution, which is used to ensure that the dimensions of the random noise vector are independent of each other and have consistent scales. It means "sampled from"; Indicates the dimension The random noise vector is used as the input of the generator to introduce random perturbations, so that each generated image has differences and diversity in details.

[0144] Next, in order to control the category attributes of the generated image, the rare category set obtained in step S2100 is filtered In the example, uniform random sampling is used to obtain a category condition The specific expression formula is as follows:

[0145]

[0146] in, Represents a uniform distribution, that is, each category has an equal probability of being selected.

[0147] Then, the sampled random noise vector With category conditions Combined Input Condition Generator , output defect composite image , the specific process formula is as follows:

[0148]

[0149] in, Represents a condition generator, represents the synthetic image output by the generator, Represents a dimension The three-dimensional real number set that meets the classification conditions in terms of visual appearance and defect characteristics , and has the diversity and detail richness of real images. Represent the height, width and number of channels of the image respectively.

[0150] Finally, based on the defect synthesis image pair And the hybrid enhanced image sample set constructed in step S2200 , construct defect synthetic image sample set , The number of samples of defect synthesized images output by the generator.

[0151] Specifically, in the real-world production environment of industrial products, certain key defect types, such as localized connection anomalies, material residue, and structural damage, are extremely rare, making collecting real samples difficult and costly. This step synthesizes a high-quality, diverse set of defect image samples based on a limited number of real-world samples of rare categories, effectively addressing the data scarcity issue. This approach not only alleviates the model training bottleneck caused by insufficient data for rare categories, but also improves the model's ability to identify rare defects and its generalization performance.

[0152] Step S2400: Based on the unlabeled image sample set Generate pseudo-label sample set .

[0153] Specifically, this step is to make full use of a large number of image resources that have not been manually annotated and improve the coverage and generalization ability of training samples. This step proposes a method based on pseudo-label generation, which automatically assigns high-confidence category labels to unlabeled images through existing deep models, thereby constructing a pseudo-label sample set that can be used for model training. .

[0154] In the specific implementation process, first, based on the unlabeled image sample set constructed in step S1300 , using the trained deep learning model to analyze the unlabeled image sample set Each unlabeled image Perform inference to obtain the probability distribution of category predictions , the specific process formula is as follows:

[0155]

[0156] in, Represents an unlabeled image Predicted to be the class probability; Indicates the total number of categories; Represents an unlabeled image The predicted probability vector of ; For the trained deep learning model.

[0157] Next, from the predicted probability vector Select the category corresponding to the maximum value as the pseudo label , the specific process formula is as follows:

[0158]

[0159] in, Indicates the Pseudo labels for unlabeled images; Indicates the parameter corresponding to the maximum value, that is, the category number; Indicates the category index range; Represents an unlabeled image Predicted to be class probability; Represents an unlabeled image The maximum predicted probability value, that is, the confidence level; Indicates the maximum value.

[0160] Furthermore, set the pseudo label confidence threshold , only when the maximum predicted probability value When As an unlabeled image Pseudo labels, construct pseudo label sample sets Otherwise, the image sample is discarded to avoid introducing noise. The specific process formula is as follows:

[0161]

[0162] Specifically, in actual industrial product defect detection production lines, some image samples are often in an "unlabeled" state due to fuzzy defect morphology, unclear boundaries, or limited manual labeling resources, making them difficult to directly participate in supervised learning training. If these unlabeled images are discarded, training data will be wasted and the model will not adequately cover the actual data distribution. This step introduces a pseudo-labeling mechanism to automatically select reliable unlabeled images as training supplements while ensuring high confidence, significantly expanding the size of the effective training set and improving the model's generalization ability for complex and ambiguous defect samples.

[0163] Step S2500: Fusing multiple source image sample sets to construct a training image sample set .

[0164] Specifically, in order to comprehensively improve the learning ability and robustness of the industrial product defect detection model, this step comprehensively utilizes multi-source image sample sets, including: labeled image sample sets , with high-quality training samples accurately annotated by humans; defect synthetic image sample set , to make up for the lack of data size of rare defect categories; pseudo-label sample set , used to mine potential effective information in unlabeled images.

[0165] In the specific implementation process, the multi-source image sample sets are fused to construct a unified training image sample set. , the specific process formula is as follows:

[0166]

[0167] Specifically, in the manufacturing process of industrial products, some rare defects, such as microcracks and structural failures, often occur at very low frequencies and are difficult to capture in the original sample collection, leading to model misses in actual production lines. Other high-frequency defects, such as scratches, contamination, and indentations, have diverse appearances, making it difficult for a single image to capture all these variations. In this context, this step fuses multi-source image sample sets to provide a more comprehensive training set covering defect categories, texture morphology, background interference, and other dimensions. For example, by generating defect appearances with different morphologies or material surfaces, the model's recognition capabilities are enhanced even when the defect is not yet seen. For another example, by simulating the interference of contamination defects in hybrid enhanced samples under varying lighting or surface conditions, the model is effectively trained to maintain stable output despite fluctuations in image quality on actual production lines. The introduction of pseudo-labeled samples further expands data coverage, enabling the extraction of potentially useful information from unlabeled images without the need for manual annotation, helping to address the issue of insufficient sample size in practical applications.

[0168] Step S3000: Based on the training image sample set , introduce transfer learning strategy and integrate attention mechanism to build and optimize the industrial product defect classification model.

[0169] Specifically, this step aims to build a high-precision industrial defect classification model suitable for industrial visual inspection tasks involving rare defect categories and small sample sizes. This model effectively extracts and enhances the perception of key defect features by incorporating a transfer learning strategy and an attention mechanism into a pre-trained image classification network. This enhances the ability to distinguish complex situations such as tiny defects, low-contrast defects, and defects with blurred boundaries, improving the model's robustness and generalization capabilities.

[0170] This step uses the training image sample set output in step S2400 As input, the training dataset includes three types of data: annotated image sample set, defect synthetic image sample set and pseudo label sample set. After the following sub-steps, the training optimized classification model parameters are finally output. And the corresponding classification prediction function , providing support for the automatic recognition and classification of defect images in subsequent steps.

[0171] Furthermore, if Figure 4 As shown, step S3000 includes:

[0172] Step S3100: training image sample set Input into the pre-trained image classification network, perform transfer learning initialization, and obtain the intermediate feature tensor .

[0173] Specifically, this step aims to improve the training efficiency and generalization performance of the industrial product defect classification model in small sample environments, alleviating the overfitting problem in these scenarios. To this end, an image classification network pre-trained on a large-scale general image dataset, such as ResNet50, is selected as the backbone network for feature extraction. Its strong generalization capabilities for low-level and mid-level visual features serve as the basis for model transfer learning.

[0174] In the specific implementation process, the training image sample set constructed in step S2400 is Input into the pre-trained image classification network and load its learned pre-trained model parameters As initialization parameters, retain the first several layers of the pre-trained image classification network, namely the convolution layer and the pooling layer, to obtain the intermediate feature tensor At the same time, replace the last fully connected layer with the adapted industrial product defect classification task. Replace the last fully connected layer of the pre-trained network with a new classification head. The output dimension of the classification head is consistent with the number of industrial product defect categories to achieve the recognition of defect categories (the classification head with the matching number of defect categories, i.e. 'dimension).

[0175] Step S3200, based on the intermediate feature tensor And channel attention mechanism SE, generate weighted feature representation .

[0176] Specifically, this step aims to improve the model's ability to focus on defective areas of industrial products, especially critical defects with small or low contrast features. This further introduces a channel attention mechanism (Squeeze-and-Excitation, SE). This mechanism adaptively learns the importance of each channel, enhancing key feature channels and suppressing non-critical channels, thereby improving overall feature expression capabilities.

[0177] In the specific implementation process, first, based on the intermediate feature tensor obtained in step S3100 , perform global average pooling operation to generate each channel description vector ,in, 'Indicates the number of channels.

[0178] Next, the channel description vector Input into two fully connected layers in sequence, corresponding to the weight matrix 、 ,in, is the compression rate, and after processing with ReLU and Sigmoid activation functions, a channel attention weight vector is generated :

[0179]

[0180] Finally, the channel attention weight vector and the intermediate feature tensor Perform element-by-element multiplication according to the channel dimension to obtain the weighted feature representation , the specific expression formula is as follows:

[0181]

[0182] in, Represents an element-wise multiplication operation in the channel dimension.

[0183] Step S3300: Based on the weighted feature representation With the overall classification loss defined , train and optimize the industrial product defect classification model, and output the classification model parameters after training and optimization And the corresponding classification prediction function .

[0184] Specifically, this step aims to implement the end-to-end supervised training process of the industrial product defect classification model.

[0185] In the specific implementation process, based on the weighted feature representation The classification loss function of the cross entropy is used as the model optimization target to construct the current model parameters The overall classification loss The specific process formula is as follows:

[0186]

[0187] in, Indicates the current model parameters The overall classification loss of Indicates the input image samples, represents the corresponding true label, Indicates the current model parameters Next, input image sample Determined to be a category The probability of , log represents the logarithmic function.

[0188] Finally, the classification model parameters after training and optimization are output And the corresponding classification prediction function .

[0189] Step S4000: Based on the training image sample set With real-time mini-batch image sample set Perform semi-supervised joint training and online learning.

[0190] Specifically, this step aims to achieve the continuous adaptation and dynamic optimization capabilities of the defect classification model in actual industrial production line scenarios. In order to deal with the problems of continuous updating of image samples and incomplete sample annotation in actual environments, the existing training image sample set is combined with the existing training image sample set. With real-time mini-batch image sample set Through the semi-supervised joint training mechanism and the periodic online incremental learning mechanism, the robustness and real-time response capability of the industrial product defect classification model in practical applications are continuously improved.

[0191] Furthermore, if Figure 5 As shown, step S4000 includes:

[0192] Step S4100, based on the overall classification loss and pseudo-supervised loss for pseudo-labeled images , construct a semi-supervised joint loss function .

[0193] Specifically, this step aims to improve the learning ability of industrial product defect recognition models with limited labeled samples and effectively utilize large amounts of unlabeled production line data. To this end, a semi-supervised joint loss function is constructed that integrates labeled and pseudo-labeled samples to achieve collaborative training optimization of multi-source data.

[0194] In the specific implementation process, first, based on the pseudo-label sample set constructed in step S2400 , construct the current model parameters Pseudo-supervised loss for pseudo-labeled images , the specific process formula is as follows:

[0195]

[0196] in, Indicates the current model parameters The pseudo-supervision loss of the pseudo-label image is used to measure the confidence match between the model's prediction results for unlabeled samples and the pseudo-label; Indicates the input pseudo-label image samples, represents its corresponding pseudo label, Indicates the current model parameters Below, the input pseudo-label image sample Determined to be a category probability.

[0197] Finally, based on the overall classification loss obtained in step S3300 Pseudo-supervised loss with pseudo-labeled images , construct a dynamic semi-supervised joint loss function , the specific process formula is as follows:

[0198]

[0199] in, ' represents the weight coefficient that regulates the impact of pseudo-label loss on the overall training, which is used to dynamically balance the contribution ratio of the two types of data in training.

[0200] Specifically, common defects in the actual industrial manufacturing process include scratches, peeling, foreign particles, indentations, and contamination. Some defects are sparsely distributed, and the number of annotations required to train deep models is insufficient. The semi-supervised joint loss function mechanism constructed in this step allows the continuous introduction of a large number of unlabeled images and iterative pseudo-label optimization, while only providing a small number of manually annotated images. This improves the model's coverage of multiple defect types, maintaining strong discrimination and robustness, especially in complex industrial scenarios such as defect variation, partial occlusion, and background interference.

[0201] Step S4200: Based on the real-time small batch image sample set , build an online incremental learning mechanism and output the parameters of the industrial product defect classification model after training and optimization And the corresponding classification prediction function .

[0202] Specifically, this step aims to address the problem of model performance degradation caused by factors such as dynamic changes in data distribution and evolving defect patterns in real industrial production line environments. By designing an online optimization mechanism based on adaptive incremental learning, this mechanism enables the model to continuously adapt to changes in the production line environment through periodic data collection and model parameter updates.

[0203] In the specific implementation process, first, every The hourly periodic collection rule automatically collects the latest batch of image data from the production line to form a small batch of image sample sets in the time series data stream. Then, based on the current model parameters and real-time mini-batch image sample sets , the stochastic gradient descent algorithm is used to update the model parameters. The specific process formula is as follows:

[0204]

[0205] in, represents the parameter update process based on stochastic gradient descent, are the updated model parameters.

[0206] For example, after deploying this model in a certain industrial product manufacturing production line, a small amount of training data set is initially used. As the production line continues to run, the system The system automatically collects the latest production line images based on a periodic collection schedule of 12 hours, and then performs self-learning optimization based on a pseudo-labeling mechanism. Even if new defect types such as "scratch texture changes" or "indentation morphology variations" emerge, the model can gradually adapt and maintain high recognition accuracy, significantly reducing manual intervention costs and improving production line stability.

[0207] Finally, the parameters of the industrial product classification model after training and optimization are output And the corresponding classification prediction function .

[0208] Specifically, this step ensures that the model can continuously and adaptively adjust according to the actual production line input, avoiding model performance degradation caused by factors such as environmental changes and changes in defect morphology.

[0209] Step S5000: Based on the real-time image stream sample set , classification model parameters And the corresponding classification prediction function , build an industrial product defect identification log.

[0210] Specifically, this step aims to realize the reasoning output, credibility assessment and model self-update closed-loop construction of the defect classification model in the actual industrial production environment. By introducing a dynamic feedback mechanism and manual review process, a defect identification closed-loop feedback system with self-learning capabilities is constructed to improve the long-term stability and practical applicability of the model in the industrial production line environment.

[0211] This step is based on the training optimized industrial product classification model parameters output in step S4200 and classification prediction function , and combined with the real-time image stream sample set of the production line , complete model reasoning, result credibility assessment and feedback collection and update in sequence, and build an end-to-end defect detection and evolution mechanism.

[0212] The real-time image stream sample set It is a real-time image stream collected from an actual industrial production line, which serves as the input data set for defect category prediction. Indicates the Input real-time image samples; ; Indicates the total number of real-time image samples in the batch.

[0213] Furthermore, if Figure 6 As shown, step S5000 includes:

[0214] Step S5100: Based on the real-time image stream sample set , combined with the classification model parameters obtained by training optimization And the corresponding classification prediction function , generate defect category prediction labels and its corresponding defect category prediction confidence .

[0215] Specifically, this step aims to achieve the output of inference prediction results and quantitative credibility evaluation of the industrial product defect classification model in the actual deployment environment, providing a judgment basis for the subsequent manual review trigger mechanism and closed-loop feedback collection.

[0216] In the specific implementation process, firstly, based on step S4200, the classification model parameters after training optimization are output. And the corresponding classification prediction function , for real-time image stream sample set Real-time image samples for each input Perform inference and select the category y with the highest probability as the defect category prediction label , the specific process formula is as follows:

[0217]

[0218] Subsequently, this step calculates the defect category prediction label based on the above reasoning results The corresponding maximum probability value is defined as the defect category prediction confidence , the specific process formula is as follows:

[0219]

[0220] Among them, the defect category prediction confidence Indicates the The confidence level of defect category prediction of image samples is used to determine whether the current model is Whether the judgment is sufficiently reliable. When the confidence threshold is lower than the system preset or specific rule conditions are met, the manual review process can be triggered, which serves as a reference for the manual review and closed-loop feedback collection in step S5200.

[0221] Step S5200: predict confidence based on defect category and review rules, build manual review and closed-loop feedback collection.

[0222] Specifically, this step aims to perform manual review operations on the defect category prediction confidence or defect image samples marked by the system as requiring review, and to construct a feedback dataset for model self-learning and updating, so as to realize the closed-loop optimization mechanism of the system in the actual industrial production environment.

[0223] In the specific implementation process, first, the defect category prediction confidence level based on the output of step S5100 is determined. Is it lower than the threshold set by the system, or does it meet the system's requirement that the image sample enter the manual review process. If any of the above conditions are met, The system is sent to the manual review module, where professional quality inspectors label the defects according to their actual characteristics and construct a feedback dataset. .

[0224] Then, the feedback dataset Summarize the image sample set that has been labeled in the current system , forming a new feedback image sample set , the specific process formula is as follows:

[0225]

[0226] For example, in actual industrial product manufacturing production lines, due to fluctuations in process parameters, changes in equipment status, or disturbances in workshop environmental conditions, the appearance of defects in different batches may have blurred boundaries or similar features. If a suspected "scratch" defect appears in a batch of images, the defect category prediction confidence output by the classification model during inference is 0.45, which is lower than the confidence threshold set by the system. The system triggers the manual review mechanism accordingly. The image samples are sent to the manual review module, and the quality inspectors perform secondary annotations based on the actual defect features to determine that their true category is a "dirt" defect rather than a "scratch". The review and annotation samples thus constructed are included in the feedback sample set, and are subsequently periodically incorporated into the training sample library for iterative optimization of the model.

[0227] Specifically, this step enables learning to distinguish the boundaries of commonly confused defects, continuously correcting the model's bias in distinguishing similar defect categories, and preventing the long-term misjudgment of specific defect types. Furthermore, the dynamic introduction of manual review feedback samples ensures the model's adaptability to the evolving production line conditions, effectively improving the system's overall recognition accuracy and stability, and establishing a closed-loop learning mechanism for defect detection tailored to industrial applications.

[0228] Step S5300: Based on the real-time image stream sample , defect category prediction label and defect category prediction confidence , build an industrial product defect identification log.

[0229] Specifically, this step aims to structure the defect identification results into industrial product defect identification logs, achieve standardized storage, unified management, efficient display and systematic call of classification prediction results, and enhance the application value of the defect detection system in industrial production scenarios.

[0230] In the specific implementation process, based on the real-time image stream sample set , defect category prediction label and defect category prediction confidence , build an industrial product defect identification log with timeliness and traceability.

[0231] The industrial product defect identification log adopts a structured data format and includes four key information elements: one is the defect number, which serves as a unique identifier for each defect information, ensuring the uniqueness and traceability of each defect record, and facilitating the precise positioning and management of defects during the production process; the second is the detection time, which adopts the international standard ISO 8601 format (YYYY-MM-DDTHH:MM:SS) to accurately record the moment when the defect is discovered, providing a time benchmark for production process analysis and timeliness assessment; the third is the defect type, which is used to reflect the specific defect category identified, such as edge damage, scratches, pits, surface contamination, etc., as a key indicator for quality control and problem analysis, it helps to reveal the essential characteristics of product quality abnormalities; the fourth is the confidence level, which quantifies the reliability of defect classification with a probability value. The higher the value, the more credible the detection result, providing a data basis for the priority sorting of the subsequent manual review process.

[0232] For example, Figure 7The figure shows an example of an industrial product defect identification log generated in an industrial product manufacturing production line. The log clearly and intuitively presents the specific circumstances of each defect, including key information such as the defect number, detection time, defect type, and confidence level. For example, defect number 1 was recorded at 13:34:01 on March 15, 2025, and was judged to be "edge damage" with a confidence level of 0.93; defect numbers 2 and 3 recorded defect types such as "nicks" and "pits", respectively, and the corresponding detection time and confidence levels were simultaneously annotated. This structured log recording method can not only accurately locate the time and type of defect occurrence, but also quantify the reliability of the detection results through confidence levels. Furthermore, the industrial product defect identification log is synchronously transmitted to multiple business systems as a structured defect identification result, including but not limited to: transmitted to the manufacturing execution system MES, used to record the defects of the batch of products and trigger the production line abnormality alarm mechanism; sent to the production control system, used to dynamically adjust the equipment parameters related to "edge damage", such as the power setting or processing path correction of laser processing equipment; uploaded to the visual dashboard terminal, so that workshop operators can view the defect statistical trends and the latest abnormal images in real time; imported into the quality analysis report system, and automatically generated the "Top 3 Defects of the Day" column in the daily report for summary and analysis by the quality inspection department.

[0233] Example 2

[0234] This embodiment provides an automatic classification system for industrial product defects based on the first embodiment. Figure 8 As shown, it includes a data acquisition module, a training image sample set construction module, a module for building and optimizing an industrial product defect classification model, a semi-supervised joint training and online learning module, and an industrial product defect recognition log construction module;

[0235] The data acquisition module is used to collect original industrial product defect image data , construct a labeled image sample set and unlabeled image sample sets ;

[0236] The training image sample set construction module: based on the labeled image sample set and unlabeled image sample sets , combining multiple image synthesis strategies to construct a training image sample set ;

[0237] The construction and optimization of the industrial product defect classification model module: based on the training image sample set , introduce transfer learning strategy and integrate attention mechanism to build and optimize industrial product defect classification model;

[0238] The semi-supervised joint training and online learning module: based on the training image sample set With real-time mini-batch image sample set Perform semi-supervised joint training and online learning;

[0239] The industrial product defect recognition log construction module: based on real-time image stream sample set , classification model parameters And the corresponding classification prediction function , build an industrial product defect identification log.

[0240] The parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0241] The above-described specific embodiments further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for automatically classifying defects in industrial products, characterized in that: include: Collect original industrial product defect image data and construct labeled image sample sets and unlabeled image sample sets; Based on the labeled image sample set and the unlabeled image sample set, a training image sample set is constructed by combining multiple image synthesis strategies; Based on the training image sample set, we introduced a transfer learning strategy and integrated the attention mechanism to build and optimize the industrial product defect classification model; Semi-supervised joint training and online learning based on training image sample sets and real-time small batch image sample sets; Based on the real-time image stream sample set, classification model parameters and corresponding classification prediction function, an industrial product defect recognition log is constructed.

2. The method for automatic classification of industrial product defects according to claim 1, characterized in that: The construction method of the labeled image sample set and the unlabeled image sample set includes: Performing standardization processing on each original industrial product defect image to obtain a standardized industrial product defect image; Perform size adjustment based on the standardized industrial product defect image to obtain an industrial product defect image with uniform resolution; The sample sets of industrial product defect images with uniform resolution are divided into labeled image sample sets and unlabeled image sample sets are constructed. The labeled image sample set contains labeled images with valid defect category labels, and the unlabeled image sample set contains unlabeled images without label information.

3. The method for automatic classification of industrial product defects according to claim 2, characterized in that: The method for obtaining an industrial product defect image with uniform resolution includes: when the size of the standardized industrial product defect image is larger than a preset target size, using proportional scaling combined with center cropping to perform size alignment processing; when the size of the standardized industrial product defect image is smaller than the preset target size, using edge symmetrical filling to expand the image to the target size.

4. The method for automatic classification of industrial product defects according to claim 1, characterized in that: The method for constructing the training image sample set includes: Filter rare category image sample sets based on the labeled image sample sets; Based on the rare category image sample set, the CutMix image hybrid enhancement algorithm is used to construct a hybrid enhanced image sample set; Based on the rare category image sample set and the mixed enhanced image sample set, a conditional generative adversarial network is used to generate a defect synthetic image sample set; Generate a pseudo-label sample set based on an unlabeled image sample set; Fuse multi-source image sample sets to construct training image sample sets; The multi-source image sample set includes: a labeled image sample set, a defect synthetic image sample set and a pseudo-label sample set.

5. The method for automatic classification of industrial product defects according to claim 4, characterized in that: Methods for filtering rare category image sample sets include: Traverse the labeled image sample set and count the number of image samples corresponding to each defect category; Based on the number of image samples of each defect category, calculate the average number of samples of all defect categories; The rarity threshold is calculated by multiplying the average number of samples of all defect categories by an adjustable hyperparameter. Based on the rarity determination threshold, the defect categories whose image sample numbers are lower than the rarity determination threshold are regarded as rare categories, and a rare category set is constructed; Samples whose defect category labels belong to the rare category set are screened from the labeled image sample set to construct a rare category image sample set.

6. The method for automatic classification of industrial product defects according to claim 4, characterized in that: Methods for generating pseudo-label sample sets include: Based on the trained deep learning model, inference is performed on each unlabeled image in the unlabeled image sample set to obtain the corresponding category prediction probability distribution; Based on the category prediction probability distribution of each unlabeled image, extracting the category with the maximum prediction probability as the pseudo label of the unlabeled image; The maximum predicted probability value of the unlabeled image is used as the confidence of the pseudo label; A pseudo label confidence threshold is set. When the pseudo label confidence of the unlabeled image is greater than or equal to the pseudo label confidence threshold, the unlabeled image and its pseudo label are constructed as a pseudo label sample set; otherwise, the unlabeled image is discarded.

7. The method for automatic classification of industrial product defects according to claim 1, characterized in that: The steps to build and optimize the industrial product defect classification model include: Input the training image sample set into the pre-trained image classification network to perform transfer learning initialization and obtain the intermediate feature tensor; Generate weighted feature representation based on intermediate feature tensors and channel attention mechanism; Based on the weighted feature representation and the defined overall classification loss, the industrial product defect classification model is trained and optimized, and the trained and optimized classification model parameters and the corresponding classification prediction function are output.

8. The method for automatic classification of industrial product defects according to claim 1, characterized in that: The steps of semi-supervised joint training and online learning include: Construct a semi-supervised joint loss function based on the overall classification loss and the pseudo-supervised loss of pseudo-labeled images; Based on a real-time small batch image sample set, an online incremental learning mechanism is constructed to output the trained and optimized industrial product defect classification model parameters and the corresponding classification prediction function; The real-time mini-batch image sample set is T Automatically collect the latest batch of image data from the production line every hour.

9. The method for automatic classification of industrial product defects according to claim 1, characterized in that: The method for constructing an industrial product defect identification log includes: Based on the real-time image stream sample set, combined with the classification model parameters obtained through training optimization and the corresponding classification prediction function, the defect category prediction label and its corresponding defect category prediction confidence are generated; The real-time image stream sample set is a real-time image stream collected from an industrial product production line, and serves as an input data set for defect category prediction; Based on the defect category prediction confidence and review rules, build manual review and closed-loop feedback collection; Based on real-time image stream samples, defect category prediction labels, and defect category prediction confidence, an industrial product defect recognition log is constructed.

10. An automatic classification system for industrial product defects, used to implement the automatic classification method for industrial product defects according to any one of claims 1 to 9, characterized in that: The system includes a data acquisition module, a training image sample set construction module, a module for constructing and optimizing an industrial product defect classification model, a semi-supervised joint training and online learning module, and a module for constructing an industrial product defect recognition log; The data acquisition module is used to collect original industrial product defect image data and construct a labeled image sample set and an unlabeled image sample set; The training image sample set construction module is used to construct a training image sample set based on the labeled image sample set and the unlabeled image sample set, in combination with multiple image synthesis strategies; The module for building and optimizing the industrial product defect classification model: based on the training image sample set, introduces a transfer learning strategy and integrates the attention mechanism to build and optimize the industrial product defect classification model; The semi-supervised joint training and online learning module performs semi-supervised joint training and online learning based on the training image sample set and the real-time small batch image sample set; The industrial product defect recognition log construction module constructs an industrial product defect recognition log based on a real-time image stream sample set, classification model parameters, and a corresponding classification prediction function.

Images