An industrial product defect automatic classification method and system

Abstract

The application provides an industrial product defect automatic classification method and system, the method comprises the following steps: collecting original industrial product defect image data, constructing labeled image sample set and unlabeled image sample set; based on the labeled image sample set and the unlabeled image sample set, combining multiple image synthesis strategies, constructing a training image sample set; based on the training image sample set, introducing a transfer learning strategy and fusing an attention mechanism, constructing and optimizing an industrial product defect classification model; based on the training image sample set and the real-time small batch image sample set, performing semi-supervised joint training and online learning; based on the real-time image stream sample set, the classification model parameters and the corresponding classification prediction function, constructing an industrial product defect recognition log. The application is based on multi-strategy image enhancement and semi-supervised training, fuses transfer learning and channel attention mechanism, realizes the expansion of industrial product defect image samples and the identification of subtle defects, and is suitable for intelligent defect detection systems in various industrial manufacturing fields.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent manufacturing and industrial automation control, and particularly relates to an industrial product defect automatic classification method and system. BACKGROUND

[0002] With the continuous development of manufacturing processes, various industrial products have higher requirements for quality control and defect detection in the production process. Especially in the processing and manufacturing links of high-precision industrial products (such as metal components, electronic devices, ceramic sheets, glass substrates, printed circuit boards, lithium battery pole pieces, etc.), there are many types of common defects and complex forms, and some defects are as small as nanometers, which can easily affect the functional performance of the product and the stability of the overall manufacturing process. Therefore, how to design an efficient and robust deep learning model for the diversity and complexity of industrial product defects has become a research hotspot and application difficulty in current industrial intelligent detection technology.

[0003] Chinese patent application with publication number CN118735885A provides an industrial product surface defect detection method based on DeepLab-MSC, which includes: collecting the surface image of the industrial product to be detected; inputting the image into a preset detection model to obtain the surface defects of the industrial product to be detected, wherein the detection model is obtained based on a training set, the training set includes a steel surface defect data set and an industrial gear defect data set, and the detection model is constructed through a DeepLab-MSC network.

[0004] However, the prior art still faces many challenges. In the process of high-precision manufacturing, the common defect types of industrial products show diversity and complexity, and the defect size is usually as small as nanometers, which can easily adversely affect the structural integrity, functional stability and overall manufacturing yield of the product. Therefore, it is necessary to accurately identify and classify defects at an early stage of production. However, in actual production line defect detection images, such industrial product defects often show complex visual features such as sparse distribution, blurred edges, irregular shapes, and low contrast, making it difficult for traditional rule-based image recognition methods to accurately extract discriminative features and effectively classify them, resulting in misidentification or missed identification. Especially in the mass production stage of industrial products, the production line will automatically collect tens of thousands of defect detection images every day, which requires a large amount of labeled data to support deep learning model training. However, the current labeling process mainly relies on manual labeling, which is not only inefficient and costly, but also extremely unbalanced in terms of labeled sample categories. Some key defect types, such as bridging, metal residue, and local structure shedding, account for a small percentage of the total samples, much lower than ordinary defect samples. If traditional pure supervised learning methods are used directly, the model may overfit to the main class samples, ignoring the learning and discrimination ability for rare defects, and thus causing misjudgment, missed judgment and other risks. If the rare but high-risk rare defects are not identified in time at the early stage of the production line, it is likely that the quality defects will spread in batches after the entire batch of industrial products is offline, ultimately causing batch quality defects. Once such defects are discovered in subsequent processes or delivery environments, it will inevitably lead to large-scale process backtracking, rework repair, or even batch rejection, causing serious economic losses and affecting subsequent delivery plans and supply chain stability. SUMMARY

[0005] To overcome the above-mentioned defects of the prior art, the present application provides an industrial product defect automatic classification method and system. The method first combines image standardization and size alignment strategies to unify the size and distribution characteristics of production line collected images, ensuring the consistency of subsequent model input.

[0006] Secondly, in view of the problem of rare key defect categories such as bridging and metal residue, a multi-source fusion training image sample set is constructed. The training image sample set fuses three types of data sources: one is the labeled image; two is the defect synthetic image sample set generated by using the CutMix image mixing enhancement algorithm and conditional generative adversarial network cGAN; three is the pseudo-label sample set generated based on the unlabeled image sample set. The training data of the model in the rare category dimension is effectively expanded, and the underfitting robustness of the model is improved.

[0007] Further, the migration learning strategy and the channel attention mechanism are introduced to enhance the recognition ability of the network to local significant features such as metal residual shadow and bridging contour in complex visual scenes such as edge blur and low contrast, thereby improving the discrimination accuracy of the classification model.

[0008] In the model training mechanism, the semi-supervised joint training and online learning mechanism are combined. On the one hand, the pseudo-label data generated based on the unlabeled images is used for model training to construct a semi-supervised joint loss function, thereby assisting the model training with unlabeled data; on the other hand, the online learning mechanism is introduced to continuously optimize and update the model parameters with the real-time small batch image sample set collected at fixed intervals, output the trained and optimized industrial product defect classification model parameters and the corresponding classification prediction function, realize the rapid adaptation of the model to the distribution change of the production line, and reduce the manual labeling cost.

[0009] Finally, in order to guarantee the traceability and automatic closed-loop management ability of the system in the actual production line deployment, an industrial product defect identification log module is constructed to record the defect number, detection time, defect type and confidence of the industrial product defect, and a closed-loop feedback system with self-learning ability is constructed to ensure that the key defects are discovered in time in the early process stage, and the risk of industrial product defect diffusion is reduced.

[0010] To achieve the above object, the present application provides the following technical scheme:

[0011] An industrial product defect automatic classification method comprises:

[0012] Collecting original industrial product defect image data , constructing labeled image sample set and unlabeled image sample set ;

[0013] Based on the labeled image sample set and the unlabeled image sample set , a variety of image synthesis strategies are combined to construct a training image sample set ;

[0014] Based on the training image sample set , the migration learning strategy is introduced and the attention mechanism is fused to construct and optimize the industrial product defect classification model;

[0015] Based on the training image sample set and the real-time small batch image sample set , the semi-supervised joint training and online learning are carried out;

[0016] Based on the real-time image stream sample set , the classification model parameters and the corresponding classification prediction function , and build an industrial product defect identification log.

[0017] Further, the labeled image sample set and the unlabeled image sample set are constructed by the following method.

[0018] Each original industrial product defect image is standardized to obtain a standardized industrial product defect image .

[0019] The standardized industrial product defect image is resized to obtain an industrial product defect image with a uniform resolution .

[0020] The industrial product defect image with a uniform resolution is divided into sample sets to construct a labeled image sample set and an unlabeled image sample set , wherein the labeled image sample set contains labeled images with valid defect category labels, and the unlabeled image sample set contains unlabeled images without label information.

[0021] Further, the method for obtaining an industrial product defect image with a uniform resolution includes: when the size of the standardized industrial product defect image is greater than a preset target size, performing size alignment processing by using a proportional scaling combined with center cropping, and when the size of the standardized industrial product defect image is less than the preset target size, expanding the image to the target size by using edge symmetric padding.

[0022] Further, the construction method of the training image sample set includes:

[0023] Based on the labeled image sample set , a rare category image sample set is screened.

[0024] Based on the rare category image sample set , a mixed enhanced image sample set is constructed by using a CutMix image mixing enhancement algorithm.

[0025] Based on the rare category image sample set and the mixed enhanced image sample set , a defect synthetic image sample set is generated by using a conditional generative adversarial network.;

[0026] Based on unlabeled image sample set Generate pseudo-label sample set ;

[0027] By fusing multiple source image sample sets, a training image sample set can be constructed. ;

[0028] The multi-source image sample set includes: a set of labeled image samples. Defect-synthesized image sample set and pseudo-label sample set .

[0029] Furthermore, the screening of rare category image sample sets The methods include:

[0030] Traverse the labeled image sample set Statistical analysis of each defect category Corresponding number of image samples ;

[0031] Based on each defect category Number of image samples Calculate the average number of samples across all defect categories. ;

[0032] Average number of samples across all defect categories With adjustable hyperparameters Multiply by the product to calculate the rareness threshold. ;

[0033] Based on the rareness determination threshold If the number of image samples is lower than the rareness threshold, The defect categories are considered as rare categories, and a set of rare categories is constructed. ;

[0034] From the labeled image sample set The defect category labels in the selection process belong to the set of rare categories. Construct a rare category image sample set from the samples. .

[0035] Furthermore, the generation of pseudo-label sample sets The methods include:

[0036] Based on the already trained deep learning model For unlabeled image sample sets Each unlabeled image in Perform inference to obtain the corresponding category prediction probability distribution. ;

[0037] Predict the probability distribution based on the category of each unlabeled image. Extract the category with the highest predicted probability as the pseudo-label for the unlabeled image. ;

[0038] The confidence level of the pseudo-label is based on the maximum predicted probability value of the unlabeled image. ;

[0039] Set the confidence threshold for pseudo-labels When the confidence level of the pseudo-label of the unlabeled image Greater than or equal to the pseudo-label confidence threshold At that time, the unlabeled images and their pseudo-labels are used to construct a pseudo-label sample set. Otherwise, discard the unlabeled image.

[0040] Furthermore, the steps for constructing and optimizing the industrial product defect classification model include:

[0041] training image sample set The input is fed into a pre-trained image classification network for transfer learning initialization to obtain intermediate feature tensors. ;

[0042] Based on intermediate feature tensor With channel attention mechanism SE, weighted feature representations are generated. ;

[0043] Based on weighted feature representation Compared with the defined overall classification loss The industrial product defect classification model is trained and optimized, and the parameters of the optimized classification model are output. and the corresponding classification prediction function .

[0044] Furthermore, the steps of the semi-supervised joint training and online learning include:

[0045] Based on overall classification loss Pseudo-supervised loss for pseudo-labeled images Construct a semi-supervised joint loss function ;

[0046] Based on real-time mini-batch image sample sets An online incremental learning mechanism is constructed to output the parameters of the trained and optimized industrial product defect classification model. and the corresponding classification prediction function ;

[0047] The real-time small-batch image sample set Automatically collect a batch of latest image data in the production line every T hours.

[0048] Further, the method for constructing the industrial product defect identification log comprises:

[0049] Based on the real-time image stream sample set , the classification model parameters obtained by combining training optimization , and the corresponding classification prediction function , a defect category prediction label and its corresponding defect category prediction confidence are generated.

[0050] The real-time image stream sample set is a real-time image stream collected from an industrial product production line, serving as an input data set for defect category prediction.

[0051] Based on the defect category prediction confidence and the review rules, artificial review and closed-loop feedback collection are constructed.

[0052] Based on the real-time image stream sample , the defect category prediction label , and the defect category prediction confidence , an industrial product defect identification log is constructed.

[0053] An industrial product defect automatic classification system for implementing the above-mentioned industrial product defect automatic classification method, comprising a data collection module, a training image sample set construction module, a construction and optimization of industrial product defect classification model module, a semi-supervised joint training and online learning module, and an industrial product defect identification log construction module.

[0054] The data collection module is used to collect raw industrial product defect image data , construct labeled image sample sets and unlabeled image sample sets .

[0055] The training image sample set construction module is based on the labeled image sample set and the unlabeled image sample set , combined with multiple image synthesis strategies, to construct a training image sample set .

[0056] The construction and optimization of industrial product defect classification model module is based on the training image sample set , introduces a transfer learning strategy and fuses an attention mechanism to construct and optimize an industrial product defect classification model.

[0057] The semi-supervised joint training and online learning module: based on the training image sample set With real-time small batch image sample set Semi-supervised joint training and online learning is carried out.

[0058] The industrial product defect identification log construction module: based on the real-time image stream sample set , classification model parameters And the corresponding classification prediction function , construct the industrial product defect identification log.

[0059] Compared with the prior art, the beneficial effects of the present application are:

[0060] The industrial product defect automatic classification method and system of the present application first carries out pretreatment on the original defect image collected in the industrial product detection production line, aiming to construct an image sample set with unified format and complete information, and provides data basis for efficient training of subsequent deep learning model and real-time defect identification. Specifically, the image is subjected to gray or color standardization processing to eliminate the illumination and contrast difference caused by different batch collection environments and equipment states; in combination with the production line running speed and model calculation performance requirements, the image is adjusted to a preset fixed resolution by using the method of equal proportion scaling combined with center cropping or edge padding, which not only ensures the integrity of the defect microstructure features, but also meets the standardization of the model input size, avoids feature information loss or distortion; and according to whether the defect category information labeled by the quality inspection personnel is available, the image sample is divided into labeled image sample set and unlabeled image sample set, which relieves the data challenge brought by insufficient labeled samples and defect type diversification.

[0061] Aiming at the problem of extremely uneven sample category distribution in industrial product defect detection, the system sets a threshold to extract low-frequency defect categories such as pinholes, scratches, and particle contamination by statistically analyzing the labeled industrial product defect image data, and constructs a rare category image sample set. Based on the rare category image sample set, the CutMix image mixing enhancement technology is used to randomly fuse images of different types of defects (such as pinholes and scratches or particle contamination), generating a mixed enhanced image sample set containing multiple defects superimposed, effectively enriching the diversity of defect morphology and alleviating the problem of monotonous single defect sample morphology and insufficient quantity. For low-frequency defects that are difficult to collect and have high labor cost for manual labeling, a high-quality defect synthetic image sample set is synthesized based on a conditional generative adversarial network. The defect synthetic image sample set is highly realistic in detail and texture and defect features, and can cover complex defect morphologies that occur in actual production, enhancing the recognition ability of the model for rare defects. In addition, for unlabeled industrial product surface inspection images, a high-confidence pseudo-label sample set is automatically generated using the trained defect detection model, which is included in the training samples to reduce the workload of manual labeling and make up for the labeling blind area, and to improve the comprehensiveness of the training data. Finally, the labeled image sample set, the defect synthetic image sample set, and the pseudo-label sample set are fused to construct a training image sample set that covers all types of defects and is balanced in samples. The purpose is to improve the detection accuracy and recall rate of the model for key rare defects such as pinholes and scratches, reduce the risk of missed detection on the production line, effectively reduce the rework and scrap rate caused by defects not being discovered in time, and thus ensure the quality stability and production efficiency of the production process.

[0062] Further, based on the ResNet50 network pre-trained on a large-scale image dataset such as ImageNet, the industrial product defect classification task is quickly adapted through a transfer learning strategy to solve the problem of insufficient training data caused by the limited number of low-frequency defect samples such as pinholes and scratches, and to improve the training efficiency and stability of the model under small sample conditions. Then, the channel attention mechanism SE is introduced to automatically enhance the key feature channels related to pinholes, scratches and other small and low-contrast defects, while suppressing irrelevant background and noise information, thereby improving the model's ability to perceive complex defect details. Finally, the model is supervised and trained end-to-end using the overall classification loss to achieve accurate classification of multiple categories of industrial product defects. This process significantly improves the detection sensitivity and discrimination ability of the model for key defects, effectively reduces the missed detection rate, and reduces the rework, scrap and quality risks caused by defects not being discovered in time, ensuring quality control and production efficiency in the industrial product manufacturing process.

[0063] Further, based on the training image sample set and the real-time small batch image sample set, a semi-supervised joint training mechanism is adopted, an overall classification loss and a pseudo-supervised loss of the pseudo-label image are combined to construct a semi-supervised joint loss function, the collaborative optimization of multi-source data is realized, and the recognition ability of the model for rare defect categories such as scratches, film peeling, and particle pollution is improved by effectively utilizing a large amount of unlabeled production line data; at the same time, through periodic automatic acquisition of the real-time small batch image sample set of the production line, an online incremental learning mechanism is constructed, and the model parameters are continuously updated by using the stochastic gradient descent algorithm, so that the model can timely respond to the dynamic evolution of the defect style and the change of the data distribution in the industrial product manufacturing environment, avoid performance degradation caused by environment drift, ensure that the missed detection and false detection rates in the defect detection process are significantly reduced, reduce the rework, scrap and quality loss caused by the new defect not being captured in time, and further reduce the cost of manual repeated labeling and intervention.

[0064] Finally, based on the real-time image stream sample set collected by the industrial product production line, the classification model parameters obtained by training and optimization, and the corresponding classification prediction function, the defect category prediction label and the corresponding defect category prediction confidence are generated; and based on the defect category prediction confidence and the review rule, the samples below the threshold or suspected to be wrong are sent to the quality inspection personnel for review, and after the label is corrected, they are included in the feedback data set, integrated with the training samples regularly, and the model is continuously updated. At the same time, based on the real-time image stream sample, the defect category prediction label and the defect category prediction confidence, a structured industrial product defect recognition log is generated, the defect number, detection time, defect type and confidence are accurately recorded, the information is traceable and easy to manage, the timely and accurate identification of difficult defects is realized, the rework, scrap and production line stagnation caused by missed detection and misjudgment are avoided, the defect processing response is accelerated, the manual labeling pressure is reduced, and the stable operation and quality control of the industrial product production line are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, a brief introduction will be given below to the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating any inventive labor.

[0066] Figure 1 is a principle flowchart of an industrial product defect automatic classification method of the present application;

[0067] Figure 2 is a method flowchart for constructing the labeled image sample set and the unlabeled image sample set in the industrial product defect automatic classification method of the present application;

[0068] Figure 3is a method flow chart for constructing a training image sample set in an industrial product defect automatic classification method of the present application;

[0069] Figure 4 is a method flow chart for constructing and optimizing an industrial product defect classification model in an industrial product defect automatic classification method of the present application;

[0070] Figure 5 is a method flow chart for semi-supervised joint training and online learning in an industrial product defect automatic classification method of the present application;

[0071] Figure 6 is a method flow chart for constructing an industrial product defect identification log in an industrial product defect automatic classification method of the present application;

[0072] Figure 7 is an example diagram of an industrial product defect identification log in an industrial product defect automatic classification method of the present application;

[0073] Figure 8 is a functional module diagram of an industrial product defect automatic classification system of the present application. DETAILED DESCRIPTION

[0074] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0075] Embodiment 1

[0076] Please refer to Figure 1 The present embodiment provides an industrial product defect automatic classification method, comprising:

[0077] Step S1000, collecting original industrial product defect image data , constructing a labeled image sample set and an unlabeled image sample set .

[0078] Specifically, this step aims to collect original industrial product defect image data and complete preprocessing operations to realize the uniformity of image input format and the optimization of subsequent model training and inference effect. The original industrial product defect image data is obtained by industrial detection equipment and constitutes an original industrial product defect image set , denoted as:

[0079]

[0080] wherein, represents an index table variable, representing the th sample, wherein . represents the total number of samples, each sample consisting of . represents the th original industrial product defect image, usually a two-dimensional gray-scale image or a color image, representing the original image data input. represents the th original industrial product defect image corresponding to the defect category label, if the image has been manually or automatically labeled. represents one of the different types of industrial product defect types; if the image is not labeled, it is recorded as an undefined state.

[0081] Further, as shown in Figure 2 , step S1000 includes:

[0082] Step S1100, based on each original industrial product defect image , performing standardization processing to obtain a standardized industrial product defect image .

[0083] Specifically, this step aims to standardize each original industrial product defect image to eliminate the inconsistency of gray-scale or color value distribution between images, and improve the numerical stability and convergence speed in the model training process. The standardization processing includes pixel normalization operation on image pixel values, converting each original industrial product defect image to a standardized industrial product defect image , and the specific process formula is as follows:

[0084]

[0085] wherein, represents the mean of all image pixels; represents the standard deviation of all image pixels.

[0086] Step S1200, based on the standardized industrial product defect image , performing size adjustment processing to obtain an industrial product defect image with uniform resolution .

[0087] Specifically, this step aims to standardize the industrial product defect image The size alignment operation is performed to generate image data of uniform input size, guarantee the data dimension consistency of the neural network model in the training and inference stages, and thus improve the calculation efficiency and feature alignment accuracy of the model in the batch processing.

[0088] In the implementation process, to ensure the adaptability of the input image size to the neural network model structure, the standardized industrial product defect image obtained in step S1100 is adjusted to a preset fixed size in this step. Thus, the industrial product defect image of uniform resolution is obtained. Considering that the industrial product defect image often contains defect structures as small as sub-nanometers, to avoid loss of defect information caused by scaling or cropping, this step is based on two types of size alignment strategies for the relationship between the size of the industrial product defect image and the target size. The specific process is as follows:

[0089] Case 1: If the standardized industrial product defect image is larger than the target size , the equal scaling combined with the center cropping method is used for processing, to ensure that the image structure is not distorted during scaling, and at the same time, the spatial feature information of the defect main body area is preserved as much as possible. The specific process formula is as follows:

[0090]

[0091]

[0092]

[0093] wherein, represents the scaling factor of the standardized industrial product defect image , which is used to ensure that at least one side is scaled to be not less than the target size, represents the selection of the maximum value; and respectively represent the height and width of the standardized industrial product defect image ; and respectively represent the set target image height and width. represents the standardized industrial product defect image after scaling; represents the equal scaling operation on the standardized industrial product defect image ; and respectively represent the image height and width after scaling; represents the final industrial product defect image of uniform resolution; represents the standardized industrial product defect image ; and represents the standardized industrial product defect image ​A center cropping operation is performed to crop the image of any size to the target size.

[0094] Case two: if the normalized industrial product defect image is smaller than the target size , in order to avoid structural distortion and information loss caused by image enlargement, an edge padding alignment strategy is adopted, that is, the image content is kept unchanged, and the image is expanded to the target size by padding white pixels above, below, left and right, and the white pixel value is 0 or the image mean value. The specific process formula is as follows:

[0095]

[0096] wherein, represents the edge symmetric padding operation with the image center as the reference.

[0097] For example, if a normalized industrial product defect image has a size of , and the preset target size is : the system first calculates the adaptive scaling factor according to the proportional relationship between the target size and the original size, and then scales the image to about accordingly. On this basis, the center cropping strategy is adopted to cut the image region with a size of from the middle of the image, to obtain a defect image with consistent size and complete structural information, effectively preserving the key features of the defect region. Conversely, if the input image size is , which is obviously lower than the preset target size, the original size is kept unchanged, and 28 pixels are padded above and below the image, and 18 pixels are padded left and right, a total of 46 pixels are padded to , and the padding value can be set to 0 or the image mean value, to avoid distortion risk caused by image enlargement, and to ensure the consistency of size uniformity and feature alignment.

[0098] Specifically, through the above strategy, the size standardization and alignment of the industrial product defect image can be completed without introducing significant information loss and distortion, providing a unified input basis for subsequent feature extraction and anomaly recognition of the model.

[0099] Step S1300, the industrial product defect images with uniform resolution are subjected to sample set division processing to construct labeled image sample set and unlabeled image sample set .

[0100] Specifically, the step aims to solve the practical problems of rare key defect samples and high labor cost of artificial labeling in industrial product defect detection process, and provide data basis support for subsequent semi-supervised learning strategy of small sample defects. According to whether each image sample of the industrial product defect image set of the same resolution has artificial labeling of the defect type , the preprocessed image samples are systematically divided, and the labeled image sample set and the unlabeled image sample set are constructed respectively to realize the collaborative modeling of supervised and semi-supervised algorithms.

[0101] In the specific implementation process, the industrial product defect image sample set of the same resolution and standardized size is . , wherein represents the th image of the industrial product defect image set of the same resolution, represents the defect category label corresponding to the th image of the industrial product defect image set of the same resolution. The system divides the images according to whether they are attached with artificial labeling of the defect type, and the division strategy is as follows:

[0102] For the image pair with artificial labeling label, the labeled image sample set is constructed, and the specific process formula is as follows:

[0103]

[0104] , wherein represents the labeled image sample set, which contains all image pairs in , and is mainly used for subsequent supervised learning stage to train the defect type classification model or as accuracy verification reference, represents an empty set.

[0105] For the image without label information, the unlabeled image sample set is constructed, and the specific process formula is as follows:

[0106]

[0107] represents the unlabeled image sample set, that is, the samples in which are not included in , which is used for subsequent semi-supervised learning, pseudo label production or synthetic training; represents the set difference operator, which means removing the elements belonging to another set from a set.

[0108] Exemplarily, taking a batch of industrial product defect image data as an example, it is assumed that the original data set contains 6 image samples, numbered to . Wherein the image Defect type labeling has been completed by quality inspection engineers, corresponding to labels , , Label number represents three typical defect types, i.e. the labeled image sample set ; the rest of the images to have not been labeled, corresponding to label , i.e. the unlabeled image sample set .

[0109] The final output data is a unified format image data set , i.e. .

[0110] Specifically, through the division process of this step, the system can realize the structured management of defect images without destroying the consistency of image data, and provide high-quality data input support for subsequent semi-supervised modeling strategies for small sample defect types. For the common challenge problems in the actual industrial product manufacturing scene, such as the serious imbalance of defect category distribution, the new defect that has not established the label standard, the complex and diverse structure of image details, etc. This step helps to systematically organize training resources, so that the model can more effectively identify the potential category information in the unlabeled samples, and enhance the perception ability of the model to the key feature area and the fineness of category division.

[0111] Step S2000, based on the labeled image sample set and the unlabeled image sample set , combined with multiple image synthesis strategies, a training image sample set is constructed.

[0112] Specifically, this step aims to solve the modeling problems caused by uneven sample category distribution or the number of key rare category samples in the industrial product defect image classification task, and to construct an image enhancement mechanism suitable for small sample scenarios. By identifying rare categories and combining multiple image synthesis strategies, such as CutMix image mixing enhancement, conditional generative adversarial network (Conditional GAN, cGAN) sample generation and pseudo-label generation mechanism, the training sample set is expanded, and the recognition ability and discrimination accuracy of the model to low-frequency categories are improved.

[0113] Further, as shown in Figure 3 , step S2000 includes:

[0114] Step S2100, based on the labeled image sample set ​Screening rare category image sample set .

[0115] Specifically, the step aims to solve the problem of insufficient number of samples in some categories and structural imbalance of training sample distribution in the industrial product defect classification task. Based on the labeled image sample set constructed in step S1300 statistical analysis is performed to set a rarity determination threshold, automatically identify low-frequency defect categories, and construct a rare category image sample set , which provides accurate input basis for subsequent step S2200 small sample data enhancement.

[0116] In the specific implementation process, first, the labeled image sample set constructed in step S1300 is traversed, the number of samples corresponding to each category is calculated, and the specific process formula is as follows:

[0117]

[0118] wherein, denotes the total number of the labeled image sample set ; II denotes an indicator function, which outputs 1 when the current bracket condition is true, and 0 otherwise; denotes the defect category label of the i-th sample; denotes the identifier of the i-th category; denotes the traversal calculation on all categories ; and denotes the total number of defect categories. Next, based on the number of samples corresponding to each category , the average number of samples of all categories

[0119] is calculated, which is used to measure the overall sample distribution of the current data set, and the specific process formula is as follows:

[0120]

[0121] Subsequently, the rarity determination threshold is set, which is used to identify categories with sample numbers significantly lower than the average, and the specific process formula is as follows:

[0122]

[0123] wherein, is an adjustable hyperparameter that controls the strictness of rarity determination, and is usually valued at 0.5, 0.3, etc.

[0124] ​​​​based on the rarity determination threshold constructing a rare category set , and the specific process is as follows:

[0125]

[0126] wherein all categories with a sample number lower than the threshold value are regarded as rare categories, which are used for subsequent small sample synthesis and enhancement.

[0127] Finally, all samples with labels belonging to the rare category set are filtered out from the labeled image sample set to construct a rare category image sample set , and the specific process is as follows:

[0128]

[0129] For example, taking a set of industrial defect image data as an example, if there are multiple defect categories such as surface scratches, edge cracks, and foreign matter pollution, if the “foreign matter pollution” category has only 20 images, and the average sample number of each category is 100, the rarity determination threshold is set to 20. At this time, the “foreign matter pollution” category will be identified as a rare category, and the corresponding image samples will be classified into the rare category image sample set .

[0130] Specifically, through the processing of this step, the system can dynamically identify low-frequency category samples with synthesis potential without human intervention, provide accurate and controllable data support for the subsequent enhancement module, and effectively solve the problem of structural deficiency of the training set.

[0131] Step S2200, based on the rare category image sample set , a mixed enhanced image sample set is constructed using the CutMix image mixing enhancement algorithm.

[0132] Specifically, this step aims to increase the sample number of rare categories and the recognition ability of the model for this category. The CutMix image mixing enhancement algorithm is used to synthesize and enhance the images in the rare category image sample set constructed in step S2100, to construct a mixed enhanced image sample set .

[0133] In the specific implementation process, first, any two images and their corresponding category labels are randomly selected from the rare category image sample set ..

[0134] Next, the first th mixed augmented image is constructed based on the CutMix image mixing augmentation strategy , and its corresponding mixed augmented label , the specific process formula is as follows:

[0135]

[0136]

[0137] wherein, represents the first th mixed augmented image constructed based on the CutMix image mixing augmentation strategy; represents a randomly generated binary mask matrix, which has the same size as the image , used to control the mixed area of the image; represents the inverse area of the mask matrix, i.e. the area not belonging to , used to fill to the remaining part of the image; represents the pixel-by-pixel multiplication operation; represents a mixing coefficient between 0 and 1, representing the proportion of the area in the image belonging to ; represents the proportion of the area in the image belonging to .

[0138] Finally, based on the mixed augmented image , the mixed augmented image sample set is constructed, containing samples, and the specific process formula is as follows:

[0139]

[0140] Specifically, in the actual production environment of actual industrial product defect detection, for the extremely rare and serious defect types such as bridging, residue, and column, this step uses the CutMix image mixing augmentation algorithm to generate synthetic image samples by randomly selecting rare category images for area mixing, significantly increasing the sample quantity and morphological diversity, and making up for the problem of single training data caused by insufficient samples. In the industrial product manufacturing line, this step realizes early and accurate identification of key rare defects, avoids large-scale quality problems and rework caused by defect omission, thereby ensuring production yield and electrical stability of chips, and improving overall manufacturing efficiency and product reliability.

[0141] Step S2300, based on the rare category image sample set with the mixed augmented image sample set , a defect synthetic image sample set is generated by using conditional generative adversarial network cGAN .

[0142] Specifically, for the rare but critical defect categories in the industrial product manufacturing process, such as local connection anomaly and foreign matter residue, a defect synthetic image sample set is generated by using conditional generative adversarial network technology cGAN to expand the training data and enhance the recognition ability and generalization performance of the model for this type of defect.

[0143] In the specific implementation process, first, in order to ensure the diversity and randomness of the generated image, a random noise vector is independently sampled from a multi-dimensional standard normal distribution , and the dimension of the noise vector is set to , wherein represents the random noise vector, represents the multi-dimensional standard normal distribution, which is used to ensure that the dimensions of the random noise vector are mutually independent and consistent in scale, represents "sampling from", represents a covariance unit matrix with a dimension of . The random noise vector is used as the input of the generator, and random disturbance is introduced, so that the generated image has differences and diversity in details each time.

[0144] Then, in order to control the category attribute of the generated image, a category condition is uniformly and randomly sampled from the filtered rare category set obtained in step S2100. The specific representation formula is as follows:

[0145]

[0146] wherein represents a uniform distribution, that is, the probability of each category being selected is equal.

[0147] Subsequently, the sampled random noise vector and the category condition are combined to input the conditional generator , and the output is a defect synthetic image . The specific process formula is as follows:

[0148]

[0149] wherein represents the conditional generator, represents the synthetic image output by the generator, represents a dimension of a three-dimensional real set, which meets the category conditions in visual appearance and defect characteristics , and has the diversity and richness of real images. respectively represent the height, width and channel number of the image.

[0150] Finally, based on the defect synthetic image pair and the mixed augmented image sample set constructed in step S2200 , the defect synthetic image sample set is constructed ,

[0151] Specifically, in the actual production environment of industrial product manufacturing, some key defect types such as local connection anomaly, material residue, structure damage, etc. are extremely rare, and it is very difficult and costly to collect real samples. Based on the limited real rare category samples, this step synthesizes a high-quality and diversified defect image sample set, effectively making up for the data rareness problem. In this way, not only does it alleviate the model training bottleneck caused by insufficient rare category data, but also improves the model's recognition ability and generalization performance for rare defects.

[0152] Step S2400, based on the unlabeled image sample set generate a pseudo-label sample set .

[0153] Specifically, this step is to make full use of a large number of images that have not been manually labeled to improve the coverage and generalization ability of the training samples. This step proposes a method based on pseudo-label generation, which automatically assigns high-confidence class labels to unlabeled images through existing deep models, thereby constructing a pseudo-label sample set that can be used for model training.

[0154] In the specific implementation process, first, based on the unlabeled image sample set constructed in step S1300 , the existing trained deep learning model is used to infer each unlabeled image in the unlabeled image sample set to obtain the probability distribution of the class prediction , and the specific process formula is as follows:

[0155]

[0156] Among them, represents the probability that the unlabeled image is predicted as the th class; represents the total number of classes; represents the The predicted probability vector; For a trained deep learning model.

[0157] Next, from the predicted probability vector Select the category corresponding to the maximum value as the pseudo-label. The specific process formula is as follows:

[0158]

[0159] in, Indicates the first Zhang's unlabeled image has a pseudo-label; The parameter representing the maximum value is the category number; Indicates the range of category indexes; Indicates unlabeled images Predicted as the first The probability of a class; Indicates unlabeled images The maximum predicted probability value, i.e., the confidence level; This represents the maximum value.

[0160] Furthermore, a confidence threshold for pseudo-labels is set. Only when the maximum predicted probability value Only then will As an unlabeled image Pseudo-labels, constructing a pseudo-label sample set. Otherwise, discard the image sample to avoid introducing noise. The specific process formula is as follows:

[0161]

[0162] Specifically, on actual industrial product defect detection production lines, some image samples are often unlabeled due to blurred defect shapes, unclear boundaries, or limited manual annotation resources, making them difficult to directly participate in supervised learning training. Discarding these unlabeled images would result in wasted training data and insufficient model coverage of the actual data distribution. This step introduces a pseudo-label mechanism, automatically selecting reliable unlabeled images as training supplements while maintaining high confidence, significantly expanding the effective training set size and improving the model's generalization ability to complex and blurred defect samples.

[0163] Step S2500: Fuse multi-source image sample sets to construct a training image sample set. .

[0164] Specifically, to comprehensively improve the learning ability and robustness of the industrial product defect detection model, this step utilizes a multi-source image sample set, including: an annotated image sample set. , with high-quality training samples manually accurately labeled; defect synthetic image sample set , to make up for the insufficient data scale of rare defect categories; pseudo-label sample set , for mining potential effective information in unlabeled images.

[0165] In the specific implementation process, the multi-source image sample set is fused to construct a unified training image sample set , and the specific process formula is as follows:

[0166]

[0167] Specifically, in the manufacturing process of industrial products, some rare defects such as micro-cracks and structural collapse are difficult to cover in the original sample collection due to their extremely low natural occurrence frequency, which may lead to missed detection in the actual production line. However, some high-frequency defects such as scratches, contamination, and indentation have diverse appearances, and a single image cannot cover all the characteristics. In this context, this step fuses the multi-source image sample set to make the training set have more comprehensive coverage in terms of defect categories, texture patterns, and background interference. For example, synthetic defect images generate defect appearances under different patterns or materials to improve the model's recognition ability for unseen defect styles. For example, the contamination defect in the mixed augmented sample can effectively train the model to have stable output ability under the fluctuation of actual production line image quality. The introduction of pseudo-label samples further expands the data coverage range, mines potential effective information from unlabeled images without manual annotation, and helps to solve the problem of insufficient sample size in practical applications.

[0168] Step S3000, based on the training image sample set , a transfer learning strategy is introduced and an attention mechanism is fused to construct and optimize an industrial product defect classification model.

[0169] Specifically, this step aims to construct an industrial product defect classification model with high-precision recognition ability, which is suitable for industrial vision detection tasks such as rare defect categories and small sample conditions. The industrial product defect classification model introduces a transfer learning strategy and fuses an attention mechanism on a pre-trained image classification network to effectively extract and enhance the perception of key defect features, enhance the discrimination ability in complex situations such as micro-defects, low-contrast defects, and fuzzy boundary defects, and improve the robustness and generalization ability of the model.

[0170] This step takes the training image sample set output by step S2400 For input, the training data set includes three types of data: a set of labeled image samples, a set of defect synthetic image samples, and a set of pseudo-label samples. Through the following sub-steps, the final output is the training optimized classification model parameters and the corresponding classification prediction function , which provides support for the automatic identification and classification of defect images in subsequent steps.

[0171] Further, as Figure 4 shown, step S3000 includes:

[0172] Step S3100, input the training image sample set to the pre-trained image classification network for transfer learning initialization to obtain the intermediate feature tensor .

[0173] Specifically, this step aims to improve the training efficiency and generalization performance of the industrial product defect classification model in a small sample environment and alleviate the overfitting problem in a small sample scenario. Therefore, an image classification network pre-trained on a large-scale general image dataset, such as ResNet50, is selected as the feature extraction backbone network, and its strong generalization ability in low-level and middle-level visual features is used as the basis for model transfer learning.

[0174] In the specific implementation process, the training image sample set constructed in step S2400 is input into the pre-trained image classification network, and the learned pre-trained model parameters of the pre-trained image classification network are loaded as initialization parameters. The first several layers of the pre-trained image classification network, i.e., the convolutional layers and the pooling layers, are retained to obtain the intermediate feature tensor ; at the same time, the last fully connected layer is replaced with an adaptive industrial product defect classification task. The last fully connected layer of the pre-trained network is replaced with a new classification head, and the output dimension of the classification head is consistent with the number of industrial product defect categories to realize the identification of defect categories (the classification head matching the defect category number, i.e. dimension).

[0175] Step S3200, based on the intermediate feature tensor and the channel attention mechanism SE, generate the weighted feature representation .

[0176] Specifically, this step aims to improve the model's attention to the industrial product defect area, especially the key defect features that are small or low in contrast, and further introduces the channel attention mechanism (Squeeze-and-Excitation, SE). This mechanism learns the importance of each channel adaptively, enhances the key feature channels, and suppresses the non-key channels, thereby improving the overall feature expression ability.

[0177] In the implementation process, first, based on the intermediate feature tensor obtained in step S3100 , a global average pooling operation is performed to generate a channel description vector , where represents the number of channels.

[0178] Next, the channel description vector is input into two fully connected layers in turn, corresponding to weight matrices , , respectively, where is the compression rate, and after ReLU and Sigmoid activation function processing, a channel attention weight vector is generated:

[0179]

[0180] Finally, the channel attention weight vector and the intermediate feature tensor are multiplied element by element in the channel dimension to obtain a weighted feature representation , and the specific expression formula is as follows:

[0181]

[0182] where represents the element-by-element multiplication operation in the channel dimension.

[0183] Step S3300, based on the weighted feature representation and the defined overall classification loss , the industrial product defect classification model is trained and optimized, and the trained and optimized classification model parameters and the corresponding classification prediction function are output.

[0184] Specifically, this step aims to implement an end-to-end supervised training process for the industrial product defect classification model.

[0185] In the implementation process, based on the weighted feature representation and the cross-entropy classification loss function as the model optimization target, the overall classification loss of the current model parameters is constructed. The specific process formula is as follows:

[0186]

[0187] where represents the overall classification loss of the current model parameters . Indicates the first One input image sample, This indicates its corresponding real label. Indicates the current model parameters Below, input image samples Classified as The probability is given by log, where log represents the logarithmic function.

[0188] Finally, the optimized classification model parameters are output. and the corresponding classification prediction function .

[0189] Step S4000, based on the training image sample set With real-time mini-batch image sample sets Conduct semi-supervised joint training and online learning.

[0190] Specifically, this step aims to achieve continuous adaptation and dynamic optimization of the defect classification model in real-world industrial production line scenarios. To address issues such as constantly updated image samples and incomplete sample annotations in real-world environments, this step combines existing training image sample sets... With real-time mini-batch image sample sets By employing a semi-supervised joint training mechanism and a periodic online incremental learning mechanism, the robustness and real-time response capability of the industrial product defect classification model in practical applications are continuously improved.

[0191] Furthermore, such as Figure 5 As shown, step S4000 includes:

[0192] Step S4100, based on the overall classification loss Pseudo-supervised loss for pseudo-labeled images Construct a semi-supervised joint loss function .

[0193] Specifically, this step aims to improve the learning ability of the industrial product defect identification model under limited labeled samples and effectively utilize a large amount of unlabeled production line data. To this end, a semi-supervised joint loss function that integrates labeled and pseudo-labeled samples is constructed to achieve collaborative training optimization of multi-source data.

[0194] In the specific implementation process, firstly, based on the pseudo-label sample set constructed in step S2400... Construct the current model parameters Pseudo-supervised loss of pseudo-labeled images The specific process formula is as follows:

[0195]

[0196] in, represents the current model parameters , the pseudo-supervised loss of the pseudo-label image, used to measure the confidence matching degree between the prediction result of the model on the unlabeled sample and the pseudo-label; represents the pseudo-label image sample of the input, represents its corresponding pseudo-label, represents the probability that the input pseudo-label image sample is judged as the class under the current model parameters .

[0197] Finally, based on the overall classification loss obtained in step S3300 and the pseudo-supervised loss of the pseudo-label image , a dynamic semi-supervised joint loss function is constructed, and the specific process formula is as follows:

[0198]

[0199] , wherein ' represents a weight coefficient for regulating the influence of the pseudo-label loss on the overall training, used to dynamically balance the contribution proportion of the two types of data in the training.

[0200] Specifically, in the actual industrial product manufacturing process, common defects such as scratches, peeling, foreign particle, indentation, pollution, etc. are distributed sparsely, and the number of annotations is difficult to meet the training needs of deep models. With the semi-supervised joint loss function mechanism constructed in this step, a large number of unlabeled images can be continuously introduced and iteratively optimized for pseudo-labels on the premise of providing only a small number of artificially annotated images, thereby improving the coverage ability of the model for multiple types of defects, especially in complex industrial scenarios such as defect variation, local occlusion and background interference, the model still maintains strong discrimination ability and robustness.

[0201] Step S4200, based on the real-time small batch image sample set , an online incremental learning mechanism is constructed to output the industrial product defect classification model parameters and the corresponding classification prediction function .

[0202] Specifically, this step aims to solve the problem of model performance degradation caused by factors such as dynamic changes in data distribution and evolution of defect styles in actual industrial production line environment, and designs an online optimization mechanism based on adaptive incremental learning. The mechanism realizes the continuous adaptation of the model to the changes in the production line environment through periodic data collection and model parameter update.

[0203] In the specific implementation process, first, every Periodic collection rule of hours, automatically collect a batch of the latest image data in the production line to form a small batch of image sample set of time series data flow Then, based on the current model parameters and the real-time small batch of image sample set , the model is updated in parameters using the stochastic gradient descent algorithm, and the specific process formula is as follows:

[0204]

[0205] Among them, represents the parameter update process based on the stochastic gradient descent, is the updated model parameter.

[0206] Exemplarily, after deploying the model in a certain industrial product manufacturing line, an initial model is constructed using a small amount of training data set at the beginning. As the production line continues to run, the system automatically collects the latest production line images according to the periodic collection rule of every hours, and performs self-learning optimization based on the pseudo-label mechanism. Even if new defect patterns such as "scratch texture change" and "indentation shape variation" appear later, the model can gradually adapt to them and maintain high recognition accuracy, significantly reducing the cost of manual intervention and improving the stability of the production line.

[0207] Finally, the industrial product classification model parameters after training and optimization and the corresponding classification prediction function are output.

[0208] Specifically, this step ensures that the model can continuously adapt to the actual production line input, avoiding the problem of model performance degradation due to environmental changes, defect pattern changes, and other factors.

[0209] Step S5000, based on the real-time image stream sample set , the classification model parameters and the corresponding classification prediction function , an industrial product defect recognition log is constructed.

[0210] Specifically, this step aims to realize the inference output, credibility evaluation and model self-update closed loop construction of the defect classification model in the actual industrial production environment. By introducing a dynamic feedback mechanism and a manual review process, a defect recognition closed loop feedback system with self-learning ability is constructed to improve the long-term stability and actual applicability of the model in the industrial product production line environment.

[0211] This step is based on the training and optimization of the industrial product classification model parameters and the classification prediction function and combining a real-time image stream sample set , sequentially complete model inference, result reliability evaluation and feedback collection and update, and build an end-to-end defect detection and evolution mechanism.

[0212] The real-time image stream sample set is a real-time image stream collected from an actual industrial product production line, serving as an input data set for defect category prediction. Among them, represents the input real-time image sample; ; represents the total number of real-time image samples in the batch.

[0213] Further, as shown in Figure 6 , step S5000 includes:

[0214] Step S5100, based on the real-time image stream sample set , combining the classification model parameters optimized by training and the corresponding classification prediction function , generating a defect category prediction label and its corresponding defect category prediction confidence .

[0215] Specifically, this step aims to realize the inference prediction result output and reliability quantitative evaluation of the industrial product defect classification model in the actual deployment environment, and to provide a basis for the subsequent artificial review triggering mechanism and closed-loop feedback collection.

[0216] In the specific implementation process, first, based on the classification model parameters optimized by training and the corresponding classification prediction function output by step S4200, each input real-time image sample in the real-time image stream sample set is inferred, and the class y with the maximum probability is selected as the defect category prediction label . The specific process formula is as follows:

[0217]

[0218] Subsequently, based on the above inference result, the maximum probability value corresponding to the defect category prediction label is calculated, which is defined as the defect category prediction confidence . The specific process formula is as follows:

[0219]

[0220] Among them, the defect category prediction confidence represents the a defect class prediction confidence of an image sample, for determining whether the current model's determination is sufficiently reliable. When the defect class prediction confidence is lower than a system preset confidence threshold or meets a specific rule condition, an artificial review process can be triggered as a reference basis for artificial review and closed-loop feedback collection in step S5200.

[0221] In step S5200, artificial review and closed-loop feedback collection are constructed based on the defect class prediction confidence and review rules.

[0222] Specifically, this step aims to implement artificial review operations on the defect class prediction confidence or defect image samples marked by the system as needing review, and to construct a feedback dataset for model self-learning and updating, thereby realizing a closed-loop optimization mechanism of the system in an actual industrial production environment.

[0223] In the specific implementation process, first, it is determined whether the defect class prediction confidence based on the output of step S5100 is lower than a system set threshold or whether it meets a system specified condition that the image sample needs to enter an artificial review process. If any of the above conditions is met, the image sample is sent to an artificial review module, and a professional quality inspector labels the class according to the actual defect characteristics to construct a feedback dataset.

[0224] Subsequently, the feedback dataset is merged into a current system labeled image sample set to form a new feedback image sample set, and the specific process formula is as follows:

[0225]

[0226] For example, in an actual industrial product manufacturing line, due to process parameter fluctuations, equipment state changes, or workshop environmental condition disturbances, the defect appearance may have boundary ambiguity or similar characteristics in different batches. If a suspected "scratch" type defect appears in a batch of images, the defect class prediction confidence output by the classification model during inference is 0.45, which is lower than the system set confidence threshold, and the system triggers an artificial review mechanism accordingly. The image sample is sent to an artificial review module, and a quality inspector performs secondary labeling according to the actual defect characteristics to determine that the true class is a "dirt" type defect, not a "scratch". The review labeled sample constructed thereby is included in the feedback sample set and periodically incorporated into the training sample library for subsequent iterative optimization of the model.

[0227] ​​​​​​​​Specifically, this step can realize the boundary distinction learning of common confusion class defects, continuously correct the discrimination bias of the model on similar defect categories, and prevent specific defect types from being misjudged for a long time. At the same time, by dynamically introducing artificial review feedback samples, the adaptive ability of the model in the face of continuous evolution of production line conditions is also guaranteed, effectively improving the overall recognition accuracy and stability of the system, and building a defect detection closed-loop learning mechanism for industrial application scenarios.

[0228] Step S5300, based on real-time image stream samples , defect category prediction labels and defect category prediction confidence , an industrial product defect recognition log is constructed.

[0229] Specifically, this step aims to structure the defect recognition results into an industrial product defect recognition log, realize the standardized storage, unified management, efficient display and systematic call of the classification prediction results, and improve the application value of the defect detection system in the industrial production scene.

[0230] In the specific implementation process, based on real-time image stream sample set , defect category prediction labels and defect category prediction confidence , an industrial product defect recognition log with timeliness and traceability is constructed.

[0231] The industrial product defect recognition log adopts a structured data format, including four key information elements: first, the defect number, which serves as the unique identifier for each defect information, ensures the uniqueness and traceability of each defect record, and facilitates accurate positioning and management of defects in the production process; second, the detection time, which uses the international standard ISO 8601 format (YYYY-MM-DDTHH:MM:SS) to accurately record the moment of defect discovery, providing a time reference for production process analysis and timeliness evaluation; third, the defect type, which reflects the specific defect category identified, such as edge damage, scratches, dents, surface contamination, etc., as a key indicator for quality control and problem analysis, helping to reveal the essential characteristics of product quality abnormalities; fourth, the confidence, which quantifies the reliability of defect classification with a probability value, with a higher value indicating stronger credibility of the detection result, providing data basis for priority sorting in subsequent artificial review process.

[0232] For example, Figure 7As shown, an example of an industrial product defect identification log generated in an industrial product manufacturing line is displayed. This log clearly and intuitively presents the specific circumstances of each defect, including defect number, detection time, defect type, and confidence, and other key information. For example, defect number 1 was recorded on March 15, 2025 at 13:34:01, and was determined to be "edge damage" with a confidence of 0.93; defect numbers 2 and 3 recorded "scratches" and "dents" respectively, and also labeled the corresponding detection times and confidences. This structured log recording method not only accurately locates the time of defect occurrence and the type of defect, but also quantifies the reliability of the detection results through the confidence. Further, the industrial product defect identification log is transmitted as a structured defect identification result to multiple business systems, including but not limited to: transmitted to the manufacturing execution system (MES) for recording the defect situation of the batch of products and triggering the abnormal alarm mechanism of the production line; sent to the production control system for dynamically adjusting the device parameters related to "edge damage", such as the power setting of the laser processing equipment or the processing path correction; uploaded to the visual board terminal for real-time viewing of defect statistical trends and the latest abnormal images by workshop operators; integrated into the quality analysis report system to automatically generate the "Top 3 Defects Today" column in the daily report for quality inspection departments to summarize and analyze.

[0233] Embodiment 2

[0234] This embodiment is based on embodiment 1 and provides an industrial product defect automatic classification system, as shown in Figure 8 including a data acquisition module, a training image sample set construction module, a construction and optimization of industrial product defect classification model module, a semi-supervised joint training and online learning module, and an industrial product defect identification log construction module;

[0235] The data acquisition module is used to acquire raw industrial product defect image data , construct labeled image sample set and unlabeled image sample set ;

[0236] The training image sample set construction module is based on the labeled image sample set and the unlabeled image sample set , combined with multiple image synthesis strategies, to construct the training image sample set ;

[0237] The construction and optimization of industrial product defect classification model module: based on the training image sample set , introduce transfer learning strategy and fuse attention mechanism, construct and optimize industrial product defect classification model;

[0238] The semi-supervised joint training and online learning module is based on the training image sample set with real-time small batch image sample set performing semi-supervised joint training and online learning;

[0239] The industrial product defect identification log construction module: based on the real-time image stream sample set , classification model parameters and the corresponding classification prediction function , construct the industrial product defect identification log.

[0240] The part of the above technical solutions provided in the embodiments of the present application which is consistent with the implementation principle of the corresponding technical solutions in the prior art is not described in detail to avoid excessive repetition.

[0241] The specific embodiments described above further illustrate the purposes, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An automatic classification method for defects in industrial products, characterized in that, include: Collect raw industrial product defect image data and construct labeled image sample set and unlabeled image sample set; A training image sample set is constructed based on the labeled and unlabeled image sample sets, combined with various image synthesis strategies; The method for constructing the training image sample set includes: filtering rare category image sample sets based on the labeled image sample set; constructing a hybrid enhanced image sample set using the CutMix image fusion enhancement algorithm based on the rare category image sample set; generating a defective synthetic image sample set using a conditional generative adversarial network based on the rare category image sample set and the hybrid enhanced image sample set; generating a pseudo-labeled sample set based on the unlabeled image sample set; and fusing the multi-source image sample sets to construct the training image sample set; wherein the multi-source image sample set includes: the labeled image sample set, the defective synthetic image sample set, and the pseudo-labeled sample set; Based on the training image sample set, a transfer learning strategy and an attention mechanism are introduced to construct and optimize an industrial product defect classification model. Semi-supervised joint training and online learning are performed based on the training image sample set and the real-time mini-batch image sample set. Based on real-time image stream sample sets, classification model parameters, and corresponding classification prediction functions, an industrial product defect identification log is constructed.

2. The automatic classification method for industrial product defects according to claim 1, characterized in that, The methods for constructing labeled and unlabeled image sample sets include: Standardized industrial product defect images are obtained by standardizing each original industrial product defect image. The size of the standardized industrial product defect images is adjusted to obtain industrial product defect images with uniform resolution. Industrial product defect images of uniform resolution are divided into sample sets to construct labeled image sample sets and unlabeled image sample sets. The labeled image sample set contains labeled images with valid defect category labels, and the unlabeled image sample set contains unlabeled images without label information.

3. The automatic classification method for defects in industrial products according to claim 2, characterized in that, The method for obtaining a uniform resolution industrial product defect image includes: when the size of the standardized industrial product defect image is larger than the preset target size, size alignment is performed by proportional scaling combined with center cropping; when the size of the standardized industrial product defect image is smaller than the preset target size, the image is expanded to the target size by edge symmetry filling.

4. The automatic classification method for industrial product defects according to claim 1, characterized in that, Methods for filtering image sample sets of rare categories include: Iterate through the labeled image sample set and count the number of image samples corresponding to each defect category; Calculate the average number of samples for all defect categories based on the number of image samples for each defect category. The rarity threshold is calculated by multiplying the average number of samples for all defect categories by an adjustable hyperparameter. Based on the rareness determination threshold, defect categories with fewer image samples than the rareness determination threshold are considered rare categories, and a rare category set is constructed; From the set of labeled image samples, samples whose defect category labels belong to the set of rare categories are selected to construct a set of rare category image samples.

5. The automatic classification method for defects in industrial products according to claim 1, characterized in that, Methods for generating pseudo-label sample sets include: Based on the trained deep learning model, reasoning is performed on each unlabeled image in the unlabeled image sample set to obtain the corresponding category prediction probability distribution; Based on the category prediction probability distribution of each unlabeled image, the category with the highest prediction probability is extracted as the pseudo label of the unlabeled image; The confidence level of the pseudo-label is based on the maximum predicted probability value of the unlabeled image; A pseudo-label confidence threshold is set. When the pseudo-label confidence of the unlabeled image is greater than or equal to the pseudo-label confidence threshold, the unlabeled image and its pseudo-label are constructed into a pseudo-label sample set; otherwise, the unlabeled image is discarded.

6. The automatic classification method for industrial product defects according to claim 1, characterized in that, The steps for constructing and optimizing an industrial product defect classification model include: The training image sample set is input into a pre-trained image classification network for transfer learning initialization to obtain an intermediate feature tensor. Based on the intermediate feature tensor and channel attention mechanism, a weighted feature representation is generated; Based on the weighted feature representation and the defined overall classification loss, the industrial product defect classification model is trained and optimized, and the parameters of the trained and optimized classification model and the corresponding classification prediction function are output.

7. The automatic classification method for defects in industrial products according to claim 1, characterized in that, The steps of semi-supervised joint training and online learning include: A semi-supervised joint loss function is constructed based on the overall classification loss and the pseudo-supervised loss of the pseudo-labeled image. Based on a real-time mini-batch image sample set, an online incremental learning mechanism is constructed to output the trained and optimized parameters of the industrial product defect classification model and the corresponding classification prediction function. The real-time small-batch image sample set is every... T The system automatically collects the latest image data from the production line every hour.

8. The automatic classification method for defects in industrial products according to claim 1, characterized in that, Methods for constructing industrial product defect identification logs include: Based on the real-time image stream sample set, combined with the classification model parameters obtained through training and optimization and the corresponding classification prediction function, defect category prediction labels and their corresponding defect category prediction confidence scores are generated. The real-time image stream sample set is a real-time image stream collected from an industrial production line, which serves as the input dataset for defect category prediction. Based on the confidence level of defect category prediction and review rules, a manual review and closed-loop feedback collection system is constructed. An industrial product defect identification log is constructed based on real-time image stream samples, defect category prediction labels, and defect category prediction confidence.

9. An automatic industrial product defect classification system, used to implement the automatic industrial product defect classification method according to any one of claims 1-8, characterized in that, The system includes a data acquisition module, a training image sample set construction module, an industrial product defect classification model construction and optimization module, a semi-supervised joint training and online learning module, and an industrial product defect identification log construction module. The data acquisition module is used to acquire raw industrial product defect image data and construct labeled image sample sets and unlabeled image sample sets. The training image sample set construction module constructs a training image sample set based on the labeled image sample set and the unlabeled image sample set, combined with various image synthesis strategies. The module for constructing and optimizing the industrial product defect classification model: Based on the training image sample set, a transfer learning strategy is introduced and an attention mechanism is integrated to construct and optimize the industrial product defect classification model; The semi-supervised joint training and online learning module performs semi-supervised joint training and online learning based on the training image sample set and the real-time mini-batch image sample set. The industrial product defect identification log construction module constructs an industrial product defect identification log based on a real-time image stream sample set, classification model parameters, and the corresponding classification prediction function.

Images