Incremental learning visual defect identification system and method for industrial quality inspection

Abstract

The invention discloses an incremental learning visual defect recognition system and method for industrial quality inspection, and particularly relates to the field of defect recognition, and the system comprises a sample library construction module, a preprocessing module, a model construction module, an initial training module, a new defect sample set construction module and a defect recognition module. According to the method, an initial visual defect sample library is constructed, standardized process collection is performed, sample images containing defect positions, sizes and categories are marked, then an incremental learning model which is based on a pre-trained convolutional network and integrates feature distillation and an elastic weight integration algorithm is constructed, and a basic recognition capability is obtained through initial training; when a new defect type appears, collecting and preprocessing a new sample, and then starting incremental learning to train and update the model; and finally, carrying out real-time detection on an online product by utilizing the updated model, and outputting an accurate result containing the existence, the category, the position and the area of the defect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect identification, more particularly, to an incremental learning visual defect identification system and method for industrial quality inspection. BACKGROUND

[0002] With the rapid development of industrial manufacturing towards intelligence and refinement, product quality detection as a key link in the production process, its efficiency and accuracy directly determine the product competitiveness and market acceptance. Currently, the industrial quality inspection field mainly adopts manual quality inspection and traditional machine vision quality inspection, but in actual application, both have significant limitations, which are difficult to meet the high requirements of modern industrial production on quality inspection.

[0003] Manual quality inspection relies on the experience and naked eye judgment of quality inspection personnel, and has inherent defects such as strong subjectivity, low efficiency, and easy to be affected by fatigue. For high-precision parts (such as automobile engine parts, electronic chip pins) or micro-defects (such as micron-level scratches on metal surface, fine bubbles inside plastic parts), the accuracy rate of manual quality inspection is often less than 85%, and the per capita detection efficiency is only 10-20 pieces per minute, which is difficult to adapt to the high-speed assembly line production rhythm of more than 300 pieces per minute. At the same time, the standard of manual quality inspection is difficult to unify, and there are differences in the defect judgment threshold of different quality inspection personnel, which leads to poor consistency of the quality inspection results of the same batch of products, increasing the difficulty of product quality control. In addition, personnel engaged in manual quality inspection for a long time are prone to misjudgment and missed detection due to visual fatigue, especially in high-intensity quality inspection scenarios, the missed detection rate can be as high as 5%-8%, which brings potential quality risks and economic losses to enterprises. SUMMARY

[0004] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide an incremental learning visual defect identification system and method for industrial quality inspection to solve the problems raised in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0006] The sample library module is constructed: an initial visual defect sample library of industrial products is constructed, and the initial visual defect sample library contains sample images corresponding to the defect types of industrial products, and the defect position, defect size and defect category information of each sample image are labeled;

[0007] The preprocessing module: the sample images in the initial visual defect sample library are preprocessed;

[0008] The model construction module: an incremental learning visual defect identification model is constructed;

[0009] An initial training module: the initial visual defect sample library is used to initially train the incremental learning visual defect recognition model;

[0010] A new defect sample set construction module: when a new product defect type appears in the industrial quality inspection process, sample images corresponding to the new defect type are collected, a new defect sample set is constructed, and the sample images in the new defect sample set are subjected to the preprocessing operations described in the preprocessing module;

[0011] A defect recognition module: the image of the industrial product to be inspected is input into the incremental learning updated visual defect recognition model, and the model outputs the defect recognition result of the product to be inspected, which includes whether there is a defect, the defect category, the defect position coordinates, and the defect area size.

[0012] Preferably, in the sample library construction module, the construction of the sample library includes four steps of sample collection, defect type coverage, information labeling, and quality control. In the sample collection stage, according to the material, shape, and production scene of the industrial product, an imaging device is selected for collection. For metal or glass products with smooth surfaces, a line array camera with a resolution of not less than 2560x1920 pixels is used, and a ring light source or coaxial light source is used for collection. For plastic or composite products with complex textures, a surface array camera with a resolution of not less than 1920x1080 pixels is selected, and a multi-angle light source combination is used for collection.

[0013] The collection environment light intensity is stable at 800-1200 lux, and the collection platform needs to be equipped with a clamp to ensure that the product is fixed in position during imaging. The collection angle is adjusted to 30°-60° according to the surface characteristics of the product, among which the flat surface product is collected vertically, and the curved surface product is collected with multiple viewing angles.

[0014] In terms of defect type coverage, the initial visual defect sample library needs to comprehensively include typical defect types in combination with common quality problems of the target industrial product, weak links in the production process, and industry standard requirements.

[0015] Preferably, in the preprocessing module, the sample images in the initial visual defect sample library are preprocessed, and first, a noise removal operation is performed, and a Gaussian filtering algorithm is used to smooth the sample images. In the Gaussian filtering parameter setting, the noise intensity and defect detail scale of the sample images are dynamically adjusted: for images with low noise intensity and fine defect details, a small convolution kernel of 3x3 is selected, and a small standard deviation of 0.5-1.0 is used; for images with high noise intensity (such as strong electromagnetic interference in a pipeline environment, equipment aging) and large defect size (such as obvious concave, large area corrosion), a larger convolution kernel of 5x5 or 7x7 is used, combined with a larger standard deviation of 1.0-2.0; during filtering processing, Gaussian convolution calculation is performed on the RGB three channels of the image respectively, and then the three channels of processed data are fused again;

[0016] After completing the noise removal, contrast enhancement is performed, and a histogram equalization algorithm is used to optimize the gray scale distribution of the sample images; for images with concentrated gray scale distribution, the gray scale level of the image is uniformly mapped to the entire gray scale range through histogram equalization, and the gray scale difference between the defect area and the background area is expanded; after enhancement processing, the contrast gain value of the image is calculated to ensure that the gain value is within the range of 1.5-3.0.

[0017] Preferably, in the model construction module, the incremental learning visual defect recognition model is based on a pre-trained convolutional neural network as a basic framework, and an incremental learning module is added after the fully connected layer of the pre-trained convolutional neural network. The incremental learning module includes a feature distillation unit and a parameter adaptive updating unit. The feature distillation unit uses a knowledge distillation algorithm to transfer the defect feature knowledge learned by the old model to the new model, and the parameter adaptive updating unit uses an elastic weight integration (EWC) algorithm to constrain the parameters related to the old defect type recognition in the model, avoiding forgetting the old defect knowledge during parameter updating.

[0018] The feature distillation unit uses a knowledge distillation algorithm to build a teacher-student learning mechanism, where the teacher model is the model trained before incremental learning (such as the model after initial training or the model after the last round of incremental learning), and the student model is the current model to be updated. In the distillation process, first, input the sample image containing the old defect type into the teacher model to obtain the feature output of the intermediate feature layer of the model as the teacher signal, which includes the teacher model's deep understanding of the old defect features (such as the distinguishing features of defects and backgrounds, and the difference features of different defect types); then input the same sample image into the student model to obtain the student signal of the corresponding intermediate feature layer; then construct a distillation loss function by calculating the mean square error (MSE) between the student signal and the teacher signal, and the calculation method is as follows:

[0019]

[0020] wherein, denotes the distillation loss, N denotes the number of samples, denotes the number of the i-th sample, denotes the student signal, denotes the teacher signal; denotes the square of the L2 norm, used to calculate the square of the Euclidean distance between two vectors;

[0021] Finally, in the student model training process, the distillation loss is combined with the classification loss of the new defect type, and the student model parameters are optimized through back propagation, so that the student model learns the characteristics of the new defect while forcing the feature output to be consistent with the teacher model;

[0022] The parameter adaptive updating unit adopts an elastic weight integration algorithm, which protects key parameters related to old defect recognition by applying differentiated constraints on model parameters. The specific process is as follows: first, before incremental learning, calculate the importance weight of all parameters of the teacher model in the old defect training task. For each parameter in the model, calculate the sum of the squares of the gradients at each parameter update in the old task training process. The calculation method is as follows:

[0023]

[0024] wherein, denotes the sum of the squares of the gradients at each parameter update in the old task training process, T denotes the number of old task training steps, denotes the loss function of the old task, denotes the j-th parameter in the current model, denotes the gradient of the parameter in the t-th step of training;

[0025] The larger the calculated , the greater the impact of the parameter on old defect recognition, and the update constraint is applied;

[0026] When training the student model in incremental learning, an EWC regularization term is added to the loss function, and the calculation method is as follows:

[0027]

[0028] wherein, denotes the total loss function, denotes the classification loss of the new defect type, denotes the distillation loss, denotes the regularization coefficient, denotes the j-th parameter in the current model, Let j be the j-th parameter in the teacher model.

[0029] Preferably, in the initial training module, before initial training, the initial visual defect sample library is divided and preprocessed for enhancement; the sample library is divided into training set, validation set, and test set in a 7:2:1 ratio, and a stratified sampling strategy is adopted during the division process; at the same time, targeted data augmentation operations are performed on the training set; in the initial model training stage, the loss function and optimizer are configured first; when using the cross-entropy loss function to calculate the loss between the model's predicted value and the sample's labeled value, the loss function is improved to address the labeling bias that may exist in industrial defect samples (such as a 1-2 pixel error between the manually labeled defect boundary and the actual boundary), and a boundary tolerance coefficient is introduced: when the pixel error between the model's predicted defect boundary and the labeled boundary is within 2 pixels, the loss weight of that area is reduced; for the core defect area (the area within 50% of the labeled boundary), the loss weight is increased; the Adam optimizer is selected as the optimizer, and a learning rate decay strategy is set; the training batch size is set to 16-64, and the training epochs are set to 50-200.

[0030] Preferably, in the module for constructing a new defect sample set, during the new defect sample collection stage, a new defect characteristic analysis is first performed. This involves tracing the source of new defect types in conjunction with production process personnel and quality inspection experts. Simultaneously, the characteristic significance index S of the new defect is calculated to determine the selection criteria for the collection equipment. The specific calculation method is as follows:

[0031]

[0032] Where S represents the significance index of the feature. This is represented by the average gray value of the new defect area. This is represented as the average grayscale value of the background area. This is expressed as the standard deviation of the grayscale values ​​in the background area.

[0033] when At that time, the new defect had high contrast with the background, so a conventional industrial camera was selected; when At that time, a high-resolution camera is selected to enhance feature capture capabilities; when At the same time, special light sources (such as polarized light and infrared light) are used, and image fusion technology is enabled;

[0034] The data acquisition equipment and environmental control adopt the standard for constructing a sample library module, and are optimized in conjunction with the "spatial feature parameters" of new defects. For minute defects, in addition to selecting an industrial line scan camera with a resolution of not less than 2560×1920 pixels, the spatial resolution of the camera needs to be calculated, i.e.: R represents the spatial resolution, L represents the physical pixel size of the camera, and W represents the object-side field of view width; the illumination intensity of the acquisition environment is controlled between 800-1200 lux;

[0035] The sample quantity and annotation are executed according to the sample library construction module, and the annotation accuracy is quantified by a formula. If new defect samples are scarce, data augmentation techniques are used to expand the sample pool. Taking random rotation as an example, the rotation angle... Follows uniform distribution Rotated pixel coordinates Calculations are performed using the affine transformation formula:

[0036]

[0037] in, Represented as raw pixel coordinates, , Expressed as translation;

[0038] During the annotation process, the boundary box error E of the defect location annotation must meet the following requirements. The specific method for calculating the bounding box error is as follows:

[0039]

[0040] in, , Represented as the coordinates of the bounding box. , The coordinates are represented by the actual defect bounding box coordinates; W represents the object-side field of view width, and H represents the object-side field of view height.

[0041] Meanwhile, the SIFT algorithm is used to calculate the feature similarity between the new defect sample and the old defect sample. The specific calculation method is as follows:

[0042]

[0043] in, Represented as feature similarity, This is expressed as the number of matched feature points. This represents the total number of feature points in the new defect sample. This represents the total number of feature points in the old defect sample.

[0044] When calculated At this point, the old defect features and the new defect features can be distinguished; when the calculated... At this point, the old and new defect features are indistinguishable, requiring processing of the new defect samples. This involves re-collecting new defect sample data and adjusting SIFT algorithm parameters, such as changing the feature point detection threshold and scale space parameters, to recalculate feature similarity. If the above methods fail to meet the requirements, other feature extraction algorithms are used to recalculate the similarity between the old and new defect samples until the requirements are met. The conditions are as follows: After verification, the new defect sample set is stored in a structured manner according to the storage format of the initial sample library.

[0045] Preferably, in the defect recognition module, after inputting the preprocessed image to be inspected into the incrementally learned and updated visual defect recognition model, the model inference process is initiated; the defect recognition result output by the incrementally learned and updated visual defect recognition model includes information on whether a defect exists, the defect category, the defect location coordinates, and the defect area size, and each piece of information must undergo precise calculation and standardized processing:

[0046] In determining whether a defect exists, the visual defect recognition model uses the output defect probability threshold for judgment. The preset probability threshold is 0.8 (adjusted according to the rigor of quality inspection; the threshold for precision parts is set to 0.9, and the threshold for general building materials is set to 0.7). If the maximum defect category probability output by the model is greater than or equal to the threshold, a defect is determined to exist; if the probabilities of all defect categories are less than the threshold, no defect is determined to exist, and the maximum defect category probability is output for reference by quality inspectors.

[0047] In defect category determination, the defect category with the highest output probability from the model is selected as the final defect category. If multiple defect categories have similar probabilities (e.g., the difference between the highest probability and the second highest probability is less than 0.1), a feature matching algorithm is further used to compare the defect features of the image to be inspected with the standard feature library of each suspected defect category, and the feature similarity is calculated. The defect category with the highest similarity is selected as the final result. At the same time, the suspected category and its corresponding probability are marked in the result (e.g., "Main defect category: scratch, probability 0.85; suspected category: dent, probability 0.78") to assist quality inspectors in review.

[0048] In the calculation of defect location coordinates, pixel coordinate positioning is used for calculation;

[0049] In the calculation of defect area size, the pixel area is calculated based on the coordinates of the defect boundary box. If the defect is irregular in shape (such as irregular cracks or flaky corrosion), the actual contour of the defect is extracted by an image segmentation algorithm, and the number of pixels (i.e., pixel area) of the area enclosed by the contour is calculated. Then, combined with the physical size scale of the image of the product to be inspected, the pixel area is converted into the actual physical area.

[0050] The technical effects and advantages of this invention are as follows:

[0051] This invention constructs a sample library, selects equipment to collect images based on product characteristics, annotates defect information, and reviews them. Preprocessing optimizes images through Gaussian filtering, histogram equalization, and size normalization. The model is based on a pre-trained convolutional neural network, with an incremental module incorporating feature distillation and adaptive parameter updates. Initial training involves dividing the dataset and training with an improved cross-entropy loss and Adam optimizer. When a new defect appears, the feature significance index is calculated to select the appropriate equipment for collection, and a sample set is constructed after preprocessing. During defect identification, the model infers and outputs results, which are then validated a second time. This invention effectively solves the problem of traditional deep learning models needing retraining when new defects appear, significantly improving the adaptability and continuous operating efficiency of industrial quality inspection systems, reducing the probability of false positives and false negatives, and mitigating potential quality risks and economic losses. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the module connection of the present invention.

[0053] Figure 2 This is a schematic diagram of the method flow of the present invention.

[0054] Figure 3 This is a schematic diagram of the model building module process of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] This invention provides an incremental learning visual defect recognition system for industrial quality inspection, comprising:

[0057] Sample library construction module: Constructs an initial visual defect sample library for industrial products. The initial visual defect sample library contains sample images corresponding to the defect types of industrial products, and labels the defect location, defect size and defect category information for each sample image.

[0058] The sample library construction module includes four steps: sample collection, defect type coverage, information labeling, and quality control. During the sample collection stage, imaging equipment is selected based on the material, shape, and production scenario of the industrial products. For smooth metal or glass products, a line scan camera with a resolution of at least 2560×1920 pixels is used, combined with a ring light source or coaxial light source. For plastic or composite material products with complex textures, an area scan camera with a resolution of at least 1920×1080 pixels is used, combined with a multi-angle light source combination.

[0059] The ambient light intensity should be kept stable at 800-1200 lux. The acquisition platform should be equipped with a fixture to ensure that the product is fixed in position during the imaging process. The acquisition angle should be adjusted to 30°-60° according to the surface characteristics of the product. For planar products, a vertical acquisition angle should be used, and for curved products, a multi-view surround acquisition should be used.

[0060] In terms of defect type coverage, the initial visual defect sample library needs to comprehensively include typical defect types by combining common quality problems of target industrial products, weak links in production processes and industry standard requirements.

[0061] In the sample image information annotation stage, image annotation tools are used for annotation. The annotation content includes defect location, defect size, and defect category information. Among them, defect location annotation adopts pixel coordinate positioning method, establishing a two-dimensional coordinate system with the upper left corner of the image as the origin. For regular-shaped defects (such as circular impurities and rectangular depressions), the upper left corner coordinates of the defect boundary box are annotated. and the coordinates of the bottom right corner And mark the coordinates of the defect center point within the box. For irregularly shaped defects (such as irregular cracks and flaky corrosion), a polygon annotation tool is used to annotate each point along the defect edge, generating a coordinate sequence of the defect outline. The defect size annotation includes pixel size and actual physical size. The pixel size is obtained by calculating the number of pixels in the defect area (the area of ​​the bounding box for regular defects, and the sum of pixels within the outline for irregular defects). The actual physical size is converted according to the scale of the image acquisition. The annotation format is "pixel size: ** pixels; actual size: ** mm". For linear defects, the length, width, and direction of extension of the defect are annotated.

[0062] The defect category labeling adopts a hierarchical classification system. The first-level category is the major category to which the defect belongs, the second-level category is the specific defect type, and the third-level category is the defect subdivision feature. When labeling, the image is uniformly labeled in the format of "first-level category - second-level category - third-level category" (such as "surface defect - scratch - moderate scratch"). At the same time, the cause of the defect is added in the labeling remarks column.

[0063] A sample entry review mechanism should be established, and sample quality assessment indicators should be set, including image clarity, defect identification, and labeling accuracy. For sample images that do not meet the standards, the labeling information should be re-collected or corrected until the requirements are met before they can be entered into the database.

[0064] Preprocessing module: preprocesses the sample images in the initial visual defect sample library;

[0065] In the preprocessing module, sample images from the initial visual defect sample library are preprocessed. First, noise removal is performed, and Gaussian filtering is used to smooth the sample images. The Gaussian filtering parameters are dynamically adjusted based on the noise intensity and defect detail scale of the sample images: for images with low noise intensity and fine defect details, a small 3×3 convolution kernel is selected, paired with a small standard deviation of 0.5-1.0; for images with high noise intensity (such as strong electromagnetic interference in a production line environment or equipment aging) and large defect size (such as obvious dents or large-area corrosion), a larger 5×5 or 7×7 convolution kernel is used, paired with a larger standard deviation of 1.0-2.0. During filtering, Gaussian convolution is performed on the RGB three channels of the image, and then the processed three-channel data are re-fused.

[0066] After noise removal, contrast enhancement is performed, and histogram equalization algorithm is used to optimize the gray-level distribution of the sample image. For images with concentrated gray-level distribution, histogram equalization is used to uniformly map the gray-level of the image to the entire gray-level range, expanding the gray-level difference between the defect area and the background area. After enhancement processing, the contrast gain value of the image is calculated to ensure that the gain value is within the range of 1.5 - 3.0.

[0067] Finally, size normalization is performed to uniformly adjust the sample images, which have undergone noise removal and contrast enhancement, to a fixed size. During the size adjustment process, a bilinear interpolation algorithm is used to scale the image. This algorithm performs a smooth size transformation by calculating the weighted average of the four pixels surrounding the interpolation point. If the aspect ratio of the original image is inconsistent with that of the target size, the original image is first centered or edge-filled: when the defect is located in the center of the image, centered cropping is used; when the defect is close to the edge of the image, edge filling is used, and then the size is scaled.

[0068] Model building module: Constructs an incremental learning visual defect recognition model;

[0069] In the model building module, the incremental learning visual defect recognition model is based on a pre-trained convolutional neural network. An incremental learning module is added after the fully connected layer of the pre-trained convolutional neural network. The incremental learning module includes a feature distillation unit and a parameter adaptive update unit. The feature distillation unit uses a knowledge distillation algorithm to transfer the defect feature knowledge learned by the old model to the new model. The parameter adaptive update unit uses an elastic weight integration (EWC) algorithm to impose constraints on the parameters in the model related to the old defect type recognition, so as to avoid forgetting the old defect knowledge during the parameter update process.

[0070] When adding an incremental learning module after the fully connected layer of a pre-trained convolutional neural network, the fully connected layer is improved by adjusting the output dimension of the original fully connected layer to match the initial number of defect types (e.g., if the initial number of defects is 5, the output dimension of the fully connected layer is set to 5), while retaining the parameter update channel of the fully connected layer.

[0071] The feature distillation unit employs a knowledge distillation algorithm to construct a teacher-student learning mechanism. The teacher model is a pre-trained model before incremental learning (e.g., the model after initial training or the model after the previous round of incremental learning), while the student model is the model currently awaiting update. During distillation, firstly, sample images containing old defect types are input into the teacher model, and the feature output of the intermediate feature layer is obtained as the teacher signal. This signal includes the teacher model's deep understanding of old defect features (e.g., features distinguishing defects from the background, and differences between different defect types). Next, the same sample image is input into the student model to obtain the corresponding student signal from the intermediate feature layer. Then, the distillation loss function is constructed by calculating the mean squared error (MSE) between the student signal and the teacher signal. The specific calculation method is as follows:

[0072]

[0073] in, This represents the distillation loss, and N represents the sample size. Let represent the number of the i-th sample. This is represented as a student signal. This is represented as a teacher signal; It is expressed as the square of the L2 norm and is used to calculate the square of the Euclidean distance between two vectors;

[0074] Finally, during the training of the student model, the distillation loss is combined with the classification loss of the new defect type, and the parameters of the student model are optimized through backpropagation, so that the student model is forced to keep its feature output consistent with the teacher model while learning the new defect features.

[0075] The parameter adaptive update unit employs an elastic weight integration algorithm, which protects key parameters related to old defect identification by imposing differential constraints on model parameters. The specific process is as follows: First, before incremental learning, the importance weights of all parameters of the teacher model in the old defect training task are calculated. For each parameter in the model, the sum of squared gradients during each parameter update in the old task training process is calculated. The specific calculation method is as follows:

[0076]

[0077] in, Let represent the sum of squared gradients of the j-th parameter during each parameter update in the old task training process, and T represent the number of training steps in the old task. This is represented as the loss function for the old task. Let j be the j-th parameter in the current model. Let be the parameters in step t of the training. The gradient;

[0078] When calculated The larger the value, the greater the influence of the parameter on the identification of old defects, and an update constraint is applied.

[0079] When training the student model using incremental learning, an EWC regularization term is added to the loss function. The specific calculation method is as follows:

[0080]

[0081] in, Represented as the total loss function, This is represented as the classification loss for the new defect type. This is expressed as distillation loss. Represented as the regularization coefficient, Let j be the j-th parameter in the current model. This is represented as the j-th parameter in the teacher model;

[0082] During training, when the model updates parameters At that time, if Larger (key parameter) regularization terms will incur penalty loss; limit the parameter. Deviation The degree; if If the parameters are relatively small (non-critical parameters), they can be freely updated to adapt to the learning needs of new defect features; at the same time, the regularization coefficient is dynamically adjusted according to the degree of difference between the new defect type and the old defect type. When the characteristics of the new defect differ significantly from those of the old defect, Set to 1e4 - 1e5 to enhance the protection of critical parameters; when the characteristics of new defects are not significantly different from those of old defects, [the following will be implemented]. Set to 1e3 - 5e3.

[0083] Initial training module: Initial training of the incremental learning visual defect recognition model based on the initial visual defect sample library;

[0084] In the initial training module, before initial training, the initial visual defect sample library is divided and preprocessed for enhancement. The sample library is divided into training, validation, and test sets in a 7:2:1 ratio, using a stratified sampling strategy. Simultaneously, targeted data augmentation is performed on the training set. Upon entering the initial model training phase, the loss function and optimizer are configured. When calculating the loss between the model's predicted value and the labeled sample value using the cross-entropy loss function, the loss function is improved to address potential labeling biases in industrial defect samples (e.g., a 1-2 pixel error between manually labeled defect boundaries and actual boundaries). A boundary tolerance coefficient is introduced: when the pixel error between the model's predicted defect boundary and the labeled boundary is within 2 pixels, the loss weight for that region is reduced; for the core defect region (50% of the area within the labeled boundary), the loss weight is increased. The Adam optimizer is selected, and a learning rate decay strategy is set. The training batch size is set to 16-64, and the training epochs are set to 50-200.

[0085] A real-time monitoring mechanism needs to be established during training, using the TensorBoard tool to record changes in key metrics: After each training round, calculate the average loss value of the training set and the recognition accuracy of various defects, while simultaneously calculating the overall defect recognition accuracy, recall and precision of various defects, and defect localization accuracy on the validation set; if it is found that the loss value of the training set continues to decrease but the loss value of the validation set begins to increase, it indicates that the model is overfitting, and immediate countermeasures should be taken, such as increasing the data augmentation intensity of the training set, adding a Dropout layer to the basic network, and reducing the learning rate; if the loss values ​​of both the training set and the validation set continue to decrease but the rate of decrease is slow, increase the learning rate to accelerate model convergence.

[0086] New Defect Sample Set Construction Module: When a new type of product defect appears during industrial quality inspection, sample images corresponding to the new defect type are collected to construct a new defect sample set. The preprocessing operations described in the steps and preprocessing module are then performed on the sample images in the new defect sample set.

[0087] In the module for constructing a new defect sample set, during the new defect sample collection phase, the characteristics of the new defects are first analyzed. The source of the new defect types is traced through collaboration between production process personnel and quality inspection experts. Simultaneously, the characteristic significance index S of the new defects is calculated to determine the selection criteria for the collection equipment. The specific calculation method is as follows:

[0088]

[0089] Where S represents the significance index of the feature. This is represented by the average gray value of the new defect area. This is represented as the average grayscale value of the background area. This is expressed as the standard deviation of the grayscale values ​​in the background area.

[0090] when At that time, the new defect had high contrast with the background, so a conventional industrial camera was selected; when At that time, a high-resolution camera is selected to enhance feature capture capabilities; when At the same time, special light sources (such as polarized light and infrared light) are used, and image fusion technology is enabled;

[0091] The data acquisition equipment and environmental control adopt the standard for constructing a sample library module, and are optimized in conjunction with the "spatial feature parameters" of new defects. For minute defects, in addition to selecting an industrial line scan camera with a resolution of not less than 2560×1920 pixels, the spatial resolution of the camera needs to be calculated, i.e.: R represents the spatial resolution, L represents the physical pixel size of the camera, and W represents the object-side field of view width; the illumination intensity of the acquisition environment is controlled between 800-1200 lux;

[0092] The sample quantity and annotation are executed according to the sample library construction module, and the annotation accuracy is quantified by a formula. If new defect samples are scarce, data augmentation techniques are used to expand the sample pool. Taking random rotation as an example, the rotation angle... Follows uniform distribution Rotated pixel coordinates Calculations are performed using the affine transformation formula:

[0093]

[0094] in, Represented as raw pixel coordinates, , Expressed as translation;

[0095] During the annotation process, the boundary box error E of the defect location annotation must meet the following requirements. The specific method for calculating the bounding box error is as follows:

[0096]

[0097] in, , Represented as the coordinates of the bounding box. , The coordinates are represented by the actual defect bounding box coordinates; W represents the object-side field of view width, and H represents the object-side field of view height.

[0098] In defect size labeling, the actual physical size is converted from pixel size to scale.

[0099] The preprocessing of the new defect sample set follows the workflow of the preprocessing module, and the preprocessing parameters are defined by formulas. In the noise removal stage, Gaussian filtering algorithm is used, and in the contrast enhancement stage, histogram equalization is used. After the preprocessing is completed, quality verification is required. The verification index is quantified by formulas, and the noise removal effect is evaluated by peak signal-to-noise ratio.

[0100] Histogram equalization is used, and the specific method for calculating the change function is as follows:

[0101]

[0102] in, Let C be the transformation function, and C be the number of gray levels. Let A represent the total number of pixels in the image, and B represent the number of rows and columns in the image. It represents the number of pixels with a grayscale value of k, and r represents the original grayscale value; This is represented as rounding down to the nearest integer.

[0103] The contrast enhancement effect is evaluated through contrast gain, and the calculation method is as follows:

[0104]

[0105] Where G represents the contrast gain. This represents the average grayscale value of the newly enhanced defective region. This represents the average grayscale value of the enhanced background area. This is represented by the average gray value of the new defect area. This is represented as the average grayscale value of the background area;

[0106] Meanwhile, the SIFT algorithm is used to calculate the feature similarity between the new defect sample and the old defect sample. The specific calculation method is as follows:

[0107]

[0108] in, Represented as feature similarity, This is expressed as the number of matched feature points. This represents the total number of feature points in the new defect sample. This represents the total number of feature points in the old defect sample.

[0109] When calculated At this point, the old defect features and the new defect features can be distinguished; when the calculated... At this point, the old and new defect features are indistinguishable, requiring processing of the new defect samples. This involves re-collecting new defect sample data and adjusting SIFT algorithm parameters, such as changing the feature point detection threshold and scale space parameters, to recalculate feature similarity. If the above methods fail to meet the requirements, other feature extraction algorithms are used to recalculate the similarity between the old and new defect samples until the requirements are met. The conditions are as follows: After verification, the new defect sample set is stored in a structured manner according to the storage format of the initial sample library.

[0110] The preprocessed new defect sample set is input into the initially trained visual defect recognition model to start the incremental learning training process;

[0111] Defect recognition module: The image of the industrial product to be inspected is input into the visual defect recognition model after incremental learning. The model outputs the defect recognition result of the product to be inspected. The defect recognition result includes whether there is a defect, the defect type, the defect location coordinates and the defect area size.

[0112] In the defect identification module, the preprocessed image to be inspected is input into the incrementally learned and updated visual defect identification model, and the model inference process is started. The defect identification result output by the incrementally learned and updated visual defect identification model includes information on whether a defect exists, the defect category, the defect location coordinates, and the defect area size. Each piece of information must undergo precise calculation and standardization processing.

[0113] In determining whether a defect exists, the visual defect recognition model uses the output defect probability threshold for judgment. The preset probability threshold is 0.8 (adjusted according to the rigor of quality inspection; the threshold for precision parts is set to 0.9, and the threshold for general building materials is set to 0.7). If the maximum defect category probability output by the model is greater than or equal to the threshold, a defect is determined to exist; if the probabilities of all defect categories are less than the threshold, no defect is determined to exist, and the maximum defect category probability is output for reference by quality inspectors.

[0114] In defect category determination, the defect category with the highest output probability from the model is selected as the final defect category. If multiple defect categories have similar probabilities (e.g., the difference between the highest probability and the second highest probability is less than 0.1), a feature matching algorithm is further used to compare the defect features of the image to be inspected with the standard feature library of each suspected defect category, and the feature similarity is calculated. The defect category with the highest similarity is selected as the final result. At the same time, the suspected category and its corresponding probability are marked in the result (e.g., "Main defect category: scratch, probability 0.85; suspected category: dent, probability 0.78") to assist quality inspectors in review.

[0115] In the calculation of defect location coordinates, pixel coordinate positioning is used for calculation;

[0116] In the calculation of defect area size, the pixel area is calculated based on the coordinates of the defect bounding box. If the defect is irregular in shape (such as irregular cracks or flaky corrosion), the actual contour of the defect is extracted by an image segmentation algorithm, and the number of pixels (i.e., pixel area) of the area enclosed by the contour is calculated. Then, combined with the physical size scale of the image of the product to be inspected, the pixel area is converted into the actual physical area.

[0117] A secondary verification mechanism is then established. For results indicating defects, the system automatically retrieves historical sample libraries for that defect category and compares the similarity between the defect features of the image to be inspected and the features of the historical samples using SIFT feature matching. If the similarity is greater than 0.85, the defect identification result is confirmed as valid; if the similarity is less than 0.6, it is marked as "pending verification," prompting quality inspectors to conduct manual verification. Simultaneously, the reasonableness of the defect location and area is verified. If the defect location exceeds the actual effective area of ​​the product (e.g., beyond the edge of the board) or the defect area is too large / too small (e.g., the area is greater than 50% of the total product area or less than 0.01 mm²), an abnormal alarm is automatically triggered to investigate whether there are image preprocessing errors or model inference anomalies. Finally, the defect identification results are output in a structured format, and the image of the product to be inspected is displayed in real time on the visualization interface, with the defect bounding box, defect category label, and area information overlaid.

[0118] Please see Figure 2 As shown, this invention provides an incremental learning visual defect recognition method for industrial quality inspection, comprising:

[0119] A1: Construct an initial visual defect sample library for industrial products. The initial visual defect sample library contains sample images corresponding to the defect types of industrial products, and labels the defect location, defect size and defect category information for each sample image.

[0120] A2: Preprocess the sample images in the initial visual defect sample library;

[0121] A3: Construct an incremental learning visual defect recognition model;

[0122] A4: Initial training of the incremental learning visual defect recognition model is performed based on the initial visual defect sample library;

[0123] A5: When a new type of product defect appears during industrial quality inspection, sample images corresponding to the new defect type are collected, a new defect sample set is constructed, and the preprocessing operations described in the steps and preprocessing module are performed on the sample images in the new defect sample set.

[0124] A6: Input the image of the industrial product to be inspected into the visual defect recognition model after incremental learning. The model outputs the defect recognition result of the product to be inspected. The defect recognition result includes whether there is a defect, the defect category, the defect location coordinates, and the defect area size.

[0125] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An incremental learning visual defect recognition system for industrial quality inspection, characterized in that, include: Sample library construction module: Constructs an initial visual defect sample library for industrial products. The initial visual defect sample library contains sample images corresponding to the defect types of industrial products, and labels the defect location, defect size and defect category information for each sample image. Preprocessing module: preprocesses the sample images in the initial visual defect sample library; Model building module: Constructs an incremental learning visual defect recognition model; Initial training module: Initial training of the incremental learning visual defect recognition model based on the initial visual defect sample library; New Defect Sample Set Construction Module: When a new type of product defect appears during industrial quality inspection, sample images corresponding to the new defect type are collected to construct a new defect sample set. The preprocessing operations described in the steps and preprocessing module are then performed on the sample images in the new defect sample set. Defect recognition module: The image of the industrial product to be inspected is input into the visual defect recognition model after incremental learning. The model outputs the defect recognition result of the product to be inspected. The defect recognition result includes whether there is a defect, the defect type, the defect location coordinates and the defect area size.

2. The incremental learning visual defect recognition system for industrial quality inspection according to claim 1, characterized in that: The sample library construction module includes four steps: sample collection, defect type coverage, information annotation, and quality control. During the sample collection phase, imaging equipment was selected for collection based on the material, shape, and production scenario of the industrial products. The ambient light intensity was 800-1200 lux, and the collection angle was adjusted to 30°-60° according to the surface characteristics of the products. For planar products, a vertical collection angle was used, while for curved products, a multi-view surround collection was used. In terms of defect type coverage, the initial visual defect sample library needs to comprehensively include typical defect types, taking into account common quality problems of the target industrial products, weak links in the production process, and industry standard requirements. In the sample image information annotation phase, image annotation tools were used for annotation, and the annotation content included defect location, defect size, and defect category information.

3. The incremental learning visual defect recognition system for industrial quality inspection according to claim 1, characterized in that: In the preprocessing module, sample images from the initial visual defect sample library are preprocessed. First, noise removal is performed by smoothing the sample images using a Gaussian filtering algorithm. After noise removal, contrast enhancement is performed by optimizing the gray-level distribution of the sample images using a histogram equalization algorithm. For images with concentrated gray-level distribution, histogram equalization is used to uniformly map the gray levels of the image to the entire gray-level range, expanding the gray-level difference between the defect area and the background area. After enhancement processing, the contrast gain value of the image is calculated. Finally, size normalization is performed to uniformly adjust the sample images that have undergone noise removal and contrast enhancement to a fixed size.

4. The incremental learning visual defect recognition system for industrial quality inspection according to claim 1, characterized in that: In the model building module, the incremental learning visual defect recognition model is based on a pre-trained convolutional neural network. An incremental learning module is added after the fully connected layer of the pre-trained convolutional neural network. The incremental learning module includes a feature distillation unit and a parameter adaptive update unit. The feature distillation unit uses a knowledge distillation algorithm to transfer the defect feature knowledge learned by the old model to the new model. The parameter adaptive update unit uses an elastic weight integration algorithm to impose constraints on the parameters in the model related to the old defect type recognition. When adding an incremental learning module after the fully connected layer of a pre-trained convolutional neural network, the fully connected layer is improved by adjusting the output dimension of the original fully connected layer to match the initial number of defect types, while retaining the parameter update channel of the fully connected layer. The feature distillation unit employs a knowledge distillation algorithm to construct a teacher-student learning mechanism. The teacher model is a pre-trained model before incremental learning, while the student model is the current model awaiting update. During distillation, firstly, sample images containing old defect types are input into the teacher model, and the feature output of the intermediate feature layer is obtained as the teacher signal. This signal reflects the teacher model's deep understanding of the old defect features. Next, the same sample image is input into the student model to obtain the corresponding student signal from the intermediate feature layer. Then, the distillation loss function is constructed by calculating the mean squared error between the student signal and the teacher signal. The specific calculation method is as follows: ； in, This represents the distillation loss, and N represents the sample size. Let represent the number of the i-th sample. This is represented as a student signal. This is represented as a teacher signal; It is expressed as the square of the L2 norm.

5. The incremental learning visual defect recognition system for industrial quality inspection according to claim 4, characterized in that: The parameter adaptive update unit employs an elastic weight integration algorithm. The specific process is as follows: Before incremental learning, the importance weights of all parameters in the teacher model in the old defect training task are calculated. For each parameter in the model, the sum of squared gradients during each parameter update in the old task training process is calculated. The specific calculation method is as follows: ； in, Let represent the sum of squared gradients of the j-th parameter during each parameter update in the old task training process, and T represent the number of training steps in the old task. This is represented as the loss function for the old task. Let j be the j-th parameter in the current model. Let be the parameters in step t of the training. The gradient; When calculated The larger the value, the greater the influence of the parameter on the identification of old defects, and an update constraint is applied. When training the student model using incremental learning, an EWC regularization term is added to the loss function. The specific calculation method is as follows: ； in, Represented as the total loss function, This is represented as the classification loss for the new defect type. This is expressed as distillation loss. Represented as the regularization coefficient, Let j be the j-th parameter in the current model. Let j be the j-th parameter in the teacher model.

6. The incremental learning visual defect recognition system for industrial quality inspection according to claim 1, characterized in that: In the initial training module, before initial training, the initial visual defect sample library is divided and preprocessed for enhancement. The sample library is divided into training, validation, and test sets in a 7:2:1 ratio, using a stratified sampling strategy. Simultaneously, targeted data augmentation operations are performed on the training set. Upon entering the initial model training phase, the loss function and optimizer are configured. When calculating the loss between the model's predicted value and the labeled value using the cross-entropy loss function, the loss function is improved to address the labeling bias present in industrial defect samples by introducing a boundary tolerance coefficient: when the pixel error between the defect boundary predicted by the incremental learning visual defect recognition model and the labeled boundary is within 2 pixels, the loss weight for that region is reduced. The Adam optimizer is selected, and a learning rate decay strategy is set. The training batch size is set to 16-64, and the training epochs are set to 50-200.

7. The incremental learning visual defect recognition system for industrial quality inspection according to claim 1, characterized in that: In the module for constructing a new defect sample set, during the new defect sample collection phase, the characteristics of the new defects are first analyzed. The source of the new defect types is traced through collaboration between production process personnel and quality inspection experts. Simultaneously, the characteristic significance index S of the new defects is calculated to determine the selection criteria for the collection equipment. The specific calculation method is as follows: ； Where S represents the significance index of the feature. This is represented by the average gray value of the new defect area. This is represented as the average grayscale value of the background area. This is expressed as the standard deviation of the grayscale values ​​in the background area. The data acquisition equipment and environmental control adopt the standard of building a sample library module, and are optimized by combining the "spatial feature parameters" of new defects. The number of samples and annotation are performed according to the sample library module, and the annotation accuracy is quantified by formula. If new defect samples are scarce, they are expanded by data augmentation technology. During the annotation process, the boundary box error E of the defect location annotation must meet the following requirements. The specific method for calculating the bounding box error is as follows: ； in, , Represented as the coordinates of the bounding box. , The coordinates are represented by the actual defect bounding box coordinates; W represents the object-side field of view width, and H represents the object-side field of view height. The preprocessing of the new defect sample set follows the workflow of the preprocessing module, and the preprocessing parameters are clearly defined using formulas. Simultaneously, the SIFT algorithm is used to calculate the feature similarity between the new and old defect samples. The specific calculation method is as follows: ； in, Represented as feature similarity, This is expressed as the number of matched feature points. This represents the total number of feature points in the new defect sample. This represents the total number of feature points in the old defect sample.

8. The incremental learning visual defect recognition system for industrial quality inspection according to claim 1, characterized in that: In the defect recognition module, after the preprocessed image to be inspected is input into the incrementally learned and updated visual defect recognition model, the model inference process is started; the defect recognition result output by the incrementally learned and updated visual defect recognition model includes information on whether a defect exists, the defect category, the defect location coordinates, and the defect area size. In determining whether a defect exists, the visual defect recognition model makes a judgment based on the output defect probability threshold. The preset probability threshold is 0.

8. If the maximum defect category probability output by the model is greater than or equal to the threshold, a defect is determined to exist; if the probabilities of all defect categories are less than the threshold, a defect is determined not to exist, and the maximum defect category probability is output. In the defect category determination, the defect category with the highest output probability from the model is selected as the final defect category; In the calculation of defect location coordinates, pixel coordinate positioning is used; in the calculation of defect area size, the pixel area is calculated based on the defect bounding box coordinates; then a secondary verification mechanism is established. For results that indicate the presence of defects, the historical sample library of the defect category is automatically retrieved, and the defect features of the image to be inspected are compared with the features of the historical samples. SIFT feature matching is used. If the similarity is greater than 0.85, the defect identification result is confirmed to be valid; if the similarity is less than 0.6, it is marked as "to be reviewed" to remind quality inspectors to conduct manual verification.

9. An incremental learning visual defect recognition method for industrial quality inspection, using an incremental learning visual defect recognition system for industrial quality inspection as described in any one of claims 1-8, characterized in that: A1: Construct an initial visual defect sample library for industrial products. The initial visual defect sample library contains sample images corresponding to the defect types of industrial products, and labels the defect location, defect size and defect category information for each sample image. A2: Preprocess the sample images in the initial visual defect sample library; A3: Construct an incremental learning visual defect recognition model; A4: Initial training of the incremental learning visual defect recognition model is performed based on the initial visual defect sample library; A5: When a new type of product defect appears during industrial quality inspection, sample images corresponding to the new defect type are collected, a new defect sample set is constructed, and the preprocessing operations described in the steps and preprocessing module are performed on the sample images in the new defect sample set. A6: Input the image of the industrial product to be inspected into the visual defect recognition model after incremental learning. The model outputs the defect recognition result of the product to be inspected. The defect recognition result includes whether there is a defect, the defect category, the defect location coordinates, and the defect area size.

Images