Coating ink cell surface abnormal defect detection method based on double twin networks
By adopting a dual-twin network and multi-variable scale fusion strategy in surface defect detection, the problems of poor universality and large inter-class ambiguity in the prior art are solved, and the accurate detection and identification of defects in complex scenarios are achieved.
Patent Information
- Application Number
- CN202411595430.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2025-02-28
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art has problems such as poor universality and high inter-class ambiguity in surface defect detection, making it difficult to accurately detect defects in complex scenarios.
Using a dual-twin network method, two twin neural networks with similar structures and weight sharing are designed. Through multi-level feature mapping and multi-variable scale fusion strategy, strong semantic information and precise position information are extracted to realize surface defect abnormality detection.
This method can use only normal samples for training under unsupervised learning, realize accurate detection of abnormal defects, reduce false detection and missed detection, and adapt to different defect types and complex backgrounds.
Smart Images
Figure CN119540177A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of machine perception and machine vision, and specifically relates to a method for detecting abnormal defects on the surface of coating cells based on a double twin network. Background Art
[0002] In surface defect detection, deep learning technology is widely used to extract and classify defect features. By training a deep neural network model, it is possible to automatically learn and understand the various morphological features of defects, achieving more intelligent and adaptive detection capabilities. This method usually involves steps such as image preprocessing, feature extraction, and similarity calculation. By processing and analyzing two or more images, the feature differences between them are extracted, and the similarity or difference index is calculated to evaluate the degree of similarity between them.
[0003] There are large differences between classes, which results in the low universality of the detection algorithm. Secondly, there is large ambiguity between classes. Inter-class ambiguity is the other extreme of large inter-class differences, that is, the apparent features of defects of different categories have certain similarities, making it difficult to distinguish the types of defects. In production scenarios, it is difficult to completely separate defects from the background, and the defect features are not obvious. Traditional methods for defect segmentation have poor versatility and generalization. When the defect contrast is not obvious, there are certain omissions and false detections. Summary of the invention
[0004] In order to solve the technical problems existing in the prior art, the present invention provides a method for detecting abnormal defects on the surface of coating cells based on a double twin network. Two twin neural networks with similar structures and a coupled architecture with weight sharing are designed, and a multi-variable scale fusion strategy is adopted for multi-level feature mapping to achieve strong semantic information and precise position information. Surface defect abnormality detection is realized by calculating the gap loss between two samples.
[0005] To achieve the above purpose, the technical solution adopted by the present invention is: a method for detecting abnormal defects on the surface of coating cells based on a double twin network, and the specific steps are as follows:
[0006] Step S1: acquiring submicron features on the surface of the lithium battery coating material, with intact surface cells as normal samples and damaged surface cells as defective samples;
[0007] Step S2: Construct a twin network architecture. The two structures are similar, and the twin network architecture establishes a neural network with a coupled architecture with weight sharing. The twin network architecture takes two samples as input, maps the input to a new space and forms a representation of the input in the new space, and outputs a representation embedded in a high-dimensional space;
[0008] Step S3: The dual-feature network is sequentially subjected to convolution calculations of different sizes, outputting feature maps containing global detail information of different sizes, extracting the output of the last three layers of the dual-Luansheng network architecture, and defining the features with high-level information in the feature map as global branch nodes;
[0009] The dual feature network is a dual native network VGG16 framework and ResNet18 architecture, and a similar architecture design is performed on the two original extraction networks. The feature map is calculated by convolution with weight sharing within the same layer. The map is generated by the filter acting on the input data to produce the output of a convolution layer as the input of the next layer. Each filter has a set size corresponding to the local receptive field of a single unit in the layer.
[0010] The five convolution modules before the VGG16 framework are retained. Each convolution module contains two convolution layers and pooling layers to stack the neural network, and the pooling layer operation of the last convolution module is removed. The filter size is 3×3.
[0011] The convolution module removes the downsampling operation and uses transposed convolution in the last convolution module to restore the resolution of the feature map and restore the feature map size to the same as the output of the third convolution module;
[0012] By linearly combining the feature maps of different channels one by one, a feature fusion module is introduced to adopt point-to-point feature fusion. A layer of 1x1 convolution kernel is added on the basis of two layers of 3x3 convolution kernels to capture feature detail points on the two layers of convolution kernels.
[0013] After the convolution calculation, add the Leaky ReLU function activation. After the convolution calculation, add the Leaky ReLU activation function to perform nonlinear transformation on the quantized output and allow negative gradient input, which increases the nonlinear response capability of the network model and improves the generalization ability of the model.
[0014] The Resnet18 architecture has five stages, each of which downsamples the feature map by a factor of 2, that is, the size of the feature map is halved, the dimension is multiplied by 2, and the downsampling of the last two stages is changed to 1;
[0015] In the last stage of the Resnet18 architecture, a bilinear interpolation and a point-wise convolution are added to restore the resolution of the feature map, and the size of the restored feature map is the same as the output of the third stage;
[0016] Design a dual twin network VGG16 framework and ResNet18 architecture, each layer outputs feature maps of the same size, normal samples and defect samples are used as input, and are input into the improved VGG16 framework and ResNet18 architecture respectively, and the representation of input data in these two networks is learned;
[0017] The twin network constructs a weight-sharing convolution operation and selects a convolution kernel of a fixed size;
[0018] The dual-feature network undergoes convolution calculations of different sizes in sequence, outputting feature maps of different sizes with global detail information, extracting the output of the last three layers of the dual-Luan Sheng network architecture, and defining the features with high-level information in the feature map as global branch nodes.
[0019] Step S4: In order to make the extracted feature map discriminative at different semantic levels, the last three layers of the network architecture are marked as upper, middle and lower global branch nodes. A multi-scale fusion strategy is adopted for the global branch node marked as middle. The mid-level global features are selected as two-part inputs. The three global branch nodes are fused in parallel using the multi-scale fusion strategy. The mid-level global branch nodes are scaled down and expanded in multiple ways, and information interaction is performed between the features containing deep information in the two global branch nodes marked as upper and lower, so as to achieve the purpose of strong semantic information and precise location information.
[0020] The two parts of input are as follows: one part is downsampled to take the maximum value of the feature value in each feature map, and then the feature value of the original area is replaced by the maximum value, which is reduced to the feature map size corresponding to the global branch marked as small;
[0021] The other part uses transposed convolution and a 4x4 convolution kernel to perform translational convolution operations on the feature maps of the intermediate levels, expanding the feature map size corresponding to the large global branch marked as large, providing a more comprehensive and rich visual context for the subsequent fusion of multi-scale branch information.
[0022] Step S5: setting the horizontal segmentation parameters to 1, 2, 4, and 8, and fusing the feature maps output at different levels to adopt horizontal segmentation, horizontally segmenting the deep feature maps generated by different convolutional layers into partition strips of different sizes, and using them for multi-level and multi-scale pooling to segment the feature maps into 1, 2, 4, and 8 strip features of varying sizes;
[0023] Step S6: Take the weighted sum of global average pooling and global maximum pooling for each strip feature, connect the strip features of different scales and semantic levels, and form the final representation of each image:
[0024] V i,j =avgpool(B i,j )+w i maxpool(B i,j )
[0025] Among them, w i denotes learnable parameters and is shared in the same proportion, B i,j Represented as the jth stripe in the i-th scale, V i,jis the feature vector;
[0026] Step S7: Use independent convolution layers to reduce the channel dimension. Independent convolution uses a convolution kernel of size 1x1 to effectively reduce the number of parameters and computational complexity, avoid overfitting, and improve computational efficiency. Each column of feature vectors is independently input into the classifier. The classifier consists of an independent fully connected layer (FC) and a softmax layer. The number of hidden nodes in the last layer is set, and a feature vector of fixed dimension is output.
[0027] Step S8: When normal samples and defective samples are input for training, a loss function is introduced to measure the difference between the output of the entire model and the true label (label). After back-propagation training, the difference between the output of the entire model and the true label is measured. The specific form of the loss function is as follows:
[0028]
[0029] In the formula, λ is set to 1 to keep the same range of the two loss terms, the size of the dense feature set is h×w, and F AEs represents the feature set of normal samples, F AEg represents the feature set of defect samples, F Ds represents the feature set after normal sample reconstruction, F Dg It represents the feature set after defect sample reconstruction, i and j represent the feature values corresponding to a certain bit position.
[0030] Definition L res The purpose is to make the feature set F reconstructed by normal samples AEs and F Ds As close as possible, define L inp The purpose is to make the feature set F after defect sample reconstruction AEg and F Ds To improve segmentation accuracy and computation speed, both losses are defined using a pixel-level L2 loss metric to avoid being unable to distinguish defects and noise well during reasoning, and to guide the twin network to perceive defects through the joint behavior of reconstruction and repair.
[0031] Obtain the weight file after training, perform change detection on the obtained coated cell samples, and splice the detection results to complete the detection of the entire area.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] In actual industrial scenarios, defect samples are often scarce and the labeling cost is high. The twin network can detect abnormal defects through unsupervised learning, using only normal samples for training. The twin network consists of two or more sub-networks with the same structure and shared weights. This structure enables the network to process input data in parallel, allowing the twin network to focus more on extracting semantic information from the image. When faced with complex scenes such as complex textures, complex backgrounds, or functional abnormalities, it can more accurately grasp different defect types and locations.
[0034] The core idea of multi-scale feature fusion is that features of different scales contain different information. The feature maps of deep networks have low resolution and weak geometric information representation capabilities; on the contrary, low-level networks have weak semantic information representation capabilities. By fusing multi-scale features, it is possible to establish connections between features at different levels, effectively integrate global context information, and enable the model to cope with diverse visual scenes and changes. It has stronger adaptability to different defect forms and scenes, which helps reduce false detections and missed detections.
[0035] The corresponding feature maps of normal samples and difference samples are reconstructed and repaired respectively, and the differences between the reconstructed and repaired features and the actual samples are calculated. This can have many advantages in defect detection, such as accurate error capture, robustness to outliers, adaptability to different defect types, and easy optimization and convergence. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 Schematic diagram of the feature fusion architecture based on the dual twin network.
[0037] Figure 2 Schematic diagram of a multi-level and multi-scale horizontal segmentation network.
[0038] Figure 3 Schematic diagram of multi-scale fusion strategy.
[0039] Figure 4 Figure 2 is a sample data analysis and experimental results diagram.
[0040] Figure 5 This is the effect diagram of twin network defect detection. DETAILED DESCRIPTION
[0041] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0042] like Figure 1 As shown in the figure, the coating cell surface abnormal defect detection method based on the double twin network has the following specific steps:
[0043] Step S1: acquiring submicron features on the surface of the lithium battery coating material, with intact surface cells as normal samples and damaged surface cells as defective samples;
[0044] Step S2: Construct a twin network architecture. The two structures are similar, and the twin network architecture establishes a neural network with a coupled architecture with weight sharing. The twin network architecture takes two samples as input, maps the input to a new space and forms a representation of the input in the new space, and outputs a representation embedded in a high-dimensional space;
[0045] Step S3: The dual-feature network is sequentially subjected to convolution calculations of different sizes, outputting feature maps containing global detail information of different sizes, extracting the output of the last three layers of the dual-Luansheng network architecture, and defining the features with high-level information in the feature map as global branch nodes;
[0046] The dual feature network is a dual native network VGG16 framework and ResNet18 architecture, and a similar architecture design is performed on the two original extraction networks. The feature map is calculated by convolution with weight sharing within the same layer. The map is generated by the filter acting on the input data to produce the output of a convolution layer as the input of the next layer. Each filter has a set size corresponding to the local receptive field of a single unit in the layer.
[0047] The five convolution modules before the VGG16 framework are retained. Each convolution module contains two convolution layers and pooling layers to stack the neural network, and the pooling layer operation of the last convolution module is removed. The filter size is 3×3.
[0048] The convolution module removes the downsampling operation and uses transposed convolution in the last convolution module to restore the resolution of the feature map and restore the feature map size to the same as the output of the third convolution module;
[0049] By linearly combining the feature maps of different channels one by one, a feature fusion module is introduced to adopt point-to-point feature fusion. A layer of 1x1 convolution kernel is added on the basis of two layers of 3x3 convolution kernels to capture feature detail points on the two layers of convolution kernels.
[0050] After the convolution calculation, the Leaky ReLU activation function is added to perform nonlinear transformation on the quantized output and allow negative gradient input, which increases the nonlinear response capability of the network model and improves the generalization ability of the model.
[0051] The Resnet18 architecture has a total of five stages. Each stage downsamples the feature map by a factor of 2, which means the size of the feature map is halved and the dimension is multiplied by 2. The downsampling of the last two stages is changed to 1.
[0052] In the last stage of the Resnet18 architecture, a bilinear interpolation and a point-wise convolution are added to restore the resolution of the feature map, and the size of the restored feature map is the same as the output of the third stage;
[0053] Design a dual twin network VGG16 framework and ResNet18 architecture, each layer outputs feature maps of the same size, normal samples and defect samples are used as input, and are input into the improved VGG16 framework and ResNet18 architecture respectively, and the representation of input data in these two networks is learned;
[0054] The twin network constructs a weight-shared convolution operation and selects a convolution kernel of a fixed size. The convolution operation ensures that each pixel has a weight coefficient, but these coefficients are shared by the entire image, greatly reducing the number of parameters in the convolution kernel.
[0055] The dual-feature network undergoes convolution calculations of different sizes in sequence, outputting feature maps of different sizes with global detail information, extracting the output of the last three layers of the dual-Luan Sheng network architecture, and defining the features with high-level information in the feature map as global branch nodes.
[0056] Step S4: Figure 3 As shown in the figure, in order to make the extracted feature map discriminative at different semantic levels, the last three layers of the network architecture are marked as upper, middle and lower global branch nodes. A multi-scale fusion strategy is adopted for the global branch node marked as middle. The mid-level global features are selected as two-part inputs. The three global branch nodes are fused in parallel using the multi-scale fusion strategy, and the mid-level global branch nodes are scaled down and expanded in multiple ways. Information interaction is carried out between the features containing deep information in the two global branch nodes marked as upper and lower, so as to achieve the purpose of strong semantic information and precise location information.
[0057] The two parts of input are as follows: one part is downsampled to take the maximum value of the feature value in each feature map, and then the feature value of the original area is replaced by the maximum value, which is reduced to the feature map size corresponding to the global branch marked as small;
[0058] The other part uses transposed convolution, which adopts a 4x 4 convolution kernel to perform translational convolution operations on the feature maps of the intermediate levels, expanding the feature map size corresponding to the large global branch, and providing a more comprehensive and rich visual context for the subsequent fusion of multi-scale branch information.
[0059] In this process, in order to make the extracted feature maps discriminative at different semantic levels, two local branches are introduced at the intermediate semantic level as fusion branches of the upper and lower branches, which are complementary for learning feature embedding.
[0060] Step S5: Figure 2 As shown, the horizontal segmentation parameters are set to 1, 2, 4, and 8, and the feature maps output from different levels are fused to adopt horizontal segmentation, which horizontally divides the deep feature maps generated by different convolutional layers into partition strips of different sizes, and is used for multi-level and multi-scale pooling to divide the feature maps into 1, 2, 4, and 8 strip features of varying sizes;
[0061] Step S6: Take the weighted sum of global average pooling and global maximum pooling for each strip feature, connect the strip features of different scales and semantic levels, and form the final representation of each image:
[0062] V i,j =avgpool(B i,j )+w i maxpool(B i,j )
[0063] Among them, w i denotes learnable parameters and is shared in the same proportion, B i,j Represented as the jth stripe in the i-th scale, V i,j is the feature vector;
[0064] Both fixed-dimensional vectors of body parts of different sizes can be obtained, and discriminative representations of partitions from global to local, from high-level to mid-level, and from coarse to fine can be captured. In addition, global and local cues are superimposed at four scales, which helps make the final prediction more reliable.
[0065] Step S7: Use independent convolution layers to reduce the channel dimension. Independent convolution uses a convolution kernel of size 1x1 to effectively reduce the number of parameters and computational complexity, avoid overfitting, and improve computational efficiency. Each column of feature vectors is independently input into the classifier. The classifier consists of an independent fully connected layer (FC) and a softmax layer. The number of hidden nodes in the last layer is set, and a feature vector of fixed dimension is output.
[0066] Step S8: When normal samples and defective samples are input for training, a loss function is introduced to measure the difference between the output of the entire model and the true label (label). After back-propagation training, the difference between the output of the entire model and the true label is measured. The specific form of the loss function is as follows:
[0067]
[0068] In the formula, λ is set to 1 to keep the same range of the two loss terms, the size of the dense feature set is h×w, and F AEs represents the feature set of normal samples, F AEg represents the feature set of defect samples, F Ds represents the feature set after normal sample reconstruction, FDg It represents the feature set after defect sample reconstruction, i and j represent the feature values corresponding to a certain bit position.
[0069] Definition L res The purpose is to make the feature set F reconstructed by normal samples AEs and F Ds As close as possible, define L inp The purpose is to make the feature set F after defect sample reconstruction AEg and F Ds To improve segmentation accuracy and computation speed, both losses are defined using a pixel-level L2 loss metric to avoid being unable to distinguish defects and noise well during reasoning, and to guide the twin network to perceive defects through the joint behavior of reconstruction and repair.
[0070] like Figure 4-5 As shown in the figure, 116 test samples are randomly selected from the coated cell surface data set, including 64 normal samples and 52 defective samples, as the test set input. The weight file after training is taken and input into the test set for defect change detection.
[0071] The confusion matrix method is used to perform data analysis for model validation. The relationship between the model prediction results and the actual labels is intuitively displayed in a tabular form, making the evaluation of model performance more intuitive and clear.
[0072] The detection image results are obtained, including the input samples, the image samples after data enhancement, and the sample images after detection. The detection results are stitched together to complete the detection of the entire area.
[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims
1. A method for detecting abnormal defects on the surface of coating cells based on a twin network, characterized in that: The specific steps are as follows: Step S1: acquiring submicron features on the surface of the lithium battery coating material, with intact surface cells as normal samples and damaged surface cells as defective samples; Step S2: construct a twin network architecture. The twin network architecture establishes a neural network with a weight-sharing coupled architecture. The twin network architecture takes two samples as input, maps the input to a new space and forms a representation of the input in the new space, and outputs a representation embedded in a high-dimensional space; Step S3: The dual-feature network is sequentially subjected to convolution calculations of different sizes, outputting feature maps containing global detail information of different sizes, extracting the output of the last three layers of the dual-Luansheng network architecture, and defining the features with high-level information in the feature map as global branch nodes; Step S4: The last three layers of the network architecture are marked as upper, middle and lower global branch nodes. A multi-scale fusion strategy is adopted for the node marked as middle. The middle-level global features are selected as two-part inputs. The three global branch nodes are fused in parallel using the multi-scale fusion strategy. The middle-level global branch nodes are scaled down and expanded in multiple scales, and information is exchanged between the features containing deep information in the two global branch nodes marked as upper and lower. Step S5: setting the horizontal segmentation parameters to 1, 2, 4, and 8, and fusing the feature maps output at different levels to adopt horizontal segmentation, horizontally segmenting the deep feature maps generated by different convolutional layers into partition strips of different sizes, and using them for multi-level and multi-scale pooling to segment the feature maps into 1, 2, 4, and 8 strip features of varying sizes; Step S6: Take the weighted sum of global average pooling and global maximum pooling for each strip feature, connect the strip features of different scales and semantic levels, and form the final representation of each image: V i,j =avgpool(B i,j )+w i *maxpool(B i,j ) Among them, w i denotes learnable parameters and is shared in the same proportion, B i,j Represented as the jth stripe in the i-th scale, V i,j is a feature vector; avgpool represents average pooling calculation, and maxpool represents maximum pooling calculation; Step S7: Use an independent convolution layer to reduce the channel dimension. The independent convolution uses a convolution kernel of size 1x1. Each column of feature vectors is independently input into the classifier. The classifier consists of an independent fully connected layer and a softmax layer. The number of hidden nodes in the last layer is set, and a feature vector of fixed dimension is output. Step S8: Input normal samples and defect samples for training, and introduce a loss function to measure the difference between the output of the entire model and the true label. The specific form of the loss function is as follows: Where, L total is the total loss, L res represents the loss in the reconstruction process, L inp represents the loss in the restoration process, λ is set to 1 to keep the same range of the two loss terms, h and w are the sizes of the dense feature set, and F AEs represents the feature set of normal samples, F AEg represents the feature set of defect samples, F Ds represents the feature set after normal sample reconstruction, F Dg It represents the feature set after defect sample reconstruction, i, j represents the feature value corresponding to a certain bit position.
2. The method for detecting abnormal defects on the surface of coating cells based on a double twin network according to claim 1, characterized in that: The dual feature network is a dual native network VGG16 framework and ResNet18 architecture, and a similar architecture design is performed on the two original extraction networks. The feature map is calculated by convolution with weight sharing within the same layer. The map is generated by the filter acting on the input data to produce the output of a convolution layer as the input of the next layer. Each filter has a set size corresponding to the local receptive field of a single unit in the layer. The five convolution modules before the VGG16 framework are retained. Each convolution module contains two convolution layers and pooling layers to stack the neural network, and the pooling layer operation of the last convolution module is removed. The filter size is 3×3. The convolution module removes the downsampling operation and uses transposed convolution in the last convolution module to restore the resolution of the feature map and restore the feature map size to the same as the output of the third convolution module; By linearly combining the feature maps of different channels one by one, a feature fusion module is introduced to adopt point-to-point feature fusion. A layer of 1x1 convolution kernel is added on the basis of two layers of 3x3 convolution kernels to capture feature detail points on the two layers of convolution kernels. After convolution calculation, add Leaky ReLU activation function, perform nonlinear transformation on quantized output and allow negative gradient input; The Resnet18 architecture has five stages, each of which downsamples the feature map by a factor of 2, that is, the size of the feature map is halved, the dimension is multiplied by 2, and the downsampling of the last two stages is changed to 1; In the last stage of the Resnet18 architecture, a bilinear interpolation and a point-wise convolution are added to restore the resolution of the feature map, and the size of the restored feature map is the same as the output of the third stage; Design a dual twin network VGG16 framework and ResNet18 architecture, each layer outputs feature maps of the same size, normal samples and defect samples are used as input, and are input into the improved VGG16 framework and ResNet18 architecture respectively, and the representation of input data in these two networks is learned; The twin network constructs a weight-sharing convolution operation and selects a convolution kernel of a fixed size; The dual-feature network undergoes convolution calculations of different sizes in sequence, outputting feature maps of different sizes with global detail information, extracting the output of the last three layers of the dual-Luan Sheng network architecture, and defining the features with high-level information in the feature map as global branch nodes.
3. The method for detecting abnormal defects on the surface of coating cells based on a double twin network according to claim 2, characterized in that: In step S4, the two parts of input are specifically: one part is downsampled to take the maximum value of the feature value in each feature map, and then the feature value of the original area is replaced by the maximum value, and the feature map size corresponding to the global branch marked as small is reduced; The other part uses transposed convolution, adopts 4x 4 convolution kernel to perform translation convolution operation on the feature map of the middle level, and expands the feature map size corresponding to the large global branch.
4. The method for detecting abnormal defects on the surface of coating cells based on a double twin network according to claim 3, characterized in that: Obtain the weight file after training, perform change detection on the obtained coated cell samples, and splice the detection results to complete the detection of the entire area.