Shaped part superficial defect detection method based on gray-scale co-production entropy increase

By introducing grayscale coproduction entropy increase technology and main and secondary dual-channel neural networks in the detection of molded parts, the shortcomings of deep learning models in domain drift and feature extraction capabilities are solved, the detection accuracy and robustness are improved, and the industry is adapted to The need for refined subclassification of defects in production.

CN118967636AActive Publication Date: 2024-11-15ZHEJIANG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411091662.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-15
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

The existing deep learning-based shallow defect detection method for molded parts has low accuracy in practical applications, especially the domain drift problem caused by the difference between the training set and the test set, and the difficulty of relying on manual feature extraction. Capture complex patterns and nonlinear relationships, and the generalization ability is limited.

Method used

The superficial defect detection method of forming parts based on grayscale cogeneration entropy increase is adopted, combined with the main and secondary dual-channel alienated neural network, and probability statistical features are generated through grayscale feature extraction and entropy increase processing, and the image is further processed by entropy. , improve the robustness and generalization of the model, and enhance the performance of the classifier through dual interface interaction.

Benefits of technology

Improve the accuracy and speed of detection and positioning, enhance the ability to extract extremely similar defect features, alleviate the problem of insufficient training volume of small samples, adapt to multi-scene applications, and realize the refined subclassification of defects in actual industrial production. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967636A_ABST
    Figure CN118967636A_ABST
Patent Text Reader

Abstract

The invention discloses a method for detecting superficial defects of formed parts based on gray-scale co-production entropy increase. Comprising the following steps: firstly, preprocessing an obtained superficial defect graph of a formed workpiece to form a source domain sample data set, and constructing a training set and a verification set by the source domain sample data set; thirdly, constructing a dissimilatory neural network of a main channel and an auxiliary channel combined with machine learning and priori knowledge, and training by utilizing the training and verification set to obtain a formed workpiece superficial defect detection classifier; and finally, detecting an actually collected superficial defect graph of the formed workpiece by using the detection method. According to the invention, a dual-channel dual-interface interactive dissimilatory neural network is provided, the gray scale co-production entropy increase method enhances the extraction and integration ability of the extremely similar defect features, and the detection accuracy can be improved under the condition of ensuring that the complexity of a classifier model is not increased and the detection speed is not reduced. The classification precision of the anchor frames with the superficial defects of the industrial formed parts under the existing deep learning strategy is improved, and the problem that the precision of small samples is not high is effectively solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for detecting and locating extremely similar defects on the surface and sub-surface of a part product in the field of industrial manufacturing, and in particular to a method for detecting superficial defects of formed parts based on grayscale co-production and entropy increase. Background Art

[0002] Mechanical manufacturing, aerospace, shipbuilding, nuclear power and other fields all require the processing and manufacturing of various metal materials, plastic materials, composite materials, etc. There are many methods for forming industrial parts, including equal material manufacturing, subtractive manufacturing, additive manufacturing, and additive and subtractive manufacturing, etc. Specifically, they can be casting, forging, welding, rolling, injection molding, blow molding, extrusion molding, molten deposition molding, laser additive manufacturing, etc. With the high-end development of the manufacturing industry, the industry has higher and higher requirements for the surface and sub-surface quality of formed parts, such as defect detection for millimeter-level ultra-small sizes and irregular and complex component structures, and defect detection of continuous fiber reinforced composite formed components. However, due to environmental factors or the influence of processing equipment and quality loss, various defects are prone to appear on the surface of the formed parts during the forming production process, such as scratches, spots, micro cracks, pitting, etc., which puts higher requirements on the quality defect detection method of formed parts.

[0003] Traditional defect detection and positioning often use manual detection and positioning methods to perform operations, relying on the detection and positioning capabilities and experience of professional workers, and carefully observing and analyzing the surface of the product formed parts. This method is not only labor-intensive, but also time-consuming. Detection and positioning workers are prone to product visual fatigue due to long-term work, which in turn causes false detection and missed detection of products. With the development of artificial intelligence methods, people have discovered that neural networks have powerful feature extraction capabilities. Many researchers have proposed methods for detecting and positioning superficial defects of formed parts based on deep learning models. The research team of the University of Findlay in the United States (Praveen Damacharla, Achuth Rao MV, Jordan Ringenberg, Ahmad Y Javaid, "TLU-Net: A Deep Learning Approach for Automatic Steel Surface Defect Detection," 2021International Conference on Applied Artificial Intelligence (ICAPAI), Halden, Norway, 2021, pp. 1-6, doi: 10.1109 / ICAPAI49758.2021.9462060.) Based on the basic framework of U-Net, two feature extraction networks, ResNet and DenseNet, are used to perform transfer learning on the NEU-DET steel plate defect dataset produced by Northeastern University in the United States to make up for the shortage of sample sets in the industry. However, since the product images taken in the actual manufacturing process will be affected by various factors and will be quite different from the sample set, this method can effectively alleviate the problems caused by the small number of samples and reduce the huge cost consumption brought by sufficient training sets, but the effect in actual application scenarios will be poor and the accuracy will not be high, that is, there is a problem of small sample domain drift.

[0004] The research team of Tokyo University of Science in Japan (Nagata F, Tokuno K, Mitarai K, et al. Defect detection method using deep revolutional neural network, support vector machine and template matching techniques [J]. Artificial Life and Robotics, 2019, 24 (4): 512-519.) used the gray-level co-occurrence matrix to extract statistical features, combined with feature vector matching such as variance, mean, contrast, and entropy, and support vector machine to implement the defect detection method. However, it relies heavily on manual feature extraction and is difficult to capture the complex patterns and nonlinear relationships hidden in the data. It has certain limitations and limited generalization ability.

[0005] The US patent (Wallingford R, Cong G, Park S. Deep learning based defect detection [P]: US11776108, 2023-10-03) provides a system that can be used for defect detection and positioning, the system comprising multiple components, including a deep learning network model, which is configured to generate a grayscale analog design data map for a location on a sample from a high-resolution map generated at the location by a high-resolution imaging system.

[0006] The above methods have their own characteristics and scope of application. Among the existing defect detection methods based on deep learning, on the one hand, the proposed defect detection method has some differences between the training set and the test set and lacks samples in the sample set, which is the domain drift problem that is prevalent in transfer learning. As a result, the trained model is not good enough in accuracy, and has poor ability to recognize extremely similar defect features, making it difficult to perform; on the other hand, the machine vision detection and positioning method based entirely on manual feature extraction is more dependent on domain knowledge and professional experience, and it is difficult to capture the complex patterns and nonlinear relationships hidden in the data. It has certain limitations and limited generalization ability. Summary of the invention

[0007] In order to solve the problems existing in the background technology, the present invention focuses on the identification of superficial defects of industrial formed parts, provides a detection and positioning method with high accuracy for the detection and positioning of superficial and extremely similar defects of formed parts, and enhances the extraction of extremely similar defect features by the method to alleviate the situation of insufficient training of small samples; the present invention provides a method for detecting superficial defects of formed parts based on grayscale co-production and entropy increase. The present invention combines auxiliary feature extraction technology with deep learning detection and positioning methods, and realizes the controllable function of neural network methods in the field of defect detection and positioning. The main channel model is responsible for defect detection and positioning. However, due to the lack of sample sets in the actual manufacturing environment and the domain drift problem of transfer learning, the accuracy is not high. Therefore, in order to improve the generalization of the model, the model of the secondary channel is additionally introduced. In terms of processing small sample sets, the traditional artificial feature extraction method has a unique a priori advantage. Therefore, the optimization method of the secondary channel is introduced-grayscale co-production entropy increase. Through multiple image processing, the input image is jointly acted and abstracted to generate a series of probabilistic statistical features, including contrast, entropy and other information. Entropy is further used to act and process it to obtain the final entropy increase map of the secondary channel. The obtained feature information is used to provide dual feedback to the model training stage of the main channel in two ways, and dual interface interaction is performed to achieve auxiliary guidance for the classifier model, further improving the robustness and generalization of the model, and finally achieving the performance improvement of the defect detection method. This method can quickly identify possible defects based on the superficial features of the input formed parts, generate anchor boxes with corresponding confidence, and improve the detection and positioning accuracy of the existing method model while ensuring fast detection and positioning speed. It has adaptability to multi-scenario applications. Based on this method, the model can be further optimized so that each defect can be refined into a sub-classification domain to meet the needs of specific defect segmentation in actual industrial production.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0009] 1. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase

[0010] Step 1: After preprocessing the superficial defect features of the formed parts, a source domain sample data set is formed, and then a training set and a validation set are constructed from it;

[0011] Step 2: Construct an alienated neural network based on the main and auxiliary dual channels;

[0012] Step 3: Use the training set to train the alienated neural network based on the main and auxiliary dual channels to obtain the trained alienated neural network, and then use the validation set to fine-tune the hyperparameters of the trained alienated neural network to obtain a superficial defect detection model for formed parts;

[0013] Step 4: Input the actually collected superficial defect features of the formed part into the superficial defect detection model of the formed part, and the model outputs the corresponding defect detection results.

[0014] In the step 1, the preprocessing of the superficial defect features of the formed part includes removing invalid areas, adjusting the size, initializing and normalizing the values, adjusting the brightness and enhancing the data.

[0015] In the step three, the alienated neural network of the main and sub-channels includes a main channel part and a sub-channel part. The main channel part includes a feature extraction module, an information hierarchy fusion module and a detection and positioning module connected in sequence. The input of the alienated neural network based on the main and sub-channels is used as the input of the feature extraction module. The sub-channel part includes a grayscale level reduction layer, a grayscale feature extraction module, an edge detection module, a hysteresis threshold segmentation module and a defect feature map generation module connected in sequence. The input of the alienated neural network based on the main and sub-channels is also used as the input of the grayscale level reduction layer. The grayscale feature extraction module is connected to the detection and positioning module, and the defect feature map generation module is connected to the deepest feature extraction module in the feature extraction module.

[0016] In the grayscale reduction layer, the grayscale level of the superficial defects of the formed part is reduced without losing the texture information, and a grayscale image after the grayscale level is reduced is obtained.

[0017] In the grayscale feature extraction module, a grayscale co-occurrence matrix and a grayscale histogram are generated according to the grayscale image after the grayscale level is reduced, features are extracted based on the grayscale co-occurrence matrix and the grayscale histogram respectively, a number of features are selected from the extracted features as grayscale features, a feature vector is composed of multiple grayscale features and used as the input of a classification model with the feature vector as the input, a classification result is obtained and used as a probabilistic posteriori for a detection and positioning module; and the grayscale image after the grayscale level is reduced is subjected to entropy increase sampling processing using the entropy in the grayscale features to obtain a corresponding grayscale feature map and use it as the input of an edge detection module.

[0018] In the edge detection module, a gradient detection method is used to extract edges in the vertical and horizontal directions of the grayscale feature map after entropy increase processing, and the vertical direction map and the horizontal direction map corresponding to all the grayscale feature maps are obtained. Then, the average values ​​of the vertical direction map and the horizontal direction map corresponding to the grayscale feature map are taken and merged into one map and recorded as the edge feature map, thereby obtaining the edge feature map.

[0019] In the hysteresis threshold segmentation module, after performing hysteresis threshold segmentation on the obtained edge feature map, the corresponding segmented edge feature map is obtained.

[0020] In the defect feature map generation module, the segmented edge feature map is eroded and expanded using a morphological closing operation method to obtain a corresponding defect feature map and use it as the input of the deepest feature extraction module in the feature extraction module.

[0021] The main channel part is specifically a one-stage defect detector, which transforms the depth-separable convolution block in the feature extraction module of the backbone network in the detector into a depth-separable attention convolution block, transforms the activation function of the convolution layer of the detector into a non-monotonic gradient optimization activation function, and transforms the loss function of the optimizer into a new function that incorporates the direction vector angle loss.

[0022] The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of a method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase are implemented.

[0023] 3. A computer-readable storage medium

[0024] A computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the steps of a method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase are implemented.

[0025] IV. A COMPUTER PROGRAM PRODUCT

[0026] The computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of a method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase.

[0027] Therefore, the present invention introduces a targeted primary and secondary dual-channel alienated neural network, which enhances the ability to extract and integrate extremely similar defect features, integrates traditional features and deep learning features, can better capture the global and local information of feature maps, improve the accuracy and robustness of feature recognition and analysis, increase the controllability of the overall method, and effectively alleviate the domain drift problem of small samples.

[0028] The present invention has the following beneficial effects:

[0029] 1. The method for detecting superficial defects of formed parts based on grayscale co-production and entropy increase proposed in the present invention can correct nonlinear distortion of features in the preprocessing stage for problems such as nonlinear distortion, barrel distortion, uneven illumination, and high metal reflectivity that occur in special industrial scenes such as endoscope images. It can directly detect and locate defects such as cracks, spots, and inclusions that may occur in actual industrial manufacturing, and predict the confidence level. It can improve the detection and positioning accuracy to a certain extent while ensuring fast detection and positioning speed and lightweight models. In addition, for extremely similar features that are difficult to identify in the general classification of extremely similar defect anchor frames, the feature extraction network architecture is adjusted in a targeted manner, thereby enhancing the method's ability to extract and integrate extremely similar defect features and effectively alleviating domain drift. Based on this method, the model can be further optimized so that each defect can obtain a refined sub-classification domain to meet the personalized needs of specific defect type segmentation in actual industrial production;

[0030] 2. The present invention adds a feature classification module of a secondary channel on the basis of the original deep learning model. The feature extraction technology of the module extracts the corresponding features, which can be fused and combined with the features extracted by the deep learning model to enhance the diversity and characterization ability of the features. The obtained classification results can provide a posteriori guidance for the classification of the deep learning model. The traditional features and deep learning features are integrated through the grayscale co-production and entropy increase dual interfaces, which can better capture the global and local information of the feature map, improve the accuracy and robustness of feature recognition and analysis, and solve the "black box" problem of deep learning. It also increases the controllability of the overall identification method, can adjust the auxiliary feature extraction method according to different working scenarios, and then perform an organic interaction of assistance and guidance on the feature extraction of the main channel, and finally achieve the performance improvement of the defect detection method.

[0031] 3. The present invention uses a convolutional layer and a converter interactive fusion module in the feature extraction network structure, which enhances the ability to integrate contextual features and improves the model's ability to detect and locate defects in actual applications with complex backgrounds and unclear features. By transforming the original activation function in the convolution module into a non-monotonic gradient optimization activation function, the generalization ability of the overall model is enhanced and the accuracy is improved. The original loss function of the optimizer is transformed into a new function that incorporates the directional vector angle loss, and an angle penalty cost function is added, so that the anchor frame predicted by the trained model is more in line with the real frame of the feature map, and the classification results of very similar defect anchor frames are more in line with actual working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The present invention is a schematic flow chart of the method for detecting superficial defects of formed parts based on grayscale co-production and entropy increase.

[0033] Figure 2Schematic diagram of the alienated neural network architecture of the present invention.

[0034] Figure 3 Schematic diagram of feature pyramid in feature fusion of the present invention.

[0035] Figure 4 This is a diagram of the converter module architecture introduced in the present invention.

[0036] Figure 5 This is a graph showing the experimental results of calculating the grayscale histogram of the c4 feature map according to the present invention.

[0037] Figure 6 The original image and the feature image are obtained after the grayscale co-production and entropy increase processing is performed on a c3 feature image by the method of the present invention.

[0038] Figure 7 It is a curve diagram of various loss data in the training process of the model of the present invention.

[0039] Figure 8 It is a curve diagram of various loss data in the verification process of the model of the present invention.

[0040] Fig. 9 This is a diagram showing the anchor frame detection and positioning results of superficial defects in a formed part without using the method of the present invention during the model verification process.

[0041] Fig.10 This is a diagram showing the anchor frame detection and positioning results of shallow defects in formed parts using the method of the present invention during model verification.

[0042] Fig.11 and is the confusion matrix obtained without using the method of the present invention for training.

[0043] Fig.12 It is the confusion matrix obtained after training with the model of the present invention.

[0044] Fig.13 It is the F1 curve diagram of each category of the experimental results of the model simulation of the present invention.

[0045] Fig.14 It is a curve diagram of the accuracy of each category of the experimental results of the model simulation of the present invention.

[0046] Fig.15 It is a recall rate curve diagram of each category of the experimental results of the model simulation of the present invention.

[0047] Fig.16 It is a precision-recall curve diagram of each category of the experimental results of the model simulation of the present invention.

[0048] Fig.17 It is the accuracy curve of the model verification process results of not using the method of the present invention and the method of the present invention.

[0049] Fig.18 It is the recall rate curve of the model verification process results of not using the method of the present invention and the method of the present invention.

[0050] Fig.19 It is the mAP_0.5 curve of the model verification process results of not using the method of the present invention and the method of the present invention.

[0051] Fig. 20 It is the mAP_0.5-0.95 curve of the model verification process results of not using the method of the present invention and the method of the present invention.

[0052] Fig.21 It is a schematic diagram comparing the defect detection effects of the method adopted by the present invention and the classical method. DETAILED DESCRIPTION

[0053] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are only used to explain the present invention rather than to limit the present invention.

[0054] like Figure 1 As shown, the method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase proposed by the present invention comprises the following steps:

[0055] Step 1: Collect surface images of defective formed parts in actual working conditions. The data set used in the present invention has a total of six types of defects in formed parts with an equal number of images, including crazing, inclusion, patches, pitted surface, rolled-in scale, and scratches. In the following description, these six defects are referred to as c1, c2, c3, c4, c5, and c6, respectively. The data set is screened and preprocessed, and the poor feature maps (i.e., invalid maps) are eliminated. The invalid maps do not contain six common manufacturing defects. The preprocessing measures include resizing, initializing and normalizing the values, adjusting the brightness, and data enhancement. Then, the feature map processed in step 1 is annotated with the real frame using the graphic annotation technology, the defect area is marked, and the defect name is written. The training set, verification set, and test set can be divided according to the ratio of 8:1:1.

[0056] Step 2: Constructing a dissimilated neural network based on the main and auxiliary dual channels; optimizing the structure of the dissimilated neural network based on the main and auxiliary dual channels. The model constructed by the present invention is as follows: Figure 2As shown. The alienated neural network of the main and secondary channels includes the main channel part and the secondary channel part. The main channel model is responsible for defect detection and positioning. However, due to the lack of sample sets in the actual manufacturing environment and the domain drift problem of transfer learning, the accuracy is not high. Therefore, in order to improve the generalization of the classifier model, the secondary channel model is additionally introduced. In terms of processing small sample sets, the traditional artificial feature extraction method has a unique a priori advantage. Therefore, the optimization method of the secondary channel is introduced-grayscale co-production entropy increase for processing. The obtained feature information is positively fed back to the model training stage of the main channel in two ways, realizing the auxiliary guidance role of the model, further improving the robustness and generalization of the model, and finally achieving the performance improvement of the defect detection method.

[0057] The main channel part includes a feature extraction module, an information level fusion module, and a detection and positioning module, which are connected in sequence. The input of the alienated neural network based on the main and secondary dual channels is used as the input of the feature extraction module. The feature extraction part is used to perform multiple downsampling of the input batch images using the convolution layer to continuously extract the image features; the information level fusion part is used to perform multi-scale information fusion of features, and the deep feature information and shallow information are mutually transmitted and fused through upsampling. The detection and positioning part is used to predict the main channel network model, using the non-maximum suppression method to screen out the bounding box with the highest confidence, and output the defect type, confidence and position vector predicted for it.

[0058] The sub-channel part includes a grayscale reduction layer, a grayscale feature extraction module, an edge detection module, a hysteresis threshold segmentation module and a defect feature map generation module which are connected in sequence. The input of the alienated neural network based on the main and sub-channels is also used as the input of the grayscale reduction layer. The grayscale feature extraction module is connected to the detection and positioning module. The defect feature map generation module is connected to the deepest feature extraction module in the feature extraction module, which is used to fuse the defect feature map and the deepest feature.

[0059] The sub-channel part includes a grayscale reduction layer, a grayscale feature extraction module, an edge detection module, a hysteresis threshold segmentation module and a defect feature map generation module which are connected in sequence. The input of the alienated neural network based on the main and sub-channels is also used as the input of the grayscale reduction layer. The grayscale feature extraction module is connected to the detection and positioning module. The defect feature map generation module is connected to the deepest feature extraction module in the feature extraction module, which is used to fuse the defect feature map and the deepest feature.

[0060] In the grayscale reduction layer, the grayscale of the superficial defect image of the formed part is reduced without losing the texture information, and the grayscale image after the grayscale level is reduced is obtained. If the superficial defect image of the formed part is an RGB image, it is also grayscaled first to make it a grayscale image, and then the grayscale level is reduced. The goal of this step is to reduce the grayscale of the image without losing the texture information. This is to reduce the calculation time of the Gray-Level Co-occurrence Matrix (GLCM). Reducing the grayscale level involves reducing the size of the GLCM;

[0061] In the grayscale feature extraction module, the grayscale co-occurrence matrix and grayscale histogram are generated according to the grayscale image after the grayscale level is reduced. Features are extracted based on the grayscale co-occurrence matrix and grayscale histogram. The extractable features include mean, variance, energy, contrast, homogeneity, homogeneity, and entropy. From the extracted features, a number of appropriate features are selected as grayscale features according to the actual conditions. A feature vector is composed of multiple grayscale features and used as the input of the classification model (such as SVM, BP neural network) with the feature vector as the input. The classification result is obtained and used as the probabilistic posterior of the detection and positioning module; and the entropy in the grayscale feature is used to perform entropy increase sampling on the grayscale image after the grayscale level is reduced to obtain the corresponding grayscale feature map and use it as one of the inputs of the edge detection module. In this step, the image is covered with an analysis window. The size of the analysis window is T and the offset is d. The goal is to calculate the GLCM of each analysis window and extract the Haralick feature-entropy. The feature will be assigned to the center point of the analysis window. Finally, the output image is obtained.

[0062] In the edge detection module, the gradient detection method is used to extract the edges in the vertical and horizontal directions of the grayscale feature map after entropy increase processing, and the vertical direction map and horizontal direction map corresponding to all the grayscale feature maps are obtained. Then, the average value of the vertical direction map and the horizontal direction map corresponding to the grayscale feature map is taken and the two maps are merged into one map and recorded as the edge feature map, thereby obtaining the edge feature map.

[0063] In the hysteresis threshold segmentation module, the obtained edge feature maps are subjected to hysteresis threshold segmentation respectively to obtain the corresponding segmented edge feature maps. Unlike the standard threshold segmentation method, the hysteresis threshold segmentation is not equal at all points of the graph. The goal is to keep the strongest edges of the graph and maintain their continuity. First, two thresholds are used, a high threshold and a low threshold. The high threshold is used to detect and locate the strongest edges of the graph. On the other hand, the low threshold can highlight the weak edges of the graph. Each weak edge is conserved only when its neighborhood is a strong edge detected and located by the high threshold.

[0064] In the defect feature map generation module, the morphological closing operation method is used to erode and dilate all the segmented edge feature maps, and the corresponding defect feature map is obtained and used as one of the inputs of the deepest feature extraction module in the feature extraction module. The morphological closing operation is used to solve the edge discontinuity problem that may be caused by the previous step. The morphological closing operator is an important operator in the field of mathematical morphology. It can be derived from the basic operations of erosion and dilation and applied to binary images. Closure tends to expand the boundaries of bright areas in the image. It is defined as dilation and erosion using the same structural element in both operations.

[0065] By fusing traditional features with deep learning features, the present invention can better capture the global and local information of the image, improve the accuracy and robustness of image recognition and analysis, while also solving the "black box" problem of deep learning, increasing the controllability of the overall algorithm, and being able to adjust the auxiliary feature extraction method according to different working scenarios, thereby organically interacting the assistance and guidance of the feature extraction of the main channel, ultimately achieving improved performance of the defect detection method.

[0066] A c3 defect image is processed by the grayscale co-production entropy increase method as shown below: Figure 6 As shown, Figure 6 (a) is the original image, Figure 6 (b) is the feature map after grayscale co-production and entropy increase processing. After processing, the edge significance of the defect is improved. The large black area in the central area of ​​the image is the defect separated from the image, which corresponds to the defect in the center of the original image. There are fewer noise and artifacts introduced, and there are some false areas but the impact is relatively small. The edge of the defect is mostly strong edge, and a small part is weak edge. It has a certain degree of coherence, and the defect features are better separated, which can further guide the training of the deep learning model.

[0067] like Figure 5 As shown, the grayscale histogram of the c4 image of the six defects is calculated to obtain the grayscale histogram, and the Peak-Guass probability density function is used for curve fitting to obtain the fitting curve in the figure. The parameters of the curve are offset 1.68538, peak center position 151.7681, peak width 47.42994, peak area 39637.90768, half-height width 55.84449, and peak height 666.80403. Through further analysis, the characteristic statistics of the image are obtained as grayscale mean 151.43, grayscale variance 584.11053, energy 39.50574, entropy 6.636535, and contrast 214. According to the same method, the grayscale feature extraction method of grayscale histogram and grayscale symbiotic entropy increase is used to extract the grayscale features of the six defects to obtain a series of feature vectors and then perform feature fusion. After the fused features are put into the support vector machine classifier for training, a classification model dominated by grayscale features can be obtained;

[0068] The main channel part is specifically a one-stage defect detector, which is mainly composed of several network layers, namely convolution layer, depth-separable convolution layer, upsampling layer, information fusion layer, etc. In the feature extraction stage of feature perception fusion, the convolution layer is combined with the depth-separable convolution layer to fully downsample the image and extract specific features. In the final stage, the present invention uses a convolution layer and a converter interactive fusion module. This module introduces a converter module under the condition of adding a small amount of parameters, collects and integrates the residual features to a certain extent, obtains the feature information of the image in a higher dimension, and expands the receptive field of each pixel of the final feature map of the entire model to improve the accuracy of the final model. In the feature perception fusion stage, the following are used: Figure 3 The feature pyramid model is used to perform hierarchical fusion of feature information, and in the last module, the prediction module, feature maps of different sizes are further convolved to extract features and then predict various data.

[0069] The convolutional layer in the feature extraction module of the main channel network model and the converter interactive fusion module, which combines the converter, enhances the ability to extract and integrate very similar defect features, enabling the model to quickly perform hierarchical fusion of features, that is, to collect and integrate residual features to a certain extent, obtain feature information of the graph in a higher dimension, expand the receptive field of each pixel in the final feature map of the entire model, and enhance the ability to extract and integrate very similar defect features through feature perception fusion, effectively alleviating the problem of domain drift. Figure 4As shown in the figure, the converter contains four submodules: encoder, decoder, residual connection and normalization, and trigonometric function position encoder. The encoder module contains 6 encoder modules, each of which contains two submodules, namely, multi-head self-attention layer and feedforward fully connected layer. The multi-head self-attention layer uses a scaling dot product attention calculation method. Experimental results show that multi-head can extract features of different heads at a more detailed level, which is better than a single head in extracting features. The feedforward fully connected layer is composed of two fully connected layers. An activation function is added in the middle of the linear transformation. The specific dimension adopts a 4-fold relationship, that is, the dimension of multi-head self-attention is 512, and the transformation dimension within the layer is 2048. The decoder module contains 6 decoder modules, each of which contains 3 submodules, namely, multi-head self-attention layer, encoder-decoder attention layer, and feedforward fully connected layer. The multi-head self-attention layer uses the same scaling dot product attention calculation method as the encoder module. The biggest difference is that a forward mask needs to be added, that is, "future information" is concealed. The main difference between the encoder-decoder attention layer and the previous multi-head self-attention layer is that Q≠K=V. The matrix Q comes from the output of the decoder module of the previous layer, while K and V come from the output of the encoder. The feedforward fully connected layer is exactly the same as the encoder; the residual connection and standardization module are connected to each encoder module and each sub-layer in the decoder module. The role of the residual connection is to transmit information deeper without loss, so as to enhance the fitting ability of the entire model to the data. Standardization is a layer-level numerical standardization operation, which is used to prevent the parameters from being too large or too small, which will cause abnormalities in the training process and slow loss convergence.

[0070] Figure 3 The figure is a schematic diagram of the feature pyramid in the neural network of the present invention. Feature Pyramid Network (FPN) is a classification detection and positioning method proposed to solve the problem of incompatibility between feature size and target to be detected. When the target size is small and the background is large, it means that in the process of layer-by-layer feature extraction, the feature map size is becoming more and more streamlined, and the receptive field will feel more parts of the background. If the network is deeper, the background features are more obvious in the continuous convolution, and small targets may even lose feature information and be mistaken for being part of the background. Conversely, if it is a large target on the feature map of a small background, a small convolution kernel will increase the amount of calculation and improve the calculation time. FPN can detect and locate simple, large targets in the shallow layer, and complex, small targets in the deep layer.

[0071] Figure 4This is the architecture diagram of the converter module introduced in the present invention. The input embedding map is position-encoded and then sent to the regional module. In this regional module, it is first processed using the multi-head attention mechanism, and the obtained output is residually connected with the unprocessed data and then normalized. The normalized output is introduced into the feedback mechanism and residually connected and normalized with the data before the introduction. The operation of this regional module is repeated N times and finally output.

[0072] Step 3: Use the training set to train the alienated neural network based on the main and auxiliary dual channels to obtain the trained alienated neural network, and then use the validation set to fine-tune the hyperparameters of the trained alienated neural network to obtain a superficial defect detection model for formed parts;

[0073] Specifically, the training set is used as the input of the alienated neural network based on the main and secondary dual channels, and the model is trained to obtain the weight file and mAP_0.5 / % and mAP_0.5:0.95 / % after training. The basic training parameters of the present invention are: the graph size is set to 640*640, the batch size is set to 16, the number of iterations is set to 200, and the optimizer is the stochastic gradient descent optimizer (SGD), and the hyperparameters use the default parameters.

[0074] The present invention transforms the original activation function of each convolution module into a non-monotonic gradient optimized activation function. The calculation formula of this activation function f(x) is as follows:

[0075] f(x)=x*tanh(ln(1+e))

[0076] Where x is the input matrix and e is the natural logarithm.

[0077] The convolution module is generally composed of three parts: a two-dimensional convolution layer, batch normalization, and an activation function. The transition of the non-monotonic gradient optimized activation function is relatively smooth, and it has a higher tolerance for slight negative values, which further enhances the stability of the overall method model process, improves the robustness and generalization of the trained model, and is conducive to improving the accuracy of the classification of very similar defect anchor boxes.

[0078] The optimizer's loss function is transformed from the original function to a new loss function that incorporates the direction vector angle loss. The function calculation formula is as follows:

[0079]

[0080]

[0081]

[0082]

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089] Among them, Λ is the angle loss, x, y are the position coordinates, Δ is the distance loss, Ω is the shape loss, t is the symbol of the cumulative parameter, γ is the angle loss variant, L box is the anchor box loss, θ is the degree of attention of the shape loss, IoU is the intersection over union loss, c h is the height difference between the center point of the real box and the predicted box, α is the angle between the center point of the real box and the predicted box, σ is the distance between the center point of the real box and the predicted box, is the horizontal coordinate of the center of the real frame, is the horizontal coordinate of the center of the prediction box, is the ordinate of the center of the real frame, is the vertical coordinate of the center of the prediction box, ρ x is the connection coefficient between the horizontal axis and the width difference, c w is the width of the minimum bounding rectangle of the real box and the predicted box, c h is the height of the minimum bounding rectangle of the real box and the predicted box, ρ y is the connection coefficient between the ordinate and the height difference, ω w is the ratio of the difference between the width of the real box and the predicted box to the maximum value of the two, ω h is the ratio of the height difference between the real box and the predicted box to the maximum value of the two, w is the width of the predicted box, | | is the absolute value symbol, w gt is the real box width, h is the predicted box height, h gt is the height of the real box, B is the area of ​​the predicted box, and B GT is the area of ​​the actual frame.

[0090] Figure 7 This is a curve diagram of various loss data in the training process of the model of the present invention. It includes the classification loss of training, calculating whether the anchor box and the corresponding calibration classification are correct, the positioning loss of training, the error between the prediction box and the calibration box, the confidence loss of training, and the confidence of the calculation network. In the positioning loss diagram, Figure 7The maximum slope of the fitting curve in (a) is -0.000173, the minimum slope is -0.0037, the average slope is -0.00062, and the average slope angle is 151.26°; in the confidence loss graph, Figure 7 The maximum slope of the fitting curve in (b) is -0.00004286, the minimum slope is -0.0034, the average slope is -0.000103, and the average slope angle is 138.75°; in the classification loss graph, Figure 7 The maximum slope of the fitting curve in (c) is -0.00008333, the minimum slope is -0.00144, the average slope is -0.000302, and the average slope angle is 177.69°.

[0091] Figure 8 This is a curve diagram of various loss data in the process of model verification of the present invention. It includes the classification loss of verification, the calculation of whether the anchor box and the corresponding calibration classification are correct, the positioning loss of verification, the error between the predicted box and the calibration box, the confidence loss of verification, and the confidence of the calculation network. In the positioning loss diagram, Figure 8 The maximum slope of the fitting curve in (a) is -0.000014815, the minimum slope is -0.0036, the average slope is -0.0062, and the average slope angle is 169.21°; in the confidence loss graph, Figure 8 The maximum slope of the fitting curve in (b) is -0.00010769, the minimum slope is -0.00123077, the average slope is 0.000103, and the average slope angle is 19.29°; in the classification loss graph, Figure 8 The maximum slope of the fitting curve in (c) is -0.00003846, the minimum slope is -0.00175, the average slope is -0.00312, and the average slope angle is -176.38°;

[0092] Step 4: Input the actually collected shallow defect image of the formed part into the shallow defect detection model of the formed part, and the model outputs the corresponding defect detection result, which includes the classification result, anchor frame position and confidence.

[0093] Fig. 9 and Fig.10This is the result of anchor frame detection and positioning of superficial and extremely similar defects of formed parts in the verification process of the present invention without using the method of the present invention and the method of the present invention. As shown in the figure, the upper left corner of each picture is the source label of the test picture, and the middle box is the anchor frame generated by the method model. The anchor frame contains the picture classification, the center point position of the anchor frame and the normalized data of width and height, and the confidence information. Comparing the two pictures, the training label of the first picture in the upper left corner is c1, and no defects were identified without using the method of the present invention, while the method of the present invention identified the c1 defect and gave a confidence of 0.3; the training label of the second picture is c5, and three defects were identified without using the method of the present invention, but two of them were obviously overlapped, and the method of the present invention did not have this problem; in the fourth picture, only one defect of c4 was identified without using the method of the present invention, but the method of the present invention identified three c4 defects with higher accuracy; through the comparison of these 16 verification pictures, it can be found that the performance of the model trained by the method of the present invention on the verification set is better than that without using the method of the present invention.

[0094] Fig.11 and Fig.12 The confusion matrices obtained after training with and without the method of the present invention and the model of the present invention are respectively a two-dimensional table, which is commonly used to evaluate the performance of classification models. In the confusion entropy increase, each column represents the predicted value and each row represents the true value. Therefore, each element in the confusion matrix represents the number of times a sample is predicted to be a certain category. Each element in the figure is also normalized. It can be seen from the figure that the confusion matrix of the model not trained by the method of the present invention is that in the sample with the true value of c1, 0.64 is c1 and 0.36 is background; in the sample with the true value of c2, 0.60 is c2 and 0.40 is background; in the sample with the true value of c3, 0.93 is the true value and 0.07 is background; in the sample with the true value of c4, 0.89 is c4 and 0.11 is background; in the sample with the true value of c5, 0.87 is c5 and 0.12 is background; in the sample with the true value of c6, 1.00 is c6; in the confusion matrix of the model trained by the method of the present invention, in the sample with the true value of c1, 0.82 is c1 , 0.18 is the background; in the sample with the true value of c2, 0.70 is c2, 0.20 is the background, and 0.10 is c3; in the sample with the true value of c3, 0.93 is c3 and 0.07 is the background; in the sample with the true value of c4, 0.89 is c4 and 0.11 is the background; in the sample with the true value of c5, 0.87 is c5 and 0.12 is the background; in the sample with the true value of c6, 0.90 is c6 and 0.10 is the background; it can be seen that compared with the case where the method of the present invention is not adopted, the model trained by the method of the present invention has significantly improved detection and positioning effects on c1 and c4 defects, and slightly decreased effects on c6 defect detection and positioning. Overall, the performance of the model has been greatly improved.

[0095] Fig.13 The F1 curves of various categories of the experimental results of the model simulation of the present invention. The indicator F1 metric is the harmonic mean of the precision rate and the recall rate, in which each category curve shows a trend of first increasing and then decreasing as the horizontal axis confidence increases. The C1 curve reaches a maximum value of 0.74 when the confidence is 0.36, the C2 curve reaches a maximum value of 0.80 when the confidence is 0.22, the C3 curve reaches a maximum value of 0.83 when the confidence is 0.56, the C4 curve reaches a maximum value of 0.89 when the confidence is 0.18, the C5 curve reaches a maximum value of 0.78 when the confidence is 0.48, and the C6 curve reaches a maximum value of 0.93 when the confidence is 0.50. It can be seen that the confidence of the method of the present invention for the two types of defects c1 and c2 is relatively low, and the difficulty of identifying the two defects is relatively large, which corresponds to the confusion matrix results.

[0096] Fig.14 It is a P (precision) curve diagram of each category of the model simulation experiment results of the present invention. As can be seen from the figure, the precision curves of each category are basically on the rise. The C1 precision curve reaches a maximum value of 1.0 when the confidence is 0.59, the C2 precision curve reaches a maximum value of 1.0 when the confidence is 0.82, the C3 precision curve reaches a maximum value of 1.0 when the confidence is 0.84, the C4 precision curve reaches a maximum value of 1.0 when the confidence is 0.43, the C5 precision curve reaches a maximum value of 1.0 when the confidence is 0.72, and the C6 precision curve reaches a maximum value of 1.0 when the confidence is 0.59.

[0097] Fig.15 The R (recall rate) curve of each category of the simulation experiment results of the model of the present invention is shown in the figure. As can be seen from the figure, the recall rate curves of each category basically show a continuous downward trend. The c1 recall rate curve reaches a minimum value of 0.0 when the confidence level is 0.70, the c2 recall rate curve reaches a minimum value of 0.0 when the confidence level is 0.81, the c3 recall rate curve reaches a minimum value of 0.0 when the confidence level is 0.90, the c4 recall rate curve reaches a minimum value of 0.0 when the confidence level is 0.88, the c5 recall rate curve reaches a minimum value of 0.0 when the confidence level is 0.79, and the c6 recall rate curve reaches a minimum value of 0.0 when the confidence level is 0.85.

[0098] Fig.16The PR curves of various categories of the experimental results of the model simulation of the present invention are shown in Figure 1. The area under the curve of each category represents the current AP_0.5 value of each category, where 0.5 represents an IoU value of 0.5. As can be seen from the figure, the AP_0.5 value of c1 is 0.687, the AP_0.5 value of c2 is 0.708, the AP_0.5 value of c3 is 0.929, the AP_0.5 value of c4 is 0.963, the AP_0.5 value of c5 is 0.829, and the AP_0.5 value of c2 is 0.977. From the results, it can be seen that the performance of the method of the present invention in identifying defects of types c1 and c2 is relatively poor, and it is more difficult.

[0099] Fig.17 The accuracy curve of the model verification process results of the method of the present invention and the method of the present invention is shown in Figure 1. The accuracy of the method of the present invention fluctuates around 0.64 after 20 iterations. The accuracy after 10 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.65121. The accuracy of the method of the present invention fluctuates around 0.75 after 10 iterations. The accuracy after 10 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.73654. It can be seen that the accuracy of the verification process of the method of the present invention is higher than that of the method of the present invention, and the detection and positioning performance of the model is better.

[0100] Fig.18 The recall rate curves of the model verification process results of the method of the present invention and the method of the present invention are shown in Figure 1. The recall rate of the method of the present invention fluctuates around 0.70 after 15 iterations. The recall rates after 15 iterations are averaged and a straight line is drawn to represent the recall rate. The ordinate of the straight line is 0.71021. The recall rate of the method of the present invention fluctuates around 0.78 after 15 iterations. The recall rates after 15 iterations are averaged and a straight line is drawn to represent the recall rate. The ordinate of the straight line is 0.77985. It can be seen that the overall recall rate of the verification process of the method of the present invention is higher than that of the method of the present invention, and the recall performance of the model is better.

[0101] Fig.19The mAP_0.5 curves of the model verification process results of the method of the present invention and the method of the present invention are shown in Figure 1. The mAP_0.5 of the method of the present invention fluctuates around 0.70 after 15 iterations. The mAP_0.5 after 15 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.70861. The mAP_0.5 of the method of the present invention fluctuates around 0.80 after 15 iterations. The mAP_0.5 after 15 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.79894. It can be seen that the mAP_0.5 of the method of the present invention is higher than that of the method of the present invention during the verification process. The average classification accuracy of the model is higher when the IoU threshold is set to 0.5.

[0102] Fig. 20 The mAP_0.5-0.95 curves of the model verification process results of the method of the present invention and the method of the present invention are shown in Figure 1. The full name of mAP is (mean average precision), that is, the average precision. The mAP_0.5-0.95 of the method of the present invention fluctuates around 0.37 after 17 iterations. The mAP_0.5-0.95 after 17 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.38162. The mAP_0.5-0.95 of the method of the present invention fluctuates around 0.44 after 17 iterations. The mAP_0.5-0.95 after 17 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.44984. It can be seen that the mAP_0.5-0.95 of the method of the present invention is higher than that of the method of the present invention in the verification process. The average classification accuracy of the model is higher when the IoU threshold is set to 0.5 to 0.95.

[0103] A comparative experiment was conducted on the alienated neural network based on the main and auxiliary dual channels and the model without the method of the present invention. The results are as follows: Fig.21 As shown. mAP_0.5 means that when IoU is set to 0.5, the AP of all images of each category is calculated, and then all categories are averaged, that is, mAP. mAP_0.5:0.95 represents the average mAP at different IoU thresholds (from 0.5 to 0.95, step size 0.05) (0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95). As can be seen from the figure, the dual-channel forming part superficial defect detection and positioning model based on grayscale co-production entropy increase has improved these two indicators by 5.8% and 6.6% respectively compared with the model that does not adopt the method of this patent. It can be seen that the forming part superficial defect detection and positioning model based on grayscale co-production entropy increase proposed by the present invention has a better effect in terms of accuracy, meets the requirements for superficial defect detection and positioning of industrial formed parts, and is conducive to practical deployment and application in industry.

[0104] The above specific implementation modes are used to explain the present invention rather than to limit the present invention. Any modification and change made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase, characterized in that: The following steps are involved: Step 1: After preprocessing multiple superficial defect images of formed parts, a source domain sample data set is formed, and then a training set and a validation set are constructed from them; Step 2: Construct an alienated neural network based on the main and auxiliary dual channels; Step 3: Use the training set to train the alienated neural network based on the main and auxiliary dual channels to obtain the trained alienated neural network, and then use the validation set to fine-tune the hyperparameters of the trained alienated neural network to obtain a superficial defect detection model for formed parts; Step 4: Input the actually collected superficial defect image of the formed part into the superficial defect detection model of the formed part, and the model outputs the corresponding defect detection result.

2. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 1, characterized in that: In the step 1, preprocessing the superficial defect map of the formed part includes removing invalid areas, adjusting the size, initializing and normalizing the values, adjusting the brightness and enhancing the data.

3. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 1, characterized in that: In the step three, the alienated neural network of the main and sub-channels includes a main channel part and a sub-channel part. The main channel part includes a feature extraction module, an information hierarchy fusion module and a detection and positioning module connected in sequence. The input of the alienated neural network based on the main and sub-channels is used as the input of the feature extraction module. The sub-channel part includes a grayscale level reduction layer, a grayscale feature extraction module, an edge detection module, a hysteresis threshold segmentation module and a defect feature map generation module connected in sequence. The input of the alienated neural network based on the main and sub-channels is also used as the input of the grayscale level reduction layer. The grayscale feature extraction module is connected to the detection and positioning module, and the defect feature map generation module is connected to the deepest feature extraction module in the feature extraction module.

4. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 3, characterized in that: In the grayscale reduction layer, the grayscale of the superficial defect image of the formed part is reduced without losing texture information, so as to obtain a grayscale image after the grayscale is reduced.

5. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 3, characterized in that: In the grayscale feature extraction module, a grayscale co-occurrence matrix and a grayscale histogram are generated according to the grayscale image after the grayscale level is reduced, features are extracted based on the grayscale co-occurrence matrix and the grayscale histogram respectively, a number of features are selected from the extracted features as grayscale features, a feature vector is composed of multiple grayscale features and used as the input of a classification model with the feature vector as the input, a classification result is obtained and used as a probabilistic posteriori for a detection and positioning module; and the grayscale image after the grayscale level is reduced is subjected to entropy increase sampling processing using the entropy in the grayscale features to obtain a corresponding grayscale feature map and use it as the input of an edge detection module.

6. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 3, characterized in that: In the edge detection module, a gradient detection method is used to extract edges in the vertical and horizontal directions of the grayscale feature map after entropy increase processing, and the vertical direction map and the horizontal direction map corresponding to all grayscale feature maps are obtained. Then, the average value of the vertical direction map and the horizontal direction map corresponding to the grayscale feature map is taken and the two maps are merged into one map and recorded as the edge feature map, thereby obtaining the edge feature map.

7. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 3, characterized in that: In the hysteresis threshold segmentation module, after performing hysteresis threshold segmentation on the obtained edge feature map, the corresponding segmented edge feature map is obtained.

8. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 3, characterized in that: In the defect feature map generation module, the segmented edge feature map is eroded and expanded using a morphological closing operation method to obtain a corresponding defect feature map and use it as the input of the deepest feature extraction module in the feature extraction module.

9. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 3, characterized in that: The main channel part is specifically a one-stage defect detector, which alienates the depth-separable convolution block in the feature extraction module of the backbone network in the detector into a depth-separable attention convolution block, and alienates the activation function of the convolution layer of the detector into a non-monotonic gradient optimization activation function; and alienates the loss function of the optimizer into a new function that integrates the direction vector angle loss.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Citation Information

Patent Citations

  • Aero-engine blade defect quick and accurate recognition method based on artificial intelligence

    CN111968084A

  • Processing quality detection method for high-strength composite material

    CN116612111A