A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase

Abstract

The present invention discloses a method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase. The method comprises the following steps: first, pre-processing the obtained superficial defect map of formed parts to form a source domain sample data set, and then constructing a training set and a validation set from it; then, constructing a primary-secondary dual-channel alienated neural network that combines machine learning with prior knowledge, and using the training and validation sets for training to obtain a superficial defect detection classifier for formed parts; finally, using it to detect the superficial defect map of formed parts actually collected. The present invention proposes a dual-channel dual-interface interactive alienated neural network. The grayscale co-generation and entropy increase method enhances the ability to extract and integrate extremely similar defect features, and can improve the classification accuracy of the anchor frame of superficial defects of industrial formed parts under the existing deep learning strategy while ensuring that the complexity of the classifier model does not increase and the detection speed does not decrease, effectively alleviating the problem of low accuracy of small samples.

Description

Technical Field

[0001] The present invention relates to a method for detecting and locating extremely similar defects on the surface and sub-surface of a part product in the field of industrial manufacturing, and in particular to a method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase. Background Art

[0002] Mechanical manufacturing, aerospace, shipbuilding, nuclear power and other fields all require the processing and manufacturing of various metal materials, plastic materials, composite materials, etc. There are many methods for forming industrial parts, including equal material manufacturing, subtractive manufacturing, additive manufacturing, and subtractive manufacturing, which can be specifically casting, forging, welding, rolling, injection molding, blow molding, extrusion molding, fused deposition modeling, laser additive manufacturing, etc. With the high-end development of the manufacturing industry, the industry has increasingly higher requirements for the quality of the surface and sub-surface of formed parts, such as defect detection for ultra-small sizes at the millimeter level and the inner surface of irregular and complex component structures, and defect detection for continuous fiber reinforced composite formed components. However, due to environmental factors or the influence of processing equipment and quality loss, various defects such as scratches, spots, micro cracks, pitting, etc. are likely to appear on the surface of the formed parts during the forming production process. This puts higher demands on the quality defect detection method of the formed parts.

[0003] Traditional defect detection and positioning often use manual detection and positioning methods to perform operations, relying on the detection and positioning capabilities and experience of professional workers, and carefully observing and analyzing the surface of the product's formed parts. This method is not only labor-intensive, but also time-consuming. Detection and positioning workers are prone to visual fatigue of the product due to long hours of work, which in turn causes false detection and missed detection of products. With the development of artificial intelligence methods, people have found that neural networks have powerful feature extraction capabilities. Many researchers have proposed methods for detecting and positioning shallow defects in formed parts based on deep learning models. The research team of the University of Findlay in the United States (Praveen Damacharla, Achuth Rao MV, Jordan Ringenberg, Ahmad Y Javaid, "TLU-Net: A Deep Learning Approach for Automatic Steel Surface Defect Detection," 2021International Conference on Applied Artificial Intelligence (ICAPAI), Halden, Norway, 2021, pp. 1-6, doi: 10.1109 / ICAPAI49758.2021.9462060.) Based on the basic framework of U-Net, two feature extraction networks, ResNet and DenseNet, were used to perform transfer learning on the NEU-DET steel plate defect dataset produced by Northeastern University in the United States to make up for the shortage of sample sets in the industry. However, since the product images taken in the actual manufacturing process will be interfered by various factors and will be quite different from the sample set, this method can effectively alleviate the problems caused by the small number of samples and reduce the huge cost consumption brought about by sufficient training sets. However, the effect in actual application scenarios will be poor and the accuracy will not be high, that is, there is a problem of small sample domain drift.

[0004] The research team of Tokyo University of Science in Japan (Nagata F, Tokuno K, Mitarai K, et al. Defect detection method using deep revolutional neural network, support vector machine and template matching techniques [J]. Artificial Life and Robotics, 2019, 24 (4): 512-519.) used the gray-level co-occurrence matrix to extract statistical features, combined with feature vector matching such as variance, mean, contrast, and entropy, and support vector machine to implement the defect detection method. However, it relies heavily on manual feature extraction and is difficult to capture the complex patterns and nonlinear relationships hidden in the data. It has certain limitations and limited generalization ability.

[0005] A U.S. patent (Wallingford R, Cong G, Park S. Deep learning based defect detection [P]: US11776108, 2023-10-03) provides a system that can be used for defect detection and positioning, the system comprising multiple components, including a deep learning network model, which is configured to generate a grayscale analog design data map for a location on a sample from a high-resolution map generated at the location by a high-resolution imaging system.

[0006] The above methods have their own characteristics and scope of application. Among the existing defect detection methods based on deep learning, on the one hand, the proposed defect detection method has some differences between the training set and the test set and the lack of samples in the sample set, which is the domain drift problem that is prevalent in transfer learning. As a result, the trained model is not good enough in accuracy, and has poor ability to recognize extremely similar defect features, making it difficult to perform; on the other hand, the machine vision detection and positioning method based entirely on manual feature extraction is more dependent on domain knowledge and professional experience, and it is difficult to capture the complex patterns and nonlinear relationships hidden in the data. It has certain limitations and limited generalization ability. Summary of the Invention

[0007] To address the problems in the background technology, this paper focuses on identifying superficial defects in industrial molded parts, providing a highly accurate detection and positioning method for detecting and locating extremely similar superficial defects on molded parts. This method also enhances the extraction of extremely similar defect features, alleviating the problem of insufficient training with small sample sizes. The present invention provides a method for detecting superficial defects in molded parts based on grayscale co-production and entropy increase. This method combines auxiliary feature extraction technology with deep learning detection and positioning methods, achieving controllable functionality of neural network methods in the field of defect detection and positioning. The main channel model is responsible for defect detection and localization. However, due to the lack of sample sets in real manufacturing environments and the domain drift problem of transfer learning, the accuracy is low. Therefore, to improve the generalization of the model, a secondary channel model is introduced. Traditional manual feature extraction methods have a unique a priori advantage in processing small sample sets. Therefore, an optimization method for the secondary channel, grayscale co-generation and entropy increase, is introduced. Through multiple image processing, the input image is jointly processed and abstracted to produce a series of probabilistic statistical features, including contrast and entropy. Entropy is further applied and processed to obtain the final entropy increase map of the secondary channel. The obtained feature information is used to provide dual feedback to the main channel model training stage through two methods, through dual interface interaction, to provide auxiliary guidance for the classifier model, further improving the robustness and generalization of the model, and ultimately achieving improved performance of the defect detection method. This method can quickly identify possible defects based on the superficial features of the input formed part and generate anchor boxes with corresponding confidence levels. While ensuring fast detection and localization speed, it improves the detection and localization accuracy of existing methods and models, and has adaptability for multi-scenario applications. Based on this method, the model can be further optimized so that each defect can be refined into a sub-classification domain to meet the needs of specific defect type segmentation in actual industrial production.

[0008] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions:

[0009] 1. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase

[0010] Step 1: After preprocessing the superficial defect features of the formed parts, a source domain sample dataset is formed, and then a training set and a validation set are constructed from it;

[0011] Step 2: Construct an alienated neural network based on the main and auxiliary dual channels;

[0012] Step 3: Use the training set to train the alienated neural network based on the main and auxiliary dual channels to obtain the trained alienated neural network. Then use the validation set to fine-tune the hyperparameters of the trained alienated neural network to obtain a superficial defect detection model for formed parts.

[0013] Step 4: Input the actually collected superficial defect features of the formed part into the superficial defect detection model of the formed part, and the model outputs the corresponding defect detection results.

[0014] In the step 1, pre-processing of the superficial defect features of the formed part includes removing invalid areas, adjusting the size, initializing and normalizing the values, adjusting the brightness, and enhancing the data.

[0015] In the step three, the alienated neural network of the main and sub-channels includes a main channel part and a sub-channel part. The main channel part includes a feature extraction module, an information hierarchy fusion module and a detection and positioning module connected in sequence. The input of the alienated neural network based on the main and sub-channels is used as the input of the feature extraction module. The sub-channel part includes a grayscale level reduction layer, a grayscale feature extraction module, an edge detection module, a hysteresis threshold segmentation module and a defect feature map generation module connected in sequence. The input of the alienated neural network based on the main and sub-channels is also used as the input of the grayscale level reduction layer. The grayscale feature extraction module is connected to the detection and positioning module, and the defect feature map generation module is connected to the deepest feature extraction module in the feature extraction module.

[0016] In the grayscale reduction layer, the grayscale level of the superficial defects of the formed part is reduced without losing texture information, and a grayscale image after the grayscale level is reduced is obtained.

[0017] In the grayscale feature extraction module, a grayscale co-occurrence matrix and a grayscale histogram are generated according to the grayscale image after the grayscale level is reduced, features are extracted based on the grayscale co-occurrence matrix and the grayscale histogram respectively, a number of features are selected from the extracted features as grayscale features, a feature vector is composed of multiple grayscale features and used as the input of a classification model with the feature vector as input, a classification result is obtained and used as the probabilistic posteriori of the detection and positioning module; and the entropy in the grayscale features is used to perform entropy increase sampling processing on the grayscale image after the grayscale level is reduced to obtain a corresponding grayscale feature map and use it as the input of the edge detection module.

[0018] In the edge detection module, a gradient detection method is used to extract edges in the vertical and horizontal directions of the grayscale feature map after entropy increase processing, and the vertical direction map and horizontal direction map corresponding to all grayscale feature maps are obtained. Then, the average values ​​of the vertical direction map and the horizontal direction map corresponding to the grayscale feature map are merged into one map and recorded as the edge feature map, thereby obtaining the edge feature map.

[0019] In the hysteresis threshold segmentation module, hysteresis threshold segmentation is performed on the obtained edge feature maps to obtain corresponding segmented edge feature maps.

[0020] In the defect feature map generation module, the segmented edge feature map is eroded and expanded using a morphological closing operation method to obtain a corresponding defect feature map and use it as the input of the deepest feature extraction module in the feature extraction module.

[0021] The main channel is specifically a one-stage defect detector, which transforms the depthwise separable convolution block in the feature extraction module of the backbone network of the detector into a depthwise separable attention convolution block, transforms the activation function of the convolution layer of the detector into a non-monotonic gradient optimization activation function, and transforms the loss function of the optimizer into a new function that integrates the direction vector angle loss.

[0022] The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of a method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase are realized.

[0023] 3. A computer-readable storage medium

[0024] A computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the steps of a method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase are implemented.

[0025] 4. A computer program product

[0026] The computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of a method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase.

[0027] Therefore, the present invention introduces a targeted primary-secondary dual-channel alienated neural network, which enhances the ability to extract and integrate extremely similar defect features, integrates traditional features and deep learning features, and can better capture the global and local information of feature maps, improve the accuracy and robustness of feature recognition and analysis, increase the controllability of the overall method, and effectively alleviate the domain drift problem of small samples.

[0028] The present invention has the following beneficial effects:

[0029] 1. The present invention proposes a method for detecting superficial defects of formed parts based on grayscale co-production and entropy increase. For problems such as nonlinear distortion, barrel distortion, uneven illumination, and high metal reflectivity that occur in special industrial scenes such as endoscopic images, the nonlinear distortion of features can be corrected in the preprocessing stage. For defects such as cracks, spots, and inclusions that may occur in actual industrial manufacturing, the method can directly detect and locate them and predict the confidence level. This method can improve the detection and positioning accuracy to a certain extent while maintaining fast detection and positioning speed and lightweight models. In addition, for extremely similar features that are difficult to identify in the general classification of extremely similar defect anchor frames, the feature extraction network architecture is specifically adjusted, thereby enhancing the method's ability to extract and integrate extremely similar defect features and effectively alleviating domain drift. Based on this method, the model can be further optimized so that each defect can obtain a refined sub-classification domain to meet the personalized needs of specific defect types in actual industrial production.

[0030] 2. The present invention adds a feature classification module of a secondary channel on the basis of the original deep learning model. The feature extraction technology of the module extracts corresponding features, which can be fused and combined with the features extracted by the deep learning model to enhance the diversity and characterization ability of the features. The obtained classification results can provide a posteriori guidance for the classification of the deep learning model. The traditional features and deep learning features are integrated through the grayscale co-production and entropy increase dual interface, which can better capture the global and local information of the feature map, improve the accuracy and robustness of feature recognition and analysis, and solve the "black box" problem of deep learning. It also increases the controllability of the overall identification method, can adjust the auxiliary feature extraction method according to different working scenarios, and then organically interact with the feature extraction of the main channel to assist and guide it, and ultimately achieve the performance improvement of the defect detection method.

[0031] 3. The present invention uses a convolutional layer and a converter interactive fusion module in the feature extraction network structure, which enhances the ability to integrate contextual features and improves the model's ability to detect and locate defects in complex backgrounds and unclear features in actual applications. By transforming the original activation function in the convolution module into a non-monotonic gradient optimization activation function, the generalization ability of the overall model is enhanced and the accuracy is improved. The original loss function of the optimizer is transformed into a new function that integrates the direction vector angle loss, and an angle penalty cost function is added, so that the anchor frame predicted by the trained model is more consistent with the real frame of the feature map, and the classification results of very similar defect anchor frames are more in line with actual working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The figure is a flow chart of the method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to the present invention.

[0033] Figure 2Schematic diagram of the alienated neural network architecture of the present invention.

[0034] Figure 3 Schematic diagram of the feature pyramid in feature fusion of the present invention.

[0035] Figure 4 This is a diagram of the converter module architecture introduced in the present invention.

[0036] Figure 5 This is the experimental result diagram of the grayscale histogram calculation of the c4 feature map in the present invention.

[0037] Figure 6 The original image and feature image are obtained after the grayscale co-generation and entropy increase processing is performed on a C3 feature image using the method of the present invention.

[0038] Figure 7 This is a curve diagram of various loss data during the training process of the model of the present invention.

[0039] Figure 8 This is a curve diagram of various loss data during the verification process of the model of the present invention.

[0040] Figure 9 This is a diagram showing the anchor frame detection and positioning results of superficial defects in formed parts without using the method of the present invention during the model verification process.

[0041] Figure 10 This is a diagram showing the anchor frame detection and positioning results of the superficial defects of the formed part using the method of the present invention during the model verification process.

[0042] Figure 11 and is the confusion matrix obtained without using the method of the present invention for training.

[0043] Figure 12 It is the confusion matrix obtained after training with the model of the present invention.

[0044] Figure 13 The F1 curves of various categories of the simulation experimental results of the model of the present invention are shown in FIG.

[0045] Figure 14 It is a curve diagram of the accuracy of each category of the experimental results of the model simulation of the present invention.

[0046] Figure 15 This is a recall rate curve diagram of each category of the simulation experiment results of the model of the present invention.

[0047] Figure 16 The figure shows the precision-recall curves of various categories of the simulation experimental results of the model of the present invention.

[0048] Figure 17 It is the accuracy curve of the model verification process results of not adopting the method of the present invention and the method of the present invention.

[0049] Figure 18 This is the recall rate curve of the model verification process results of the method of the present invention and the method of the present invention.

[0050] Figure 19 The mAP_0.5 curves are the results of the model verification process without using the method of the present invention and the method of the present invention.

[0051] Figure 20 The mAP_0.5-0.95 curves are the results of the model verification process without using the method of the present invention and the method of the present invention.

[0052] Figure 21 3. It is a schematic diagram comparing the defect detection effects of the method adopted by the present invention and the classical method. DETAILED DESCRIPTION

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention.

[0054] like Figure 1 As shown, the method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase proposed by the present invention comprises the following steps:

[0055] Step 1: Collect surface images of defective formed parts under actual working conditions. The data set used in the present invention has an equal number of images of six types of defects in formed parts, including cracks (crazing), inclusions (inclusion), spots (patches), pitted_surface, rolled-in_scale, and scratches (scratches). In the following description, these six defects are referred to as c1, c2, c3, c4, c5, and c6 respectively. The data set is screened and preprocessed, and the relatively poor feature maps (i.e., invalid maps) are eliminated. Invalid maps do not contain the six common manufacturing defects. The preprocessing measures include resizing, numerical initialization and normalization, brightness adjustment, data enhancement and other operations. Then, the feature map processed in step 1 is annotated with the real box using the image annotation technology, the defect area is marked and the defect name is written. The training set, validation set and test set can be divided into training set, validation set and test set according to the ratio of 8:1:1.

[0056] Step 2: Constructing a dissimilated neural network based on the main and auxiliary dual channels; optimizing the dissimilated neural network structure of the main and auxiliary dual channels. The model constructed by the present invention is as follows: Figure 2As shown in the figure. The alienated neural network of the main and secondary channels consists of a main channel part and a secondary channel part. The main channel model is the main body responsible for defect detection and positioning. However, due to the lack of sample sets in the actual manufacturing environment and the domain drift problem of transfer learning, the accuracy rate is not high. Therefore, in order to improve the generalization of the classifier model, an additional secondary channel model is introduced. In terms of processing small sample sets, traditional manual feature extraction methods have unique a priori advantages. Therefore, an optimization method for the secondary channel—grayscale co-generation entropy increase—is introduced for processing. The obtained feature information is used to provide positive feedback to the main channel model training stage in two ways, realizing an auxiliary guidance role for the model, further improving the robustness and generalization of the model, and ultimately achieving improved performance of the defect detection method.

[0057] The main channel consists of a sequentially connected feature extraction module, an information hierarchy fusion module, and a detection and localization module. The input of the alienated neural network based on the primary and secondary channels serves as the input of the feature extraction module. The feature extraction module uses convolutional layers to perform multiple downsampling of the input batch images, continuously extracting image features. The information hierarchy fusion module performs multi-scale information fusion of features, transferring and fusing deep feature information with shallower information through upsampling. The detection and localization module is used to make predictions based on the main channel network model. It uses non-maximum suppression to select the bounding box with the highest confidence and outputs the predicted defect type, confidence level, and location vector.

[0058] The secondary channel part includes a grayscale level reduction layer, a grayscale feature extraction module, an edge detection module, a hysteresis threshold segmentation module and a defect feature map generation module which are connected in sequence. The input of the alienated neural network based on the main and secondary dual channels is also used as the input of the grayscale level reduction layer. The grayscale feature extraction module is connected to the detection and positioning module. The defect feature map generation module is connected to the deepest feature extraction module in the feature extraction module, which is used to fuse the defect feature map and the deepest feature.

[0059] The secondary channel part includes a grayscale level reduction layer, a grayscale feature extraction module, an edge detection module, a hysteresis threshold segmentation module and a defect feature map generation module which are connected in sequence. The input of the alienated neural network based on the main and secondary dual channels is also used as the input of the grayscale level reduction layer. The grayscale feature extraction module is connected to the detection and positioning module. The defect feature map generation module is connected to the deepest feature extraction module in the feature extraction module, which is used to fuse the defect feature map and the deepest feature.

[0060] In the grayscale reduction layer, the grayscale of the superficial defect image of the molded part is reduced without losing the texture information, and a grayscale image after the grayscale level reduction is obtained. If the superficial defect image of the molded part is an RGB image, it is also first grayscaled to become a grayscale image, and then the grayscale level is reduced. The goal of this step is to reduce the grayscale of the image without losing the texture information. This is to reduce the calculation time of the Gray-Level Co-occurrence Matrix (GLCM). Reducing the grayscale level involves reducing the size of the GLCM;

[0061] In the grayscale feature extraction module, a grayscale co-occurrence matrix and grayscale histogram are generated from the grayscale image after grayscale reduction. Features are extracted based on the grayscale co-occurrence matrix and grayscale histogram, including mean, variance, energy, contrast, homogeneity, uniformity, and entropy. Appropriate features are selected from these extracted features based on the actual situation. A feature vector is formed from these features and used as the input to a classification model (such as a support vector machine (SVM) or backpropagation neural network) that takes this feature vector as input. The classification result is then used as a probabilistic posterior in the detection and localization module. Furthermore, the entropy of the grayscale features is used to perform entropy-increasing sampling on the grayscale image after grayscale reduction. This results in a corresponding grayscale feature map, which is then used as one of the inputs to the edge detection module. In this step, the image is overlaid with an analysis window of size T and offset d. The goal is to calculate the GLCM for each analysis window and extract the Haralick feature—the entropy. This feature is assigned to the center point of the analysis window. Finally, the output image is obtained.

[0062] In the edge detection module, the gradient detection method is used to extract the edges in the vertical and horizontal directions of the grayscale feature map after entropy increase processing, and the vertical direction map and horizontal direction map corresponding to all grayscale feature maps are obtained. Then, the average value of the vertical direction map and the horizontal direction map corresponding to the grayscale feature map is taken and the two maps are merged into one map and recorded as the edge feature map, thereby obtaining the edge feature map.

[0063] In the hysteresis threshold segmentation module, hysteresis threshold segmentation is performed on each of the acquired edge feature maps to obtain the corresponding segmented edge feature maps. Unlike standard threshold segmentation methods, hysteresis threshold segmentation is not equal at all points in the graph. The goal is to preserve the strongest edges of the graph and maintain their continuity. First, two thresholds are used: a high threshold and a low threshold. The high threshold is used to detect and locate the strongest edges of the graph. On the other hand, the low threshold can highlight the weaker edges of the graph. Each weak edge is conserved only if its neighbors are strong edges detected and located by the high threshold.

[0064] In the defect feature map generation module, all segmented edge feature maps are eroded and dilated using a morphological closing operation. The resulting defect feature map is then used as one of the inputs to the deepest feature extraction module in the feature extraction module. Morphological closing is used to address edge discontinuities that may have occurred in the previous step. The morphological closing operator is a key operator in the field of mathematical morphology. It can be derived from the basic operations of erosion and dilation and applied to binary images. Closing tends to expand the boundaries of bright areas in the image. It is defined as dilation and erosion using the same structuring element in both operations.

[0065] By fusing traditional features with deep learning features, the present invention can better capture the global and local information of the image, improve the accuracy and robustness of image recognition and analysis, and also solve the "black box" problem of deep learning, increase the controllability of the overall algorithm, and be able to adjust the auxiliary feature extraction method according to different working scenarios, thereby organically interacting the assistance and guidance of the feature extraction of the main channel, and ultimately achieving performance improvement of the defect detection method.

[0066] A c3 defect image is processed by grayscale co-generation entropy increase method as shown below: Figure 6 As shown, Figure 6 (a) is the original image, Figure 6 (b) is the feature map after grayscale co-production and entropy increase processing. After processing, the edge significance of the defect is improved. The large black area in the central area of ​​the image is the defect separated from the image, which corresponds to the defect in the center of the original image. There are fewer noise and artifacts introduced, and there are some false areas but the impact is relatively small. The edge of the defect is mostly strong edge, and a small part is weak edge. It has a certain degree of coherence, which can better separate the defect features and further guide the training of the deep learning model.

[0067] like Figure 5 As shown, the grayscale histogram of the c4 image of the six defects was calculated to obtain a grayscale histogram. The Peak-Guass probability density function was used for curve fitting to obtain the fitting curve shown in the figure. The parameters of this curve are offset 1.68538, peak center position 151.7681, peak width 47.42994, peak area 39637.90768, half-height width 55.84449, and peak height 666.80403. Through further analysis, the characteristic statistics of the image were obtained as grayscale mean 151.43, grayscale variance 584.11053, energy 39.50574, entropy 6.636535, and contrast 214. Following the same method, the grayscale feature extraction method of grayscale histogram and grayscale symbiotic entropy increase was performed on the six defects to obtain a series of feature vectors, which were then fused. The fused features were then trained in a support vector machine classifier to obtain a classification model dominated by grayscale features.

[0068] The main channel part is specifically a one-stage defect detector, which is mainly composed of several network layers, namely convolution layer, depthwise separable convolution layer, upsampling layer, information fusion layer, etc. In the feature extraction stage of feature perception fusion, the convolution layer is combined with the depthwise separable convolution layer to fully downsample the image and extract specific features. In the final stage, the present invention uses the convolution layer and the converter interactive fusion module. This module introduces the converter module under the condition of adding a small amount of parameters, collects and integrates the residual features to a certain extent, obtains the feature information of the image in a higher dimension, and expands the receptive field of each pixel of the final feature map of the entire model to improve the accuracy of the final model. In the feature perception fusion stage, the following methods are used: Figure 3 The feature pyramid model is used to perform hierarchical fusion of feature information, and in the last module, the prediction module, feature maps of different sizes are further convolved to extract features and predict various data.

[0069] The convolution layer in the feature extraction module of the main channel network model interacts with the converter fusion module. This module combines the converter to enhance the ability to extract and integrate very similar defect features, enabling the model to quickly perform hierarchical fusion of features, that is, to collect and integrate residual features to a certain extent, obtain feature information of the graph at a higher dimension, expand the receptive field of each pixel in the final feature map of the entire model, and enhance the ability to extract and integrate very similar defect features through feature perception fusion, effectively alleviating the problem of domain drift. Figure 4As shown, the converter consists of four submodules: an encoder, a decoder, residual connections and normalization, and a trigonometric position encoder. The encoder module consists of six encoder modules, each of which contains two submodules: a multi-head self-attention layer and a feed-forward fully connected layer. The multi-head self-attention layer uses a scaled dot-product attention algorithm. Experimental results show that multiple heads can extract features from different heads at a more detailed level, achieving better feature extraction than a single head. The feed-forward fully connected layer consists of two fully connected layers with an activation function added between the linear transformations. The specific dimensions are quadrupled, meaning that if the multi-head self-attention dimension is 512, the transformation dimension within the layer is 2048. The decoder module consists of six decoder modules, each of which contains three submodules: a multi-head self-attention layer, an encoder-decoder attention layer, and a feed-forward fully connected layer. The multi-head self-attention layer uses the same scaled dot-product attention algorithm as the encoder module, with the key difference being the addition of a look-ahead mask to mask future information. The key difference between the encoder-decoder attention layer and the previous multi-head self-attention layer is that Q ≠ K = V. The matrix Q comes from the output of the previous decoder module, while K and V come from the encoder output. The feed-forward fully connected layer is exactly the same as the encoder. Residual connections and normalization modules are connected after each sublayer in each encoder and decoder module. The purpose of residual connections is to transmit information deeper without loss, thereby enhancing the overall model's ability to fit the data. Normalization is a layer-level numerical normalization operation that prevents training anomalies and slow loss convergence caused by excessively large or small parameters.

[0070] Figure 3 The figure is a schematic diagram of the feature pyramid in the neural network of the present invention. Feature Pyramid Network (hereinafter referred to as FPN) is a classification detection and positioning method proposed to solve the problem of incompatibility between feature size and target to be detected. When the target size is small and the background is large, it means that in the process of layer-by-layer feature extraction, the feature map size is becoming more and more streamlined, and the receptive field will feel more parts of the background. If the network is relatively deep, the background features will be more obvious in the continuous convolution, and small targets may even lose their feature information and be mistaken for being part of the background. Conversely, if it is a large target on the feature map of a small background, the small convolution kernel will increase the amount of calculation and improve the calculation time. FPN can detect and locate simple, large targets in the shallow layer, and complex, small targets in the deep layer.

[0071] Figure 4This is the architecture of the converter module introduced in this invention. The input embedding map is positionally encoded and then fed into the region module. In this region module, it is first processed using a multi-head attention mechanism. The resulting output is residually concatenated with the unprocessed data and then normalized. The normalized output is then fed into a feedback mechanism and residually concatenated with the unprocessed data again, followed by normalization. This region module operation is repeated N times before outputting the final result.

[0072] Step 3: Use the training set to train the alienated neural network based on the main and auxiliary dual channels to obtain the trained alienated neural network. Then use the validation set to fine-tune the hyperparameters of the trained alienated neural network to obtain a superficial defect detection model for formed parts.

[0073] Specifically, the training set was used as input for a primary-secondary dual-channel alienated neural network, and the model was trained to obtain a weight file, mAP_0.5 / %, and mAP_0.5:0.95 / %. The basic training parameters of the present invention were: graph size was set to 640*640, batch size was set to 16, number of iterations was set to 200, and the optimizer was stochastic gradient descent (SGD), with default hyperparameters.

[0074] The present invention transforms the original activation function of each convolution module into a non-monotonic gradient optimized activation function. The calculation formula of this activation function f(x) is as follows:

[0075] f(x)=x*tanh(ln(1+e))

[0076] Where x is the input matrix and e is the natural logarithm.

[0077] The convolution module generally consists of three parts: a two-dimensional convolutional layer, batch normalization, and an activation function. The non-monotonic gradient-optimized activation function provides smoother transitions and greater tolerance for slightly negative values, further enhancing the stability of the overall model process, improving the robustness and generalization of the trained model, and ultimately improving the accuracy of anchor box classification for very similar defects.

[0078] The optimizer's loss function is transformed from the original function to a new loss function that integrates the direction vector angle loss. The function calculation formula is as follows:

[0079]

[0080]

[0081]

[0082]

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089] Among them, Λ is the angle loss, x, y are the position coordinates, Δ is the distance loss, Ω is the shape loss, t is the cumulative parameter symbol, γ is the angle loss variant, L box is the anchor box loss, θ is the degree of attention of the shape loss, IoU is the (Intersection over Union) loss, c h is the height difference between the center of the real box and the predicted box, α is the angle between the center of the real box and the predicted box, and σ is the distance between the center of the real box and the predicted box. is the horizontal coordinate of the center of the real frame, is the horizontal coordinate of the center of the prediction box, is the ordinate of the center of the real frame, is the vertical coordinate of the center of the prediction box, ρ x is the correlation coefficient between the horizontal coordinate and the width difference, c w is the width of the minimum bounding rectangle of the real box and the predicted box, c h is the height of the minimum bounding rectangle of the real box and the predicted box, ρ y is the connection coefficient between the ordinate and the height difference, ω w is the ratio of the width difference between the real box and the predicted box to the maximum value of the two, ω h is the ratio of the height difference between the real box and the predicted box to the maximum value of the two, w is the width of the predicted box, | | is the absolute value symbol, w gt is the real box width, h is the predicted box height, h gt is the height of the real box, B is the area of ​​the predicted box, and B GT is the area of ​​the real frame.

[0090] Figure 7 This is a graph of various loss data during the model training process of the present invention. It includes the classification loss of training, the calculation of whether the anchor box and the corresponding calibration classification are correct, the positioning loss of training, the error between the prediction box and the calibration box, the confidence loss of training, and the confidence of the calculated network. In the positioning loss graph, Figure 7The maximum slope of the fitting curve in (a) is -0.000173, the minimum slope is -0.0037, the average slope is -0.00062, and the average slope angle is 151.26°; in the confidence loss graph, Figure 7 The maximum slope of the fitting curve in (b) is -0.00004286, the minimum slope is -0.0034, the average slope is -0.000103, and the average slope angle is 138.75°; in the classification loss graph, Figure 7 The maximum slope of the fitting curve in (c) is -0.00008333, the minimum slope is -0.00144, the average slope is -0.000302, and the average slope angle is 177.69°.

[0091] Figure 8 This is a graph of various loss data during the model verification process of the present invention. It includes the classification loss of verification, the calculation of whether the anchor box and the corresponding calibration classification are correct, the positioning loss of verification, the error between the predicted box and the calibration box, the confidence loss of verification, and the confidence of the calculated network. In the positioning loss graph, Figure 8 The maximum slope of the fitting curve in (a) is -0.000014815, the minimum slope is -0.0036, the average slope is -0.0062, and the average slope angle is 169.21°; in the confidence loss graph, Figure 8 The maximum slope of the fitting curve in (b) is -0.00010769, the minimum slope is -0.00123077, the average slope is 0.000103, and the average slope angle is 19.29°; in the classification loss graph, Figure 8 The maximum slope of the fitting curve in (c) is -0.00003846, the minimum slope is -0.00175, the average slope is -0.00312, and the average slope angle is -176.38°;

[0092] Step 4: Input the actual collected superficial defect image of the formed part into the superficial defect detection model of the formed part. The model outputs the corresponding defect detection result, which includes the classification result, anchor box position and confidence level.

[0093] Figure 9 and Figure 10The following is a diagram showing the anchor frame detection and positioning results of superficial and very similar defects on formed parts during the verification process of the present invention without the use of the method of the present invention and the method of the present invention. As shown in the figure, the upper left corner of each image is the source label of the test image, and the middle box is the anchor frame generated by the method model. The anchor frame contains the image classification, the center point position of the anchor frame, the width and height normalized data, and the confidence information. Comparing the two images, the training label of the first image in the upper left corner is c1. The method of the present invention did not use the method and did not identify the defect, while the method of the present invention identified the c1 defect and gave a confidence of 0.3; the training label of the second image is c5. The method of the present invention did not use the method of the present invention and identified three defects, but two of them were obviously overlapped, while the method of the present invention did not have this problem; in the fourth image, the method of the present invention did not use the method of the present invention and only identified one defect of c4, but the method of the present invention identified three c4 defects with higher accuracy; through the comparison of these 16 verification images, it can be found that the performance of the model trained by the method of the present invention on the verification set is better than that of the model trained by the method of the present invention.

[0094] Figure 11 and Figure 12 The confusion matrices obtained after training with and without the method of the present invention and the model of the present invention are respectively. It is a two-dimensional table commonly used to evaluate the performance of classification models. In the confusion entropy increase, each column represents the predicted value and each row represents the true value. Therefore, each element in the confusion matrix represents the number of times a sample is predicted to be a certain category. Each element in the figure is also normalized. It can be seen from the figure that the confusion matrix of the model not trained by the method of the present invention is that in the sample with the true value of c1, 0.64 is c1 and 0.36 is background; in the sample with the true value of c2, 0.60 is c2 and 0.40 is background; in the sample with the true value of c3, 0.93 is the true value and 0.07 is background; in the sample with the true value of c4, 0.89 is c4 and 0.11 is background; in the sample with the true value of c5, 0.87 is c5 and 0.12 is background; in the sample with the true value of c6, 1.00 is c6; the confusion matrix of the model trained by the method of the present invention is that in the sample with the true value of c1, 0.82 is c1 , 0.18 is the background; in the sample with the true value of c2, 0.70 is c2, 0.20 is the background, and 0.10 is c3; in the sample with the true value of c3, 0.93 is c3, and 0.07 is the background; in the sample with the true value of c4, 0.89 is c4, and 0.11 is the background; in the sample with the true value of c5, 0.87 is c5, and 0.12 is the background; in the sample with the true value of c6, 0.90 is c6, and 0.10 is the background. It can be seen that compared with the method not adopted, the model trained by the method of the present invention has significantly improved detection and positioning effects on c1 and c4 defects, and slightly decreased in detection and positioning of c6 defects. Overall, the performance of the model has been greatly improved.

[0095] Figure 13 The following is an F1 curve diagram of each category of the experimental results of the model simulation of the present invention. The F1 metric is the harmonic mean of the precision and recall rates, in which the curve of each category shows a trend of first increasing and then decreasing as the confidence level of the horizontal axis increases. The C1 curve reaches a maximum value of 0.74 when the confidence level is 0.36, the C2 curve reaches a maximum value of 0.80 when the confidence level is 0.22, the C3 curve reaches a maximum value of 0.83 when the confidence level is 0.56, the C4 curve reaches a maximum value of 0.89 when the confidence level is 0.18, the C5 curve reaches a maximum value of 0.78 when the confidence level is 0.48, and the C6 curve reaches a maximum value of 0.93 when the confidence level is 0.50. It can be seen that the confidence level of the method of the present invention for the two types of defects c1 and c2 is relatively low, and the difficulty in identifying the two types of defects is relatively large, which corresponds to the confusion matrix results.

[0096] Figure 14 The following is a graph of the P (precision) curves for each category of the model simulation experiment results of the present invention. As can be seen from the figure, the precision curves of each category generally show an upward trend. The C1 precision curve reaches a maximum value of 1.0 when the confidence level is 0.59, the C2 precision curve reaches a maximum value of 1.0 when the confidence level is 0.82, the C3 precision curve reaches a maximum value of 1.0 when the confidence level is 0.84, the C4 precision curve reaches a maximum value of 1.0 when the confidence level is 0.43, the C5 precision curve reaches a maximum value of 1.0 when the confidence level is 0.72, and the C6 precision curve reaches a maximum value of 1.0 when the confidence level is 0.59.

[0097] Figure 15 The following is a graph of the R (recall rate) for each category of the experimental results of the model simulation of the present invention. As can be seen from the figure, the recall rate curves of each category basically show a continuous downward trend. The c1 recall rate curve reaches a minimum value of 0.0 at a confidence level of 0.70, the c2 recall rate curve reaches a minimum value of 0.0 at a confidence level of 0.81, the c3 recall rate curve reaches a minimum value of 0.0 at a confidence level of 0.90, the c4 recall rate curve reaches a minimum value of 0.0 at a confidence level of 0.88, the c5 recall rate curve reaches a minimum value of 0.0 at a confidence level of 0.79, and the c6 recall rate curve reaches a minimum value of 0.0 at a confidence level of 0.85.

[0098] Figure 16The following is a PR curve diagram of each category of the experimental results of the model of the present invention. The area under each category curve represents the current AP_0.5 value of each category, where 0.5 represents an IoU value of 0.5. As can be seen from the figure, the AP_0.5 value of c1 is 0.687, the AP_0.5 value of c2 is 0.708, the AP_0.5 value of c3 is 0.929, the AP_0.5 value of c4 is 0.963, the AP_0.5 value of c5 is 0.829, and the AP_0.5 value of c2 is 0.977. From the results, it can be seen that the performance of the method of the present invention in identifying defects of types c1 and c2 is relatively poor, and it is more difficult.

[0099] Figure 17 The accuracy curves of the model verification process results of the method of the present invention and the method of the present invention are shown. The accuracy of the method of the present invention fluctuates around 0.64 after 20 iterations. The accuracy after 10 iterations is averaged and a straight line is drawn to represent it. The vertical coordinate of the straight line is 0.65121. The accuracy of the method of the present invention fluctuates around 0.75 after 10 iterations. The accuracy after 10 iterations is averaged and a straight line is drawn to represent it. The vertical coordinate of the straight line is 0.73654. This shows that the accuracy of the verification process of the method of the present invention is higher than that of the method of the present invention, and the detection and positioning performance of the model is better.

[0100] Figure 18 The recall rate curves of the model verification process results of the method of the present invention and the method of the present invention are shown. The recall rate of the method of the present invention fluctuates around 0.70 after 15 iterations. The recall rate after 15 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.71021. The recall rate of the method of the present invention fluctuates around 0.78 after 15 iterations. The recall rate after 15 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.77985. This shows that the overall recall rate during the verification process of the method of the present invention is higher than that of the method of the present invention, and the recall performance of the model is better.

[0101] Figure 19The mAP_0.5 curves for the model validation process without the method of the present invention and without the method of the present invention are shown. The mAP_0.5 without the method of the present invention fluctuates around 0.70 after 15 iterations. The mAP_0.5 after 15 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.70861. The mAP_0.5 of the method of the present invention fluctuates around 0.80 after 15 iterations. The mAP_0.5 after 15 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.79894. This shows that the mAP_0.5 during the validation process of the method of the present invention is generally higher than that without the method of the present invention, and the average classification accuracy of the model is higher when the IoU threshold is set to 0.5.

[0102] Figure 20 The mAP_0.5-0.95 curves are the results of the model verification process without the method of the present invention and the method of the present invention. The full name of mAP is (mean average precision), that is, the average precision. The mAP_0.5-0.95 without the method of the present invention basically fluctuates around 0.37 after 17 iterations. The mAP_0.5-0.95 after 17 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.38162; the mAP_0.5-0.95 of the method of the present invention basically fluctuates around 0.44 after 17 iterations. The mAP_0.5-0.95 after 17 iterations is averaged and a straight line is drawn to represent it. The ordinate of the straight line is 0.44984. This shows that the mAP_0.5-0.95 during the verification process of the method of the present invention is higher than that without the method of the present invention. The average classification accuracy of the model is higher when the IoU threshold is set to 0.5-0.95.

[0103] A comparative experiment was conducted on the alienated neural network based on the main and auxiliary dual channels and the model without the method of the present invention. The results are as follows: Figure 21 As shown. mAP_0.5 is to calculate the AP of all images of each category when IoU is set to 0.5, and then the average of all categories is mAP. mAP_0.5:0.95 represents the average mAP at different IoU thresholds (from 0.5 to 0.95, step size 0.05) (0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95). As can be seen from the figure, the dual-channel model for superficial defect detection and positioning of formed parts based on grayscale co-generation and entropy increase has improved these two indicators by 5.8% and 6.6% respectively compared with the model that does not adopt the method of this patent. It can be seen that the model for superficial defect detection and positioning of formed parts based on grayscale co-generation and entropy increase proposed by the present invention has a better effect in terms of accuracy, meets the requirements for superficial defect detection and positioning of industrial formed parts, and is conducive to actual deployment and application in industry.

[0104] The above specific embodiments are used to illustrate the present invention rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase, characterized in that: The following steps are involved: Step 1: After preprocessing multiple superficial defect images of formed parts, a source domain sample dataset is formed, which is then used to construct the training set and validation set; Step 2: Construct an alienated neural network based on the main and auxiliary dual channels; Step 3: Use the training set to train the alienated neural network based on the main and auxiliary dual channels to obtain the trained alienated neural network. Then use the validation set to fine-tune the hyperparameters of the trained alienated neural network to obtain a superficial defect detection model for formed parts. In the step 3, the alienated neural network of the main and secondary dual channels includes a main channel part and a secondary channel part. The main channel part includes a feature extraction module, an information hierarchy fusion module, and a detection and positioning module connected in sequence. The input of the alienated neural network based on the main and secondary dual channels serves as the input of the feature extraction module. The secondary channel part includes a grayscale level reduction layer, a grayscale feature extraction module, an edge detection module, a hysteresis threshold segmentation module, and a defect feature map generation module connected in sequence. The input of the alienated neural network based on the main and secondary dual channels also serves as the input of the grayscale level reduction layer. The grayscale feature extraction module is connected to the detection and positioning module, and the defect feature map generation module is connected to the deepest feature extraction module in the feature extraction module. In the grayscale feature extraction module, a grayscale co-occurrence matrix and a grayscale histogram are generated based on the grayscale image after the grayscale level is reduced, features are extracted based on the grayscale co-occurrence matrix and the grayscale histogram, a plurality of features are selected from the extracted features as grayscale features, a feature vector is formed by the plurality of grayscale features and used as input to a classification model whose input is the feature vector, a classification result is obtained and used as a probabilistic posteriori for the detection and positioning module; and the grayscale image after the grayscale level is reduced is subjected to entropy increase sampling processing using the entropy in the grayscale features to obtain a corresponding grayscale feature map and use it as input to the edge detection module; In the defect feature map generation module, the segmented edge feature map is eroded and expanded using a morphological closing operation method to obtain a corresponding defect feature map and serve as the input of the deepest feature extraction module in the feature extraction module; Step 4: Input the actually collected superficial defect image of the formed part into the superficial defect detection model of the formed part, and the model outputs the corresponding defect detection result.

2. The method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 1, characterized in that: In the step 1, preprocessing the superficial defect map of the formed part includes removing invalid areas, adjusting the size, initializing and normalizing the values, adjusting the brightness, and enhancing the data.

3. The method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 1, characterized in that: In the grayscale reduction layer, the grayscale of the superficial defect image of the formed part is reduced without losing texture information, thereby obtaining a grayscale image after the grayscale is reduced.

4. The method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 1, characterized in that: In the edge detection module, a gradient detection method is used to extract edges in the vertical and horizontal directions of the grayscale feature map after entropy increase processing, and the vertical direction map and horizontal direction map corresponding to all grayscale feature maps are obtained. Then, the average value of the vertical direction map and the horizontal direction map corresponding to the grayscale feature map is taken and the two maps are merged into one map and recorded as the edge feature map, thereby obtaining the edge feature map.

5. The method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 1, characterized in that: In the hysteresis threshold segmentation module, hysteresis threshold segmentation is performed on the obtained edge feature maps to obtain corresponding segmented edge feature maps.

6. The method for detecting superficial defects of formed parts based on grayscale co-generation and entropy increase according to claim 1, characterized in that: The main channel part is specifically a one-stage defect detector, which alienates the depth-separable convolution block in the feature extraction module of the backbone network in the detector into a depth-separable attention convolution block, and alienates the activation function of the convolution layer of the detector into a non-monotonic gradient optimization activation function; and alienates the loss function of the optimizer into a new function that integrates the direction vector angle loss.

7. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images