Industrial topology type defect detection, classification and segmentation method

Abstract

The invention discloses a detection, classification and segmentation method for industrial topology type defects, and relates to the technical field of deep learning and digital image processing, and the method comprises a topological structure feature extraction module, a multi-view feature fusion module and a loss constraint and segmentation module. The feature extraction module extracts multi-level features from the input image to obtain topological structure feature information of the defect; the multi-view feature fusion module supplements attention to features from multiple angles in the feature fusion process, and important information from different global forms is reserved; the loss constraint and segmentation module constrains loss based on continuity of continuous homology so as to better constrain topological continuity of segmentation and obtain a saliency defect prediction map. According to the invention, topology type defects can be efficiently identified.

Description

Technical Field

[0001] The present application relates to the technical fields of deep learning and digital image processing, and in particular to a method for detecting, classifying, and segmenting industrial topological defects. Background Art

[0002] With the rapid advancement of deep learning technology and visual attention models, salient object detection, classification, and segmentation have become a key area in computer vision research. This technology imitates the characteristics of the human visual system through deep learning methods, aiming to efficiently identify and locate specific objects in images or videos. The ability of the human visual system to selectively focus on areas of interest based on previous knowledge has inspired the development of salient detection, classification, and segmentation techniques. Salient detection, classification, and segmentation are not only crucial for object recognition, but also show great application potential in many aspects such as image and video compression, image retrieval, and image retargeting. The basic architecture of modern neural networks can be traced back to 1998, when the backpropagation algorithm was introduced, which enabled the network to learn and optimize by passing errors backward while passing information forward.

[0003] In recent years, deep learning models have been widely used in industrial defect detection, classification, and segmentation. For example, neural networks with advanced feature extraction are used for metal surface defect detection, classification, and segmentation, effectively improving the detection, classification, and segmentation accuracy; deep neural networks have shown superior performance in solar panel defect detection, classification, and segmentation; real-time detection, classification, and segmentation models based on real-time detection, classification, and segmentation frameworks are used for rapid defect screening of packaging materials. The application of these technologies has significantly improved the level of automation and product quality in industrial production.

[0004] In the manufacturing process of digital products such as mobile phones, defects inside or on the surface of tiny parts may seem insignificant, but in fact they may have a serious impact on the performance and reliability of the product. These defects may not only affect the electrical performance of electronic components, leading to unstable signal transmission, poor contact and other problems, but may also become stress concentration points, reduce the mechanical strength of components, and increase the risk of product failure during use. Therefore, it is crucial to detect, classify, and segment defects in such tiny parts.

[0005] Efficient defect detection, classification, and segmentation can not only help manufacturers to promptly discover and correct problems in the production process and prevent defective products from entering the market, but also significantly improve product quality and customer satisfaction, thereby enhancing the competitiveness of enterprises. At present, the mainstream method of defect detection, classification, and segmentation is still manual visual inspection, where operators check for defects on the surface of parts with the naked eye or with the help of a magnifying glass or microscope.

[0006] However, manual detection, classification, and segmentation are inefficient, prone to missed detection and false detection due to differences in workers' personal qualities and technical levels, and have high operating costs. Although deep learning methods are superior to traditional methods in detection, classification, and segmentation accuracy and efficiency, their application in the industrial field has not yet been popularized. In addition, the accuracy and efficiency of detection, classification, and segmentation need to be improved, and the level of intelligence and automation of industrial production needs to be improved. Summary of the invention

[0007] Purpose of the invention: This application provides a method for detecting, classifying and segmenting industrial topological type defects, aiming to improve the technical problems mentioned above.

[0008] Technical solution: The present application provides a method for detecting, classifying and segmenting industrial topological type defects, including: a topological structure feature extraction module, a multi-view feature fusion module, and a loss constraint and segmentation module; the feature extraction module extracts multi-level features from the input image to obtain the topological structure feature information of the defect; the multi-view feature fusion module supplements the focus on features from multiple angles during the feature fusion process, and retains important information from different global forms; the loss constraint and segmentation module is based on a continuity constraint loss of continuous homology to better constrain the topological continuity of the segmentation and obtain a significant defect prediction map.

[0009] Furthermore, the topological structure feature extraction module includes a dynamic deformable convolution operator, and the feature extraction module performs two-dimensional changes on the dynamic deformable convolution operator by introducing an offset to focus on local features of the tubular structure and bending.

[0010] Furthermore, the multi-view feature fusion module is based on multiple morphological kernel templates generated by dynamic deformable convolution, which can observe the structural characteristics of defects from multiple angles.

[0011] Furthermore, the loss constraint and segmentation module includes a decoupling head module and a topological continuity constraint loss module. The decoupling head module restores the detail information of the image and improves the spatial resolution by upsampling to facilitate accurate segmentation; the topological continuity constraint loss module is used to constrain the continuity of the segmentation, guide the network to focus on the broken areas with abnormal pixels, and realize continuity constraints from a topological perspective.

[0012] Furthermore, in step 1, the feature extraction module introduces an offset to perform a two-dimensional change on the dynamic deformable convolution operator to focus on the local features of the tubular structure and the bend. The dynamic deformable convolution operator is used to extract multi-level features from the input image to obtain the topological structural feature information of the defect; in step 2, the multi-view feature fuses multiple morphological kernel templates generated based on the dynamic deformable convolution to observe the structural features of the defect from multiple angles; in step 3, the fusion result of each layer processed in step 2 is passed to the decoupling head module for decoding, defect location and segmentation operations, and finally a significant defect prediction map is obtained; in step 4, the loss value is calculated.

[0013] Furthermore, the specific operation of the two-dimensional change of the dynamic deformable convolution operator in step 1 is as follows: Among them, for a given 2D convolution coordinate M, the center coordinate is Mi = (ui, vi), then a 3×3 convolution kernel M is expressed as: M = {(u-1, v-1), (u-1, v), ..., (u+1, v+1)}, wherein in the u-axis direction, the specific position of each grid in M ​​is expressed as: Mi±e = (ui±e, vi±e) In the v-axis direction, the specific position of each grid in M ​​is expressed as: Mj±e = (uj±e, vj±e) e = 0, 1, 2, 3, 4 represents the horizontal distance from the center grid, wherein the offset Δ = {δ|δ∈[-1, 1]} is the amount by which Mi+1 increases relative to Mi.

[0014] Furthermore, the specific operation of the multi-view feature fusion strategy for a given deformable convolution in step 2 is as follows: Among them, P is each convolution kernel, where g m (P u ), g m (P v ) and represent the feature maps extracted by the deformable convolution operator from the u-axis and v-axis respectively.

[0015] Furthermore, a random dropout strategy is introduced in step 2 to set a random dropout probability p for each feature in each layer of feature maps.

[0016] Furthermore, the topological continuity constraint loss in step 4 is the difference between the true value y and the predicted value y^.

[0017] In summary, this application has at least the following beneficial technical effects: The multi-view feature fusion module supplements the focus on features from multiple angles during the feature fusion process, retaining important information from different global forms; the loss constraint and segmentation module is based on a continuous homology continuity constraint loss to better constrain the topological continuity of the segmentation and obtain a significant defect prediction map. Through the feature extraction and multi-view fusion of the topological structure, better accuracy and continuity are provided in the defect segmentation task; it can effectively detect, classify and segment topological defects including but not limited to scratches, bruises, wrinkles, deformations, dirt, scratches, cracks, holes, stutters, solder defects and line defects; and, through the deformable convolution operator, it can achieve extraction that is more in line with the feature geometry, accurately locate and segment these defects, thereby improving the efficiency and accuracy of industrial product quality control; in addition, it can also effectively reduce the labor cost in the industrial production process, and replace the traditional manual inspection through automated defect detection, classification and segmentation processes to achieve dual optimization of cost savings and production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of a method for detecting, classifying and segmenting industrial topological type defects of the present application; Figure 2 It is a network structure framework diagram of the deep neural network in this application; Figure 3 It is a schematic diagram of a two-dimensional transformation of a dynamic deformable convolution to adapt to a topological structure in this application; Figure 4 It is a schematic diagram of the multi-view fusion strategy in this application; Figure 5 is a schematic diagram of a decoder head in this application; Figure 6 It is a training flowchart of a deep neural network. DETAILED DESCRIPTION

[0019] The embodiment of the present application discloses a method for detecting, classifying, and segmenting industrial topological type defects.

[0020] The following is combined with Figure 1-6 This application is described in further detail.

[0021] Reference Figure 1 ,A method for detecting, classifying and segmenting industrial topology type defects includes the following steps: Step S1, using a camera to obtain a part surface defect image.

[0022] Step S2: pre-process the surface defect image to obtain a data set.

[0023] Specifically, all industrial parts defect images are resized to a uniform size and subjected to a variety of data enhancement operations, such as rotation, translation, scaling, flipping, etc. The images are randomly divided into training sets and validation sets according to the specified ratio to obtain the data set.

[0024] At the same time, the above operations are synchronously applied to the segmentation labels of the image to obtain the data set.

[0025] Step S3: During training, each time an image I with a dimension of m×m×c is obtained from the data set img as input.

[0026] Where m represents the image I img Size, c represents image I img The number of channels.

[0027] Specifically, during training, a bunch of images are randomly selected from the dataset each time: one is a surface defect image and the other is the corresponding segmentation label.

[0028] Step S4: construct a deep neural network and transform the image I img As input, a deep neural network is trained.

[0029] During training, image I img Enter the network.

[0030] In this embodiment, the deep neural network includes: a backbone network, a neck, and a decoder head; wherein the backbone network includes a multi-layer feature extraction module, the neck includes a multi-scale feature extraction and fusion module, and the decoder head uses a decoupled head to output predicted bounding boxes, categories, and segmentation masks, which can be used to calculate corresponding losses to support back propagation.

[0031] It should be noted that the present invention improves the upsampling convolution operator for defects with topological structures, and uses dynamic deformable convolution to adaptively focus on the subtle and curved features of the tubular structure, aiming to achieve high-precision structural segmentation.

[0032] like Figure 2 As shown, in this embodiment, the deep neural network is used for training; specifically, after the image is input into the backbone network, it first passes through a layer of convolution with a convolution kernel size of 3, a step size of 2, and a padding of 1, followed by an activation function, and down-sampling is performed using maximum pooling. After that, after passing through several convolution layers and feature fusion module layers, the output is obtained through a fixed pooling module, and the output of the fixed pooling module is input into the neck, and a splicing operation is performed simultaneously through several dynamically deformable convolution layers and feature fusion module layers, and the outputs of different resolutions are respectively input into the three segmentation modules of the decoder head.

[0033] This embodiment takes a two-dimensional image as an example, and the input image size is m×m×3 (m is the model entry size). For the backbone network part, it first passes through a layer of convolution with a convolution kernel size of 3, a step size of 2, and a padding of 1 to obtain a feature map of size m / 2×m / 2×64; after passing through the second convolution layer, a feature map of m / 4×m / 4×128 is obtained; after passing through the first feature fusion module layer, a feature map of m / 4×m / 4×128 is obtained; after passing through the third convolution layer, a feature map of m / 8×m / 8×256 is obtained and input into the second feature fusion module layer; after passing through several identical convolution layers and feature fusion module layers, the output of the fourth feature fusion module layer is finally input into the fixed pooling layer.

[0034] For the neck part, the m / 8×m / 8×256 feature map of the second feature fusion module layer, the m / 16×m / 16×512 feature map of the third feature fusion module layer, and the m / 32×m / 32×512 feature map of the fixed pooling layer are respectively input to the neck part; the m / 16×m / 16×512 feature map is downsampled using CBS, concatenated with the m / 32×m / 32×512 feature map, and output to the P3 segmentation head through the feature fusion module layer; the m / 32×m / 32×512 feature map is downsampled using CBS, concatenated with the m / 32×m / 32×512 feature map, and output to the P3 segmentation head through the feature fusion module layer; After upsampling and multi-view feature fusion using dynamic deformable convolution, the feature map is spliced ​​with the m / 16×m / 16×512 feature map, passed through the feature fusion module layer, and then spliced ​​with the m / 8×m / 8×256 feature map downsampled by the spatial convolution module, and finally input to the P4 segmentation head; the m / 16×m / 16×512 feature map is upsampled and fused with multi-view features using the dynamic deformable convolution layer, spliced ​​with the m / 8×m / 8×256 feature map, passed through the feature fusion module layer and output to the P5 segmentation head.

[0035] Among them, the two-dimensional transformation of dynamic deformable convolution is as follows Figure 3 shown.

[0036] It should be noted that by allowing the model to freely learn deformation offsets, it can more flexibly focus on the complex geometric features of the target. At the same time, each target to be processed is restricted to not deviate from the observation position, thereby ensuring the continuity of attention and preventing the receptive area from spreading too far due to large offsets.

[0037] Among them, multi-view fusion strategies such as Figure 4 shown.

[0038] It should be noted that the present invention focuses on the local features of topological structure and bending, and adopts a multi-view fusion strategy to observe the structural features of surface defects of parts from multiple angles. For a given 2D convolution coordinate M, the center coordinate is Mi = (ui, vi), then a 3×3 convolution kernel M is expressed as: M={(u-1,v-1),(u-1,v),...,(u+1,v+1)} In order to increase the flexibility of convolution, we introduce a deformation offset Δ and adopt an iterative strategy to select the next position of each target to be processed for observation in turn, ensuring the continuity of attention. At the same time, we linearize the standard convolution kernel on the x-axis and y-axis. In the u-axis direction, the specific position of each grid in M ​​is expressed as: Mi±e=(ui±e,vi±e)where e=0,1,2,3,4 represents the horizontal distance from the center grid. In the v-axis direction, the specific position of each grid in M ​​is expressed as: Mj±e=(uj±e,vj±e)e=0,1,2,3,4 represents the horizontal distance from the center grid, where the offset Δ={δ|δ∈[-1,1]} is the amount by which Mi+1 increases relative to Mi.

[0039] In order to ensure that the convolution conforms to the linear structure, the offset needs to be accumulated Σ.

[0040] The change of u-axis direction is: The change in the v-axis direction is: Through two-dimensional changes, the 9×9 receptive field selection range of the dynamic deformable convolution during the deformation process is covered, which can better adapt to the defect topology structure and thus better perceive its key features.

[0041] It should be noted that deformable convolution can adapt to the characteristics of other specific structures through two-dimensional transformation, not just limited to topological structures.

[0042] For a given deformable convolutional multi-view feature fusion strategy, the specific operations are as follows: Among them, P is each convolution kernel, where g m (P u ), g m (P v ) and represent the feature maps extracted by the deformable convolution operator from the u-axis and v-axis respectively.

[0043] In order to eliminate the influence of redundant noise, a random dropout strategy is introduced in the training stage to improve the performance of the model and prevent overfitting. The random dropout strategy sets a random dropout probability p for each feature in each layer of feature map, and p follows the Bernoulli distribution.

[0044] The internal structure of the decoder head is as follows: Figure 5 shown.

[0045] The decoder head includes three branches, namely P5, P4 and P3 branches, and the corresponding feature map resolutions are from high to low; among them, for the P5 branch, the m / 8×m / 8×256 feature map is input and then enters three secondary branches, among which the first secondary branch first passes through two layers of spatial convolution modules, and then is processed by a convolution layer, and finally outputs a feature map of m / 8×m / 8×64, the second secondary branch passes through two layers of spatial convolution modules and a convolution layer and then outputs a feature map of m / 8×m / 8×nc, where nc is the number of training categories, and the third secondary branch passes through two layers of spatial convolution modules and a convolution layer and then outputs a feature map of m / 8×m / 8×32; for the P4 branch, the m / 16×m / 16×256 feature map is input and then enters three secondary branches, among which the first secondary branch first passes through two layers of spatial convolution modules, and then is processed by a convolution layer. The first secondary branch passes through two layers of spatial convolution modules and one convolution layer, and finally outputs a feature map of m / 16×m / 16×64. The second secondary branch passes through two layers of spatial convolution modules and one convolution layer, and outputs a feature map of m / 16×m / 16×nc. The third secondary branch passes through two layers of spatial convolution modules and one convolution layer, and outputs a feature map of m / 16×m / 16×32. For the P3 branch, the m / 32×m / 32×256 feature map is input and enters three secondary branches. Among them, the first secondary branch first passes through two layers of spatial convolution modules, and then passes through a convolution layer, and finally outputs a feature map of m / 32×m / 32×64. The second secondary branch passes through two layers of spatial convolution modules and one convolution layer, and outputs a feature map of m / 32×m / 32×nc. The third secondary branch passes through two layers of spatial convolution modules and one convolution layer, and outputs a feature map of m / 32×m / 32×32.

[0046] In this embodiment, the deep neural network is trained by minimizing the loss.

[0047] It should be noted that the minimization loss comes from the topological continuity constraint loss L, which is equal to half the square of the Euclidean distance between the true value y and the predicted value y^.

[0048] In this embodiment, the number of training iterations and the iteration batch size are set, and the cosine annealing strategy is used to dynamically adjust the learning rate or other methods of adjusting the learning rate. After the number of iterations is completed, the final model is obtained.

[0049] In this embodiment, the deep neural network is trained iteratively.

[0050] In this embodiment, for the trained deep neural network, the intersection over union (IoU), mAP (multi-category average precision), Precision, Recall, and F1 Score are used as indicators of the performance of the model segmentation.

[0051] It should be noted that the intersection over union (IoU) is a set similarity measurement function, which is usually used to calculate the similarity of two samples. The value range is [0, 1]. The better the segmentation effect, the closer the IoU value is to 1. The IoU score expression is Where y is the segmentation label and y^ is the network output. mAP (multi-category average precision) is the most commonly used evaluation metric in target detection, classification, and segmentation. It is used to measure the average detection, classification, and segmentation performance of the model across all categories. Precision is an indicator of how many of the targets predicted by the model as positive samples are truly positive samples. Recall indicates how many of all real targets are correctly detected, classified, and segmented. F1 Score is a weighted average of Precision and Recall. F1 is a good indicator when you need to balance Precision and Recall.

[0052] It should be noted that the trained deep neural network is used for inference testing; specifically, during the test, one image is selected in sequence in the test set as input, and the segmentation label corresponding to the image is also input.

[0053] The present invention discloses a detection, classification, segmentation, classification and segmentation method of industrial topological type defects based on advanced feature extraction of deformable convolution operators, and the method realizes detection, classification and segmentation based on a deep neural network of deformable convolution operators. Specifically, the present invention first uses a high-precision camera to collect images of industrial parts containing defects, and performs multiple data enhancement measures including rotation on these original images to generate a preprocessed data set. Next, the data set will be randomly divided into a training set and a validation set in a user-specified ratio to ensure the effectiveness and reliability of model training and evaluation. In the model training stage, the preprocessed images will be fed into a deep neural network, which is responsible for performing high-level feature extraction and feature fusion tasks. In order to ensure the spatial resolution of the segmentation result, the present invention adopts the feature upsampling technology to restore the extracted features to the size of the original image, and then outputs the final detection, classification and segmentation results through the segmentation head. During the entire training process, the present invention introduces the loss of topological continuity constraints, aiming to optimize the model parameters until the model reaches the best performance state. After the training is completed, the weights of the model will be saved for subsequent applications. In the testing phase, the present invention stipulates that the size of the input image is m×m×c (c is the number of image channels), and uses the previously trained deep neural network for prediction, thereby generating a segmentation map of size m×m. This method particularly emphasizes the capture and integration of the topological structural features of surface defects of industrial parts, aiming to improve the segmentation accuracy and achieve more accurate identification and positioning of defects. In practical applications, the present invention can significantly improve the effects of industrial parts defect detection, classification, and segmentation, and has important practical value and broad application prospects.

[0054] The existing solutions usually only improve the network, but do not make targeted improvements to the target geometry at the component and feature fusion strategy level. Based on the use of deep neural networks, the present invention introduces dynamic deformable convolution and multi-view fusion strategies to enhance the ability to capture topological features, and adopts a random discard strategy to suppress the influence of noise, thereby improving segmentation accuracy.

[0055] The present invention designs a detection, classification and segmentation method based on advanced feature extraction of industrial topological type defects based on deformable convolution operators, which is an implementation method based on the deep learning network theory. The present invention focuses on the effect of extracting topological structure features, improving segmentation accuracy, and adopts the loss of topological continuity constraints to ensure the convergence effect during training and the segmentation accuracy during reasoning.

[0056] like Figure 6 As shown, in this embodiment, the detection, classification, and segmentation method of industrial topology type defects based on advanced feature extraction of deformable convolution operators of the present invention includes the following steps: Step (1), read the configuration file.

[0057] In this embodiment, the configuration file read in step (1) includes hyperparameters such as model task type, original data path, selected model information, model size, data enhancement, training batch, training round, etc.

[0058] Step (2), set the task.

[0059] In this embodiment, the task set in step (2) is determined according to the model task type read in step (1).

[0060] Step (3), clear the working path.

[0061] Step (4), start recording logs.

[0062] Step (5), update the settings file.

[0063] In this embodiment, the setting file updated in step (5) includes various hyperparameter settings required for model training.

[0064] Step (6), update the font data.

[0065] Step (7), read the hyper parameters.

[0066] Step (8): Select the task type.

[0067] Step (9), re-modify the configuration file.

[0068] Step (10), data set preparation.

[0069] In this embodiment, step (10) preprocesses the original image and the label, including data enhancement, and randomly divides the training set and the validation set according to the ratio.

[0070] Step (11), training the deep neural network.

[0071] In this embodiment, the deep neural network includes: a topological structure feature extraction module, a multi-view feature fusion module, and a loss constraint and segmentation module; the topological structure feature extraction module extracts multi-level features from the input image to obtain the topological structure feature information of the defect; wherein, the multi-view feature fusion module supplements the focus on features from multiple angles in the feature fusion process and retains important information from different global forms; the loss constraint and segmentation module is based on a continuity constraint loss of continuous homology to better constrain the topological continuity of the segmentation and obtain a significant defect prediction map.

[0072] In this embodiment, the number of the topological structure feature extraction module and the multi-view feature fusion module is two; the decoder includes three heads, which respectively output results and calculate corresponding losses.

[0073] In this embodiment, the decoding head includes: two spatial convolution layers and one convolution layer.

[0074] Step (11), save the weight file.

[0075] Step (12), test the optimal confidence.

[0076] In this embodiment, different confidence thresholds are set to use the model for reasoning, and the F2 score is used to evaluate the model accuracy performance, and finally the optimal confidence threshold is obtained.

[0077] Step (13), model format conversion, In this embodiment, the generated model weight file is converted into a variety of available model formats and saved.

[0078] In this embodiment, the deep neural network of the training set is used to obtain a trained target neural network model, so as to perform surface defect detection, classification, and segmentation on the target industrial parts based on the target neural network.

[0079] In this embodiment, the step of training the deep neural network using the data set includes the following steps: Step (14), select training images from the data set.

[0080] Step (15), inputting the training image into the deep neural network to train the deep neural network.

[0081] In this embodiment, during the process of training the deep neural network, the deep neural network is trained by minimizing the continuity constraint loss.

[0082] In this embodiment, during the process of training the neural network, the deep neural network is trained through iterative training.

[0083] In this embodiment, the step of training the deep neural network using the data set further includes the following steps: evaluating the deep neural network using an intersection-over-union score.

[0084] In this embodiment, performing surface defect detection, classification, and segmentation on the target industrial parts based on the target neural network includes: inputting the surface defect image of the target industrial parts into the target neural network to enable the target neural network to detect, classify, segment, classify, and segment the surface defects of the target industrial parts; the output of the target neural network is the result of its detection, classification, and segmentation of the surface defects of the target industrial parts.

[0085] It should be noted that the deformable convolution operator proposed in the present invention has a high degree of flexibility and adaptability. Although this embodiment specifically demonstrates how to adapt to the topological structure of strip defects through two-dimensional transformation, this does not limit its scope of application. In fact, the deformable convolution operator can perform a variety of geometric transformations to match various shapes and structural features according to different needs and specific application scenarios. Therefore, the deformable convolution operator in the present invention is not limited to adapting to topological structures. It is a general concept that can be freely adjusted to cope with a variety of morphological changes and is suitable for a wide range of image processing and analysis tasks.

[0086] It should be noted that the working principle of the detection, classification, and segmentation method of industrial topology type defects based on advanced feature extraction of deformable convolution operators provided in this embodiment can be referred to the introduction of the detection, classification, and segmentation method of industrial topology type defects based on advanced feature extraction of deformable convolution operators in the above specific embodiments, and will not be repeated here in detail.

[0087] It should be noted that the protection scope of the detection, classification and segmentation method of industrial topological type defects based on advanced feature extraction of deformable convolution operators described in the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0088] It should be noted that the method for detecting, classifying, and segmenting industrial topology type defects based on advanced feature extraction of deformable convolution operators of the present invention can realize the method for detecting, classifying, and segmenting industrial topology type defects based on advanced feature extraction of deformable convolution operators of the present invention, but the implementation device of the method for detecting, classifying, and segmenting industrial topology type defects based on advanced feature extraction of deformable convolution operators of the present invention includes but is not limited to the structure of the detection, classification, and segmentation system for detecting, classifying, and segmenting industrial topology type defects based on advanced feature extraction of deformable convolution operators listed in this embodiment. All structural deformations and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0089] In summary, the present invention proposes a detection, classification, and segmentation method based on advanced feature extraction of deformable convolution operators, which is specifically used for the detection, classification, and segmentation of industrial parts defects. This method effectively solves the problem of insufficient attention to specific defect geometric structures in the prior art by introducing a topological structure feature extraction module and a multi-view feature fusion module, and greatly improves the accuracy of defect detection, classification, and segmentation. In addition, the present invention uses continuity constraint loss to accelerate the convergence speed of model training, while ensuring that high-standard segmentation accuracy can be achieved in the inference test stage. Therefore, the present invention not only overcomes many limitations of the prior art, but also significantly improves the efficiency and accuracy of industrial parts defect detection, classification, and segmentation through efficient training and testing mechanisms and the application of deep learning technology, showing extremely high industrial application value.

[0090] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for detecting, classifying and segmenting industrial topological defects, characterized in that: include: Topological structure feature extraction module, multi-view feature fusion module, loss constraint and segmentation module; The feature extraction module extracts multi-level features from the input image to obtain the topological structure feature information of the defect; The multi-view feature fusion module supplements the focus on features from multiple perspectives in the feature fusion process, retaining important information from different global modalities; The loss constraint and segmentation module is based on a continuous homology continuity constraint loss to better constrain the topological continuity of the segmentation and obtain a significant defect prediction map.

2. The method for detecting, classifying and segmenting industrial topological defects according to claim 1, characterized in that: The topological structure feature extraction module includes a dynamic deformable convolution operator. The feature extraction module performs two-dimensional changes on the dynamic deformable convolution operator by introducing an offset to focus on the local features of the tubular structure and bending.

3. The method for detecting, classifying and segmenting industrial topological type defects according to claim 1, characterized in that: The multi-view feature fusion module is based on multiple morphological kernel templates generated by dynamic deformable convolution, which can observe the structural characteristics of defects from multiple angles.

4. The method for detecting, classifying and segmenting industrial topological defects according to claim 1, characterized in that: The loss constraint and segmentation module includes a decoupling head module and a topological continuity constraint loss module. The decoupling head module restores the image detail information and improves the spatial resolution through upsampling for accurate segmentation; the topological continuity constraint loss module is used to constrain the continuity of the segmentation, guide the network to focus on the broken areas with abnormal pixels, and realize continuity constraints from a topological perspective.

5. The method for detecting, classifying and segmenting industrial topology type defects according to claim 1, characterized in that: Step 1: The feature extraction module introduces an offset to perform a two-dimensional change on the dynamic deformable convolution operator to focus on the local features of the tubular structure and the bend. The dynamic deformable convolution operator is used to extract multi-level features from the input image to obtain the topological structural feature information of the defect; Step 2: Multi-view features fuse multiple morphological kernel templates generated based on dynamic deformable convolution to observe the structural characteristics of defects from multiple angles; Step 3: The fusion results of each layer processed in step 2 are passed to the decoupling head module for decoding, defect location and segmentation operations, and finally a significant defect prediction map is obtained; Step 4: Calculate the loss value.

6. The method for detecting, classifying and segmenting industrial topological type defects according to claim 5, characterized in that: The specific operation of the two-dimensional change of the dynamic deformable convolution operator in step 1 is as follows: Among them, for a given 2D convolution coordinate M, the center coordinate is Mi = (ui, vi), then a 3×3 convolution kernel M is expressed as: M = {(u-1, v-1), (u-1, v), ..., (u+1, v+1)}, wherein in the u-axis direction, the specific position of each grid in M ​​is expressed as: Mi±e = (ui±e, vi±e) In the v-axis direction, the specific position of each grid in M ​​is expressed as: Mj±e = (uj±e, vj±e) e = 0, 1, 2, 3, 4 represents the horizontal distance from the center grid, wherein the offset Δ = {δ|δ∈[-1, 1]} is the amount by which Mi+1 increases relative to Mi.

7. The method for detecting, classifying and segmenting industrial part defects based on a deformable convolution operator according to claim 5, characterized in that: The specific operation of the multi-view feature fusion strategy for a given deformable convolution in step 2 is as follows: Among them, P is each convolution kernel, where g m (P u ), g m (P v ) and represent the feature maps extracted by the deformable convolution operator from the u-axis and v-axis respectively.

8. The method for detecting, classifying and segmenting industrial topological defects according to claim 5, characterized in that: The random dropout strategy introduced in step 2 is to set a random dropout probability p for each feature in each layer of feature maps.

9. The method for detecting, classifying and segmenting industrial topological defects according to claim 5, characterized in that: The topological continuity constraint loss in step 4 is the difference between the true value y and the predicted value y^.

Images