Method for detecting, classifying, segmenting defects of industrial topology type

Abstract

The application discloses a kind of detection, classification, segmentation method of industrial topological type defect, it is related to deep learning, digital image processing technical field, the method includes topological structure feature extraction module, multi-view feature fusion module, loss constraint and segmentation module;Feature extraction module extracts multistage features to the input image, obtains the topological structure feature information of defect;Multi-view feature fusion module supplements the attention to features from multiple angles in the feature fusion process, retains important information from different global morphologies;Loss constraint and segmentation module are based on a continuous constraint loss of homology, to better constrain the topological continuity of segmentation, and obtain significant defect prediction map.The application can efficiently identify topological type defect.

Description

Technical Field

[0001] This application relates to the fields of deep learning and digital image processing technology, and in particular to a method for detecting, classifying, and segmenting industrial topology defects. Background Technology

[0002] With the rapid advancements in deep learning technology and visual attention models, salient object detection, classification, and segmentation have become a key area of ​​computer vision research. This technology mimics the characteristics of the human visual system through deep learning methods, aiming to efficiently identify and locate specific objects in images or videos. The human visual system's ability to selectively focus on regions of interest based on prior knowledge has inspired the development of salient object detection, classification, and segmentation techniques. Salient object detection, classification, and segmentation are not only crucial for object recognition but also demonstrate enormous application potential in image and video compression, image retrieval, and image retargeting. The basic architecture of modern neural networks can be traced back to 1998, when the backpropagation algorithm was introduced, enabling the network to learn and optimize through backpropagation errors while information is being passed forward.

[0003] In recent years, deep learning models have been widely used in industrial defect detection, classification, and segmentation. For example, advanced feature extraction neural networks have been used for the detection, classification, and segmentation of defects on metal surfaces, effectively improving the accuracy of detection, classification, and segmentation; deep neural networks have demonstrated superior performance in the detection, classification, and segmentation of defects in solar panels; and real-time detection, classification, and segmentation models based on real-time detection, classification, and segmentation frameworks have been applied to the rapid defect screening of packaging materials. The application of these technologies has significantly improved the automation level of industrial production and product quality.

[0004] In the manufacturing process of mobile phones and other digital products, defects inside or on the surface of tiny parts, though seemingly insignificant, can actually have a serious impact on the product's performance and reliability. These defects can not only affect the electrical performance of electronic components, leading to problems such as unstable signal transmission and poor contact, but can also become stress concentration points, reducing the mechanical strength of components and increasing the risk of product failure during use. Therefore, detecting, classifying, and separating these defects in tiny parts is crucial.

[0005] Efficient defect detection, classification, and segmentation not only help manufacturers promptly identify and correct problems in the production process, preventing defective products from entering the market, but also significantly improve product quality and customer satisfaction, thereby enhancing the company's competitiveness. Currently, the mainstream method for defect detection, classification, and segmentation is still manual visual inspection, where operators examine the surface of parts with the naked eye or with the aid of magnifying glasses or microscopes.

[0006] However, manual detection, classification, and segmentation are inefficient, prone to missed detections and false positives due to variations in worker skills and abilities, and incur high operating costs. Although deep learning methods outperform traditional methods in terms of accuracy and efficiency in detection, classification, and segmentation, their application in industry is not yet widespread. Furthermore, the accuracy and efficiency of deep learning in detection, classification, and segmentation need improvement, and the level of intelligence and automation in industrial production needs further advancement. Summary of the Invention

[0007] Purpose of the invention: This application provides a method for detecting, classifying, and segmenting defects in industrial topology, aiming to improve the technical problems mentioned above.

[0008] Technical Solution: This application provides a method for detecting, classifying, and segmenting industrial topology defects, including: a topology feature extraction module, a multi-view feature fusion module, and a loss constraint and segmentation module; the feature extraction module extracts multi-level features from the input image to obtain the topology feature information of the defects; the multi-view feature fusion module supplements the feature fusion process by focusing on features from multiple angles, retaining important information from different global morphologies; the loss constraint and segmentation module is based on a continuous constraint loss of persistent homology to better constrain the topological continuity of the segmentation and obtain a salient defect prediction map.

[0009] Furthermore, the topology feature extraction module includes a dynamic deformable convolution operator. The feature extraction module performs two-dimensional transformations on the dynamic deformable convolution operator by introducing an offset, in order to focus on the local features of tubular structures and bends.

[0010] Furthermore, the multi-view feature fusion module, based on multiple morphological kernel templates generated by dynamic deformable convolution, can observe the structural features of defects from multiple angles.

[0011] Furthermore, the loss constraint and segmentation module includes a decoupling head module and a topological continuity constraint loss module. The decoupling head module recovers the image's detailed information and improves spatial resolution through upsampling to enable accurate segmentation. The topological continuity constraint loss module is used to constrain the continuity of segmentation, guiding the network to focus on broken regions with abnormal pixels, thus achieving continuity constraints from a topological perspective.

[0012] Further, in step one, the feature extraction module introduces an offset to perform two-dimensional transformation on the dynamically deformable convolution operator to focus on the local features of tubular structures and bends. The dynamically deformable convolution operator is used to extract multi-level features from the input image to obtain the topological structural features of the defects. In step two, the multi-view feature fusion integrates multiple morphological kernel templates generated based on the dynamically deformable convolution, allowing observation of the structural features of the defects from multiple angles. In step three, the fusion results processed through each layer in step two are fed into the decoupling head module for decoding, defect localization, and segmentation operations, finally obtaining a salient defect prediction map. In step four, the loss value is calculated.

[0013] Furthermore, the specific operations for the two-dimensional transformation of the dynamically deformable convolution operator in step 1 are as follows:

[0014]

[0015] For a given 2D convolution coordinate M, with center coordinates Mi = (ui, vi), a 3×3 convolution kernel M is represented as: M = {(u-1, v-1), (u-1, v), ..., (u+1, v+1)}. In the u-axis direction, the specific position of each grid in M ​​is represented as: Mi ± e = (ui ± e, vi ± e). In the v-axis direction, the specific position of each grid in M ​​is represented as: Mj ± e = (uj ± e, vj ± e). e = 0, 1, 2, 3, 4 represents the horizontal distance from the center grid, where the offset Δ = {δ | δ ∈ [-1, 1]} is the amount by which Mi+1 increases relative to Mi.

[0016] Furthermore, the specific operation of the multi-view feature fusion strategy for a given deformable convolution in step 2 is as follows:

[0017]

[0018] Where P is each convolution kernel, and g m (P u ), g m (P v ) and represent the feature maps extracted from the u-axis and v-axis by the deformable convolution operator, respectively.

[0019] Furthermore, in step 2, a random discarding strategy is introduced to set a random discarding probability p for each feature in each layer of feature map.

[0020] Furthermore, in step 4, the topological continuity constraint loss is the difference between the true value y and the predicted value y^.

[0021] In summary, this application has at least the following beneficial technical effects:

[0022] The multi-view feature fusion module supplements the feature fusion process by focusing on features from multiple perspectives, retaining important information from different global morphologies. The loss constraint and segmentation module is based on a continuous constraint loss of persistent homology to better constrain the topological continuity of the segmentation and obtain a salient defect prediction map. Through feature extraction of the topological structure and multi-view fusion, better accuracy and continuity are provided in the defect segmentation task. It can effectively detect, classify, and segment topological defects, including but not limited to scratches, dents, wrinkles, deformations, dirt, cracks, holes, knots, solder defects, and circuit defects. Furthermore, through deformable convolution operators, extraction is achieved that is more closely aligned with the feature geometry, accurately locating and segmenting these defects, thereby improving the efficiency and accuracy of industrial product quality control. In addition, it can effectively reduce labor costs in the industrial production process, replacing traditional manual inspection with automated defect detection, classification, and segmentation processes, achieving dual optimization of cost savings and production efficiency. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a method for detecting, classifying, and segmenting industrial topology defects according to this application.

[0024] Figure 2 This is a network structure framework diagram of the deep neural network in this application;

[0025] Figure 3 This is a two-dimensional transformation diagram of the dynamically deformable convolution adapting to the topology in this application;

[0026] Figure 4 This is a schematic diagram of the multi-view fusion strategy in this application;

[0027] Figure 5 This is a schematic diagram of the decoder head in this application;

[0028] Figure 6 This is a flowchart of the training process for a deep neural network. Detailed Implementation

[0029] This application discloses a method for detecting, classifying, and segmenting defects in industrial topology.

[0030] The following is in conjunction with the appendix Figure 1-6 This application will be described in further detail.

[0031] Reference Figure 1 A method for detecting, classifying, and segmenting defects in industrial topology includes the following steps:

[0032] Step S1: Use a camera to acquire images of surface defects on the part.

[0033] Step S2: Preprocess the surface defect image to obtain the dataset.

[0034] Specifically, all images of defective industrial parts were resized to a uniform size and subjected to various data augmentation operations, such as rotation, translation, scaling, and flipping. The dataset was then randomly divided according to a specified ratio of images in the training and validation sets.

[0035] Simultaneously, the above operations are applied to the segmentation labels of the images to obtain the dataset.

[0036] Step S3: During training, each time an image I of dimension m×m×c is obtained from the dataset. img As input.

[0037] Where m represents image I img Size, c represents image I img The number of channels.

[0038] Specifically, during training, a set of images is randomly selected from the dataset each time: one is a surface defect image, and the other is the corresponding segmentation label.

[0039] Step S4: Construct a deep neural network and then process the image I. img As input, train a deep neural network.

[0040] During training, image I img Input into the network.

[0041] In this embodiment, the deep neural network includes a backbone network, a neck, and a decoder head; wherein the backbone network includes a multi-layer feature extraction module, the neck includes a multi-scale feature extraction and fusion module, and the decoder head outputs the predicted bounding box, category, and segmentation mask in the form of a decoupled head, which can be used to calculate the corresponding loss to support backpropagation.

[0042] It should be noted that this invention improves the upsampling convolution operator to address the defects of topological structures, and uses dynamic deformable convolution to adaptively focus on the subtle and curved features of tubular structures, aiming to achieve high-precision structural segmentation.

[0043] like Figure 2As shown, in this embodiment, the deep neural network is used for training. Specifically, after the image is input into the backbone network, it first passes through a convolutional layer with a kernel size of 3, a stride of 2, and padding of 1. Then, an activation function is applied, and downsampling is performed using max pooling. After passing through several convolutional layers and feature fusion module layers, the output is obtained through a fixed pooling module. The output of the fixed pooling module is input into the neck, and a splicing operation is performed simultaneously through several dynamically deformable convolutional layers and feature fusion module layers. The outputs of different resolutions are then input into the three segmentation modules of the decoder head.

[0044] This embodiment takes a two-dimensional image as an example. The input image size is m×m×3 (m is the model entry size). For the backbone network, it first goes through a convolutional layer with a kernel size of 3, a stride of 2, and padding of 1 to obtain a feature map of size m / 2×m / 2×64. After the second convolutional layer, a feature map of size m / 4×m / 4×128 is obtained. After the first feature fusion module layer, a feature map of size m / 4×m / 4×128 is obtained. After the third convolutional layer, a feature map of size m / 8×m / 8×256 is obtained and input into the second feature fusion module layer. After several identical convolutional layers and feature fusion module layers, the output of the fourth feature fusion module layer is finally input into a fixed pooling layer.

[0045] For the neck region, the m / 8×m / 8×256 feature map from the second feature fusion module layer, the m / 16×m / 16×512 feature map from the third feature fusion module layer, and the m / 32×m / 32×512 feature map from the fixed pooling layer are input to the neck region respectively; the m / 16×m / 16×512 feature map is downsampled using CBS and then concatenated with the m / 32×m / 32×512 feature map, and output through the feature fusion module layer to the P3 segmentation head; the m / 32×m / 32×512 feature map is then... After upsampling and fusing with multi-view features using dynamic deformable convolution, the feature map is concatenated with an m / 16×m / 16×512 feature map and then passed through a feature fusion module layer. Finally, it is concatenated with an m / 8×m / 8×256 feature map after downsampling by a spatial convolution module and input to the P4 segmentation head. After upsampling and fusing with multi-view features using dynamic deformable convolution layers, the feature map is concatenated with an m / 8×m / 8×256 feature map and then passed through a feature fusion module layer and output to the P5 segmentation head.

[0046] Among them, the two-dimensional transformation of dynamically deformable convolution is as follows: Figure 3 As shown.

[0047] It should be noted that by allowing the model to freely learn deformation offsets, it is possible to focus more flexibly on the complex geometric features of the target. At the same time, restricting each target to be processed to stay within the observation position ensures the continuity of attention and prevents the receptive area from spreading too far due to large offsets.

[0048] Among them, multi-view fusion strategies such as Figure 4 As shown.

[0049] It should be noted that this invention focuses on the local features of topology and curvature, while employing a multi-view fusion strategy to observe the structural features of surface defects of the part from multiple angles. Specifically, for a given 2D convolutional coordinate M, with center coordinates Mi = (ui, vi), a 3×3 convolutional kernel M is represented as:

[0050] M={(u-1,v-1),(u-1,v),...,(u+1,v+1)}

[0051] To increase the flexibility of convolution, we introduce a deformation offset Δ and employ an iterative strategy, sequentially selecting the next position of each target to be processed for observation, ensuring continuity of focus. Simultaneously, we linearize the standard convolution kernel along the x and y axes. In the u-axis direction, the specific position of each grid in M ​​is represented as: Mi±e=(ui±e,vi±e), where e=0,1,2,3,4 represents the horizontal distance from the center grid. In the v-axis direction, the specific position of each grid in M ​​is represented as: Mj±e=(uj±e,vj±e), where e=0,1,2,3,4 represents the horizontal distance from the center grid, and the offset Δ={δ|δ∈[-1,1]} is the increase in Mi+1 relative to Mi.

[0052] To ensure that the convolution conforms to a linear structure, the offset needs to be accumulated by Σ.

[0053] The change in the direction of the u-axis is as follows:

[0054]

[0055] The change in the v-axis direction is as follows:

[0056]

[0057] By employing two-dimensional transformations, the 9×9 receptive field selectability range of dynamically deformable convolutions during deformation is covered. This allows for better adaptation to defective topologies, thereby improving the perception of their key features.

[0058] It should be noted that deformable convolution can adapt to the characteristics of other specific structures through two-dimensional transformations, and is not limited to topological structures.

[0059] For a given deformable convolutional multi-view feature fusion strategy, the specific operation is as follows:

[0060]

[0061] Where P is each convolution kernel, and g m (P u ), g m (P v ) and represent the feature maps extracted from the u-axis and v-axis by the deformable convolution operator, respectively.

[0062] To eliminate the influence of redundant noise, a random dropout strategy is introduced during the training phase to improve model performance and prevent overfitting. The random dropout strategy sets a random dropout probability p for each feature in each layer of feature maps, where p follows a Bernoulli distribution.

[0063] The internal structure of the decoder head is as follows: Figure 5 As shown.

[0064] The decoder head consists of three branches: P5, P4, and P3, with corresponding feature map resolutions decreasing from high to low. For the P5 branch, an m / 8×m / 8×256 feature map is input and then fed into three sub-branches. The first sub-branch first processes the map through two spatial convolutional layers followed by one convolutional layer, outputting an m / 8×m / 8×64 feature map. The second sub-branch processes the map through two spatial convolutional layers and one convolutional layer, outputting an m / 8×m / 8×nc feature map, where nc is the number of training classes. The third sub-branch processes the map through two spatial convolutional layers and one convolutional layer, outputting an m / 8×m / 8×32 feature map. For the P4 branch, an m / 16×m / 16×256 feature map is input and then fed into three sub-branches. The first sub-branch first processes the map through two spatial convolutional layers followed by one convolutional layer. The first sub-branch processes the data through two spatial convolutional modules and one convolutional layer, outputting an m / 16×m / 16×64 feature map. The second sub-branch processes the data through two spatial convolutional modules and one convolutional layer, outputting an m / 16×m / 16×nc feature map. The third sub-branch processes the data through two spatial convolutional modules and one convolutional layer, outputting an m / 16×m / 16×32 feature map. For the P3 branch, the m / 32×m / 32×256 feature map is input and then enters three sub-branches. The first sub-branch processes the data through two spatial convolutional modules and one convolutional layer, outputting an m / 32×m / 32×64 feature map. The second sub-branch processes the data through two spatial convolutional modules and one convolutional layer, outputting an m / 32×m / 32×nc feature map. The third sub-branch processes the data through two spatial convolutional modules and one convolutional layer, outputting an m / 32×m / 32×32 feature map.

[0065] In this embodiment, a deep neural network is trained by minimizing the loss.

[0066] It should be noted that the minimized loss comes from the topological continuity constraint loss L, which is equal to half the square of the Euclidean distance between the difference between the true value y and the predicted value y^.

[0067] In this embodiment, the number of training iterations and the batch size are set. The learning rate is dynamically adjusted using a cosine annealing strategy or other methods to adjust the learning rate. After completing the number of iterations, the final model is obtained.

[0068] In this embodiment, the deep neural network is trained iteratively.

[0069] In this embodiment, for a trained deep neural network, the Intersection over Union (IoU), mAP (multi-class average precision), Precision, Recall, and F1 Score are used as indicators of the model's segmentation performance.

[0070] It should be noted that the Intersection over Union (IoU) coefficient is a set similarity measure, commonly used to calculate the similarity between two samples. The value ranges from [0, 1], and the better the segmentation, the closer the IoU value is to 1. The IoU scoring expression is as follows: Where y is the segmentation label, and y^ is the network output. mAP (Multi-class Average Precision) is the most commonly used evaluation metric for object detection, classification, and segmentation, used to measure the model's average detection, classification, and segmentation performance across all classes. Precision indicates how many of the targets predicted as positive by the model are actually positive. Recall indicates how many of all true targets are correctly detected, classified, and segmented. F1 Score is a weighted average of Precision and Recall. When it is necessary to balance Precision and Recall, F1 is a good metric.

[0071] It should be noted that the inference test is performed using a pre-trained deep neural network; specifically, during the test, one image is selected sequentially from the test set as input, and the corresponding segmentation label for each image is also input.

[0072] This invention discloses an advanced feature extraction method for detecting, classifying, and segmenting industrial topological defects based on deformable convolution operators. This method utilizes a deep neural network with deformable convolution operators to achieve detection, classification, and segmentation. Specifically, this invention first uses a high-precision camera to acquire images of industrial parts containing defects and performs various data augmentation measures, including rotation, on these original images to generate a preprocessed dataset. Next, this dataset is randomly divided into training and validation sets at a user-specified ratio to ensure the effectiveness and reliability of model training and evaluation. During model training, the preprocessed images are fed into a deep neural network responsible for performing high-level feature extraction and feature fusion tasks. To ensure the spatial resolution of the segmentation results, this invention employs feature upsampling technology to restore the extracted features to the original image size, and then outputs the final detection, classification, and segmentation results through a segmentation head. Throughout the training process, this invention introduces a loss based on topological continuity constraints to optimize model parameters until the model reaches its optimal performance state. After training, the model weights are saved for subsequent applications. During the testing phase, this invention specifies the input image size as m×m×c (where c is the number of image channels). A previously trained deep neural network is used for prediction to generate a segmentation map of size m×m. This method particularly emphasizes the capture and integration of the topological features of surface defects on industrial parts, aiming to improve segmentation accuracy and achieve more precise defect identification and localization. In practical applications, this invention can significantly improve the effectiveness of defect detection, classification, and segmentation in industrial parts, demonstrating significant practical value and broad application prospects.

[0073] Existing solutions typically only improve the network without addressing the specific geometric shape of the target object through component and feature fusion strategies. This invention, based on a deep neural network, introduces dynamically deformable convolution and a multi-view fusion strategy to enhance the capture of topological features, and employs a random dropout strategy to suppress the influence of noise, thereby improving segmentation accuracy.

[0074] This invention designs an advanced feature extraction method for detection, classification, and segmentation based on deformable convolution operators for defects in industrial topology types. It is an implementation method grounded in deep learning networks. This invention focuses on improving the extraction effect of topological features to enhance segmentation accuracy. Simultaneously, it employs a loss function constrained by topological continuity to ensure convergence during training and segmentation accuracy during inference.

[0075] like Figure 6 As shown, in this embodiment, the detection, classification, and segmentation method for industrial topology defects based on advanced feature extraction using deformable convolution operators of the present invention includes the following steps:

[0076] Step (1): Read the configuration file.

[0077] In this embodiment, the configuration file content read in step (1) includes hyperparameters such as model task type, original data path, selected model information, model size, data augmentation, training batch, and training epoch.

[0078] Step (2): Set up the task.

[0079] In this embodiment, the task set in step (2) is determined according to the model task type read in step (1).

[0080] Step (3): Clear the working path.

[0081] Step (4): Start recording logs.

[0082] Step (5): Update the settings file.

[0083] In this embodiment, the settings file updated in step (5) contains various hyperparameter settings required for model training.

[0084] Step (6): Update font data.

[0085] Step (7): Read the hyperparameters.

[0086] Step (8): Select the task type.

[0087] Step (9): Modify the configuration file again.

[0088] Step (10): Dataset creation.

[0089] In this embodiment, step (10) preprocesses the original image with labels, including data augmentation, and randomly divides the training set and validation set according to the ratio.

[0090] Step (11): Train the deep neural network.

[0091] In this embodiment, the deep neural network includes: a topology feature extraction module, a multi-view feature fusion module, and a loss constraint and segmentation module. The topology feature extraction module extracts multi-level features from the input image to obtain the topology feature information of the defect. The multi-view feature fusion module supplements the feature fusion process by focusing on features from multiple angles, retaining important information from different global forms. The loss constraint and segmentation module is based on a continuous constraint loss of persistent homology to better constrain the topological continuity of the segmentation and obtain a salient defect prediction map.

[0092] In this embodiment, there are two modules: the topology feature extraction module and the multi-view feature fusion module; the decoder contains three heads, which output results and calculate corresponding losses respectively.

[0093] In this embodiment, the decoding head includes two spatial convolutional layers and one convolutional layer.

[0094] Step (11): Save the weight file.

[0095] Step (12): Test the optimal confidence level.

[0096] In this embodiment, the model is used for inference by setting different confidence thresholds, and the F2 score is used to evaluate the accuracy performance of the model, so as to obtain the optimal confidence threshold.

[0097] Step (13), Model format conversion,

[0098] In this embodiment, the generated model weight file is converted into various usable model formats and saved.

[0099] In this embodiment, the deep neural network in the training set is used to obtain a trained target neural network model, so as to perform surface defect detection, classification and segmentation on the target industrial parts based on the target neural network.

[0100] In this embodiment, training the deep neural network using the dataset includes the following steps:

[0101] Step (14): Select training images from the dataset.

[0102] Step (15): Input the training image into the deep neural network to train the deep neural network.

[0103] In this embodiment, the deep neural network is trained by minimizing the continuity constraint loss.

[0104] In this embodiment, the deep neural network is trained through iterative training during the training process.

[0105] In this embodiment, training the deep neural network using the dataset further includes the following step: evaluating the deep neural network using the intersection-union score.

[0106] In this embodiment, the surface defect detection, classification, and segmentation of the target industrial part based on the target neural network includes: inputting the surface defect image of the target industrial part into the target neural network to realize the detection, classification, and segmentation of the surface defects of the target industrial part by the target neural network; the output of the target neural network is the result of its detection, classification, and segmentation of the surface defects of the target industrial part.

[0107] It should be noted that the deformable convolution operator proposed in this invention possesses high flexibility and adaptability. Although this embodiment specifically demonstrates how it adapts to the topological structure of strip-shaped defects through two-dimensional transformation, this does not limit its application scope. In fact, this deformable convolution operator can perform diverse geometric transformations according to different needs and specific application scenarios to match various shapes and structural features. Therefore, the deformable convolution operator in this invention is not limited to adapting to topological structures; it is a general concept that can be freely adjusted to cope with various morphological changes and is applicable to a wide range of image processing and analysis tasks.

[0108] It should be noted that the working principle of the detection, classification, and segmentation method for industrial topology defects based on advanced feature extraction using deformable convolution operators provided in this embodiment can be referred to the description of the detection, classification, and segmentation method for industrial topology defects based on advanced feature extraction using deformable convolution operators in the above specific embodiments, and will not be repeated in detail here.

[0109] It should be noted that the protection scope of the advanced feature extraction detection, classification and segmentation method for industrial topology defects based on deformable convolution operators described in this invention is not limited to the execution order of the steps exemplified in this embodiment. Any scheme implemented by adding, subtracting or replacing steps in the prior art based on the principles of this invention is included within the protection scope of this invention.

[0110] It should be noted that the detection, classification, and segmentation method for industrial topology defects based on advanced feature extraction using deformable convolution operators of the present invention can implement the detection, classification, and segmentation method for industrial topology defects based on advanced feature extraction using deformable convolution operators of the present invention. However, the implementation device for the detection, classification, and segmentation method for industrial topology defects based on advanced feature extraction using deformable convolution operators of the present invention includes, but is not limited to, the structure of the detection, classification, and segmentation system for the detection, classification, and segmentation method for industrial topology defects based on advanced feature extraction using deformable convolution operators of the present invention. Any structural modifications and substitutions of the prior art made according to the principles of the present invention are included within the protection scope of the present invention.

[0111] In summary, this invention proposes an advanced feature extraction method for detection, classification, and segmentation based on deformable convolution operators, specifically designed for the detection, classification, and segmentation of defects in industrial parts. This method effectively addresses the problem of insufficient attention to the geometric structure of specific defects in existing technologies by introducing a topological feature extraction module and a multi-view feature fusion module, significantly improving the accuracy of defect detection, classification, and segmentation. Furthermore, this invention employs continuity constraint loss to accelerate the convergence speed of model training while ensuring high segmentation accuracy during the inference and testing phase. Therefore, this invention not only overcomes many limitations of existing technologies but also significantly improves the efficiency and accuracy of defect detection, classification, and segmentation in industrial parts through efficient training and testing mechanisms and the application of deep learning technology, demonstrating extremely high industrial application value.

[0112] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for detecting, classifying, and segmenting industrial topology defects, characterized in that, include: The module includes a topology feature extraction module, a multi-view feature fusion module, and a loss constraint and segmentation module. The feature extraction module extracts multi-level features from the input image to obtain the topological structural features of the defects. The multi-view feature fusion module supplements the feature fusion process by focusing on features from multiple angles, retaining important information from different global morphologies. The loss constraint and segmentation module is based on a continuous constraint loss of persistent homology to better constrain the topological continuity of the segmentation and obtain a salient defect prediction map. The topological feature extraction module includes a dynamically deformable convolution operator. The feature extraction module introduces an offset to perform two-dimensional transformation on the dynamically deformable convolution operator to focus on the local features of tubular structures and bends. The multi-view feature fusion module is based on multiple morphological kernel templates generated by the dynamically deformable convolution, which can observe the structural features of the defects from multiple angles. The loss constraint and segmentation module includes a decoupling head module and a topological continuity constraint loss module. The decoupling head module recovers the image's detailed information and improves spatial resolution through upsampling to enable accurate segmentation. The topological continuity constraint loss module is used to constrain the continuity of the segmentation, guiding the network to focus on broken regions with abnormal pixels, thus achieving continuity constraints from a topological perspective. Step 1: The feature extraction module performs a two-dimensional transformation on the dynamically deformable convolution operator by introducing an offset, focusing on the local features of tubular structures and bends. The dynamically deformable convolution operator is used to extract multi-level features from the input image to obtain the topological structural features of the defects. The specific operation of the two-dimensional transformation of the dynamically deformable convolution operator in Step 1 is as follows: For a given two-dimensional convolution coordinate, the center coordinate is M. i = (u i , v i A 3×3 convolution kernel M is represented as: M = {(u-1,v-1), (u-1, v), ..., (u+1, v+1)}. Linearizing the standard convolution kernel along the x and y axes, the specific position of each grid cell in M ​​along the u-axis is represented as: M i±e = (u i±e , v i±e In the v-axis direction, the specific location of each grid in M ​​is represented as: M j±e = (u j±e , v j±e ); e = 0,1,2,3,4 represents the horizontal distance from the center grid, where the deformation offset Δ = {δ | δ ∈ [-1,1]} is M i+1 The amount of increase relative to Mi; the accumulated deformation offset Σ; The change in the u-axis direction is as follows: The change in the v-axis direction is as follows: Step two involves fusing multiple morphological kernel templates generated based on dynamically deformable convolutions to observe the structural features of defects from multiple angles. The specific operation of the multi-view feature fusion strategy for a given deformable convolution in step two is as follows: Among them, g m (P u ) and g m (P v ) represent the feature maps extracted from the u-axis and v-axis by the deformable convolution operator, respectively; Step 3: The fusion results that have undergone processing at each layer in Step 2 are passed into the decoupling head module for decoding, defect localization and segmentation operations, and finally the salient defect prediction map is obtained. Step 4: Calculate the loss value.

2. The method for detecting, classifying, and segmenting industrial topology defects according to claim 1, characterized in that, Step two introduces a random discarding strategy, setting a random discarding probability p for each feature in each layer of the feature map.

3. The method for detecting, classifying, and segmenting industrial topology defects according to claim 2, characterized in that, In step four, the topological continuity constraint loss is the difference between the true value y and the predicted value y^.

Images