Weakly supervised pipe surface defect detection method and system based on generative network

Abstract

The application discloses a weakly supervised pipe surface defect detection method and system based on a generative network, collects surface defect feature information of a pipe to be detected to form an original data set, expands the original data set through an image enhancement technology, divides the expanded data into a training set and a test set, constructs a semantic segmentation network HRNet comprising a feature extraction module, a feature segmentation module and a semantic segmentation module, and introduces a deformable large kernel attention mechanism, iteratively trains the semantic segmentation network HRNet using the training set and the test set, saves optimal weights of the semantic segmentation network HRNet, initializes the semantic segmentation network HRNet by using the optimal weights, and realizes defect detection on a pipe surface by using the initialized semantic segmentation network HRNet. The application improves the precision and efficiency of pipe defect detection, reduces the dependence on accurate pixel-level labeled data, and has significant technical advantages and application value.

Description

Technical Field

[0001] This invention belongs to the field of equipment testing technology, specifically relating to a weakly supervised method and system for detecting surface defects in pipes based on generative networks. Background Technology

[0002] In the pipe manufacturing industry, accurate identification and classification of surface defects are crucial for ensuring product quality. However, the classic semantic segmentation network method, which classifies each pixel in an image into regions with different semantic meanings, typically relies on a large amount of precise pixel-level labeled data. This data acquisition is costly and very time-consuming.

[0003] To address the data annotation problem, researchers have proposed weakly supervised methods for training semantic segmentation networks. These methods aim to perform semantic segmentation tasks using simpler and more readily available annotation information, such as image-level labels, bounding boxes, point annotations, or the contours of objects in images. These weakly supervised methods have achieved significant results in reducing annotation costs and time, making semantic segmentation possible even with limited annotation resources. However, they typically struggle to match the accuracy and detail of fully supervised methods.

[0004] A key challenge in weakly supervised semantic segmentation is extracting sufficient information from these coarse annotations for accurate pixel-level classification. Existing methods utilize class activation maps (CAMs) generated by classification networks to obtain the approximate location of target classes in the image, thereby generating initial salient mask regions. However, this approach typically requires pre-training the classification network on large datasets, and due to the limited resolution of CAMs and their focus on the most salient feature regions, they often fail to capture details and edge information in the image, resulting in insufficient performance for detecting small target defects. Summary of the Invention

[0005] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing a weakly supervised method and system for detecting surface defects in pipes based on generative networks. This method replaces the existing weakly supervised method of obtaining salient mask regions through class activation maps of classification networks with a generative network, thus avoiding reliance on large datasets for training the classification network. This addresses the technical problem of insufficient industrial defect data leading to poor mask region localization during training, discontinuous output mask regions, and inability to obtain accurate edge information of the target object. Furthermore, considering the small size of pipe surface defect targets, a deformable convolutional large-kernel attention module is introduced to improve the semantic segmentation network, thereby enhancing the detection performance of pipe surface defects.

[0006] The present invention adopts the following technical solution:

[0007] A weakly supervised method for detecting surface defects in pipes based on generative networks includes the following steps:

[0008] S1. Collect surface defect feature information of the pipe to be inspected to form the original dataset;

[0009] S2. The original dataset obtained in step S1 is augmented using image enhancement techniques, and the augmented data is divided into a training set and a test set.

[0010] S3. Construct the semantic segmentation network HRNet and introduce a deformable large kernel attention mechanism;

[0011] S4. Use the training set and test set obtained in step S2 to iteratively train the semantic segmentation network HRNet obtained in step S3, save the optimal weights of the semantic segmentation network HRNet, initialize the semantic segmentation network HRNet with the optimal weights, and use the initialized semantic segmentation network HRNet to realize defect detection on the surface of the pipe.

[0012] Preferably, step S1 specifically includes:

[0013] The surface defect feature information of the pipe to be detected is filtered by the sliding window method, the background area is removed, and a surface defect slice of size 256×256 is uniformly generated from each pipe surface image. Based on prior knowledge, a corresponding image-level category label is added to each pipe surface image. The category label of each image corresponds to its defect type, which constitutes the original dataset.

[0014] Preferably, in step S2, 80% of the original dataset is divided into a training set and 20% into a test set.

[0015] Preferably, in step S2, the image enhancement techniques include rotation, scaling, and flipping.

[0016] Preferably, in step S3, the semantic segmentation network HRNet includes:

[0017] The feature extraction module uses ResNet18 as the backbone network for feature extraction to extract features from the input image.

[0018] The feature segmentation module uses Fastflow, a generative network based on a flow model, to quickly and efficiently extract features from the input image and perform preliminary feature segmentation, providing a salient mask region for locating defects.

[0019] The semantic segmentation module uses the HRNet network to enhance the capture capability by preserving the feature map and then performs semantic segmentation.

[0020] More preferably, the loss function of the feature extraction module is:

[0021]

[0022] Where N is the number of samples in the batch; M is the product of the dimensions of each latent variable; z ij It is the j-th latent variable element of the i-th sample; J i It is the Jacobian determinant of the i-th sample.

[0023] More preferably, in the feature segmentation module, the specific steps of the generative network Fastflow are as follows:

[0024] The feature map X is regarded as a two-dimensional data distribution. It is transformed into a standard normal distribution f:X→Z using a two-dimensional normalization flow, resulting in an H×W×D dimensional latent space matrix Z.

[0025] Calculate the Jacobian determinant of the transformation from X to Z to obtain an H×W dimensional Jacobian matrix J;

[0026] Calculate the probability density at each position in Z to obtain an H×W dimensional probability matrix P;

[0027] Multiplying J and P together yields an H×W dimensional anomaly matrix A;

[0028] By concatenating A and X, we obtain an output matrix Y of H×W×(D+1) dimensions that contains both the original features and the anomalous features.

[0029] Preferably, the loss function L of the semantic segmentation network HRNet is... total Specifically:

[0030] L total =αL CE +βL Dice

[0031] Where α and β are the weighting coefficients used to adjust the cross-entropy loss and Dice loss in the total loss, L CE For cross-entropy loss, L Dice This is a loss for Dice.

[0032] Preferably, step S4 specifically includes:

[0033] The generative network Fastflow in the feature segmentation module is trained using the training set, and the network parameters are updated in each iteration.

[0034] The semantic segmentation network HRNet is initialized, and the semantic segmentation network is further adjusted and optimized using samples and saliency mask regions generated by the generative network.

[0035] Save the optimal weights of the semantic segmentation network based on the best accuracy on the test set;

[0036] The semantic segmentation network HRNet is initialized with optimal weights to detect the surface of pipes and identify the location and type of defects.

[0037] Secondly, embodiments of the present invention provide a weakly supervised pipe surface defect detection system based on generative networks, comprising:

[0038] The data module collects surface defect feature information of the pipe to be inspected, forming the original dataset;

[0039] The expansion module expands the original dataset obtained from the data module using image enhancement technology, and divides the expanded data into training and testing sets.

[0040] The network module constructs a semantic segmentation network HRNet, which includes a feature extraction module, a feature segmentation module, and a semantic segmentation module, and introduces a deformable large kernel attention mechanism.

[0041] The detection module uses the training and test sets obtained from the expansion module to iteratively train the semantic segmentation network HRNet obtained from the network module, saves the optimal weights of the semantic segmentation network HRNet, and initializes semantic segmentation using the optimal weights.

[0042] Thirdly, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the aforementioned weakly supervised pipe surface defect detection method based on generative networks.

[0043] Fourthly, embodiments of the present invention provide an electronic device, including a computer program, which, when executed by the electronic device, implements the steps of the above-described weakly supervised pipe surface defect detection method based on generative networks.

[0044] Compared with existing technologies, this invention has at least the following advantages: A weakly supervised pipe surface defect detection method based on generative networks utilizes a generative network to generate salient mask regions, aiming to effectively detect defects on the pipe surface. This method employs a pipeline composed of modules such as generative networks and high-level semantic segmentation networks, where the input of each subsequent model depends on the output of the previous model, performing iterative training to improve detection accuracy and efficiency while reducing the need for large amounts of labeled data.

[0045] Furthermore, in step S1, the sliding window method is used to filter and remove background areas, allowing for focused analysis of key areas containing defects. This helps reduce interference in subsequent processing and improves detection accuracy. Generating defect slices of uniform size and adding image-level category labels helps standardize the training process and simplifies the learning process for the network, enabling it to focus more on the defects themselves rather than the surrounding environment.

[0046] Furthermore, image enhancement techniques such as rotation, scaling, and flipping can significantly increase the diversity and robustness of the dataset. This technique allows the model to be exposed to images with various transformations during training, thereby enhancing its ability to detect defects at different angles and scales, and improving the model's generalization ability. Furthermore, HRNet is designed to maintain high-resolution feature maps throughout the network, improving the detection of fine defects. By combining the ResNet18 feature extraction module and the Fastflow feature segmentation module, image features can be efficiently extracted and processed, providing strong data support for the final defect identification and classification. Furthermore, the loss function of the feature extraction module considers the latent variable dimension and Jacobian determinant of the samples, helping to maintain the integrity of information during feature transformation and ensuring effective representation of features during learning. This loss function is beneficial for optimizing feature extraction efficiency and improving the accuracy of subsequent defect identification. Furthermore, the Fastflow network effectively generates anomaly matrices by transforming feature maps to a standard normal distribution and calculating the Jacobian matrix and probability density, which can highlight potential defect regions. This method enhances the network's sensitivity to anomalous features, helping to improve detection accuracy and response speed. Furthermore, the semantic segmentation network HRNet combines cross-entropy loss and Dice loss in its loss function. This combination helps to simultaneously optimize classification accuracy and region overlap. This design ensures that the network can not only identify the presence of defects but also accurately locate defect regions, improving overall detection performance.

[0047] Furthermore, in step S4, the generative networks Fastflow and HRNet are trained iteratively, and optimal weights are initialized, which helps to quickly achieve the best performance of the algorithm. This training strategy can ensure that the model reduces the risk of missed detections and false detections caused by improper model tuning in practical applications.

[0048] It is understandable that the beneficial effects of the second aspect mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0049] In summary, this invention can reduce the reliance on large defect datasets while improving the detection accuracy and classification performance of small target defects.

[0050] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0051] Figure 1 This is a flowchart of the method of the present invention;

[0052] Figure 2 The images show surface defects of the pipe generated by the sliding window method, where (a) is an indentation defect, (b) is a scratch defect, and (c) is an oxidation corrosion defect.

[0053] Figure 3 This is a flowchart illustrating the joint training process of the method of this invention on the MVTec dataset;

[0054] Figure 4 A heatmap is provided to visualize the inference process on the MVTec dataset, where (a) is the HRNet network and (b) is the method of this invention.

[0055] Figure 5 This is a diagram showing the results of detecting surface defects in pipes using the method of the present invention.

[0056] Figure 6 A schematic diagram of a computer device provided in an embodiment of the present invention;

[0057] Figure 7 This is a block diagram of a chip provided according to an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0060] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0061] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0062] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0063] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0064] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0065] This invention provides a weakly supervised method for detecting defects on pipe surfaces based on generative networks. The method detects the location of defects contained in the input pipe surface image and determines the type of defects contained therein.

[0066] Please see Figure 1 This invention discloses a weakly supervised method for detecting surface defects in pipes based on generative networks, comprising the following steps:

[0067] S1. Collect surface defect feature information of the pipe to be inspected to form the original dataset;

[0068] Please see Figure 2The surface defect feature information of the pipe to be detected is filtered by the sliding window method to remove a large number of background areas, and a batch of surface defect slices with a size of 256×256 are uniformly generated from each pipe surface image. Since the images are all color images, the number of channels of the images is c=3. In the collected sample images, the defect type in each pipe surface image is known. Based on prior knowledge, a corresponding image-level category label is added to each pipe surface image. The category label of each image corresponds to its defect type, which constitutes the original dataset.

[0069] S2. Expand the original dataset using image enhancement technology, and divide the expanded data into a training set and a test set, with a ratio of 80% for the training set and 20% for the test set;

[0070] Image enhancement techniques include methods such as rotation, scaling, and flipping.

[0071] When renaming the dataset augmented with image enhancement techniques, ensure that the category label of each enhanced image remains consistent with that of the original image. This ensures that there is no overlap between the training and test sets.

[0072] S3. This invention proposes a weakly supervised defect detection method, which constructs a semantic segmentation network HRNet that includes a feature extraction module, a feature segmentation module, and a semantic segmentation module. Through the collaborative work of the modules, a defect detection process from coarse to fine is realized.

[0073] Please see Figure 3 The specific architecture of the network model is as follows:

[0074] 1. Feature Extraction Module: Its main function is to extract rich feature information from the input pipe surface image. This step is the foundation for subsequent recognition and classification tasks. ResNet18 (Residual Network) is used as the backbone network for feature extraction. ResNet18 solves the degradation problem in deep network training by introducing a residual learning framework, enabling the network to maintain or improve accuracy while increasing network depth.

[0075] The feature segmentation module aims to quickly and efficiently extract features from the input image and perform preliminary segmentation on these features, providing salient mask regions for locating defects. Therefore, this module is implemented using the Fastflow generative network based on a flow model. The network's loss function is based on the logarithm of the latent variables and the Jacobian determinant, and the formula for the loss function is as follows:

[0076]

[0077] In this loss function, the first term Regularize the latent variables, and the second term -log|Ji The Jacobian determinant allows the model to learn the inverse transformation.

[0078] Where L is the overall loss function of the Fastflow network; N is the number of samples in the batch, i.e., the number of elements in the latent variables and the Jacobian determinant; M is the product of the dimensions of each latent variable; z ij It is the j-th latent variable element of the i-th sample; J i It is the Jacobian determinant of the i-th sample.

[0079] 2. Feature segmentation module: Fastflow, a generative network based on a flow model, is used to quickly and effectively extract features from the input image and perform preliminary feature segmentation, providing a salient mask region for locating defects.

[0080] Generative networks like FastFlow involve the following steps:

[0081] (1) Treat the feature map X as a two-dimensional data distribution and use the two-dimensional normalization flow to transform it into a standard normal distribution f:X→Z, to obtain an H×W×D dimension latent space matrix Z.

[0082] For each feature map, the transformation of the two-dimensional normalized flow can be expressed as:

[0083] z,log|detJ f |=NF(X)

[0084] Where X is the input feature map; z is the transformed latent representation; log|detJ f | is the logarithm of the Jacobian determinant of the transformation, which quantifies the invertibility of the transformation; NF denotes the normalized flow transformation.

[0085] (2) Calculate the Jacobian determinant of the transformation from X to Z to obtain an H×W dimensional Jacobian matrix J:

[0086]

[0087] The Jacobian determinant J reflects the complexity and nonlinearity of the transformation, where Represent f NF Jacobian matrix with respect to X.

[0088] (3) Calculate the probability density at each position in Z to obtain an H×W dimensional probability matrix P, which can be estimated using the Gaussian distribution formula. In the case of a standard normal distribution:

[0089]

[0090] Where k is the dimension of Z.

[0091] (4) Multiply J and P to obtain an H×W dimensional anomaly matrix A:

[0092] A = J × P(Z)

[0093] In this matrix, each element represents an anomaly index for each pixel in the feature map X.

[0094] (5) Concatenate A and X to obtain an H×W×(D+1) dimensional output matrix Y. The output matrix contains the original features and anomalous features, which are used for subsequent classification or localization tasks.

[0095] 3. Semantic segmentation module: It adopts the efficient HRNet network, which enhances the ability to capture details by maintaining high-resolution feature maps and performs fine semantic segmentation.

[0096] Classical semantic segmentation networks perform poorly on small objects. Most existing methods utilize high-resolution features from multi-scale features to address this problem, but this incurs high time complexity. Building upon HRNet, we introduce Deformable Large Kernel Attention (D-LKA-Attention). This mechanism allows the network to more flexibly adjust its receptive field, making it particularly effective in handling defects in small objects.

[0097] Deformable convolution learns a bias at each position on top of basic convolution, which means adding an offset to each position, allowing the network to focus on the position of interest.

[0098] Deformable convolution does not involve deformation in the convolution kernel, but rather in the original image, which is offset. After passing through normal convolution, it achieves the effect of variable convolution, which is feature offset + normal convolution.

[0099]

[0100] Where, x∈R C×H×W The input is a feature map; here, q represents the query, which in the attention mechanism is the element used to match the keys; the query typically comes from the feature representation of the input data, and the elements include the constant z. q and 2D location information p q ;m is the index of the multi-head attention mechanism, k is the index of the key sampled by the m-th attention head, K is the total number of keys sampled by that attention head, K < <H W;Δp mqk Let A represent the sampling offset of the k-th sampling point of the m-th attention head. mqkIt is the attention weight for that sampling point.

[0101] The salient mask regions output by the feature segmentation module, along with the samples, are input into the semantic segmentation network HRNet. HRNet is an efficient semantic segmentation network that enhances its ability to capture details by preserving high-resolution feature maps. HRNet typically uses cross-entropy loss as its loss function; here, Dice loss is combined to improve the segmentation performance for small object defects.

[0102] Cross-entropy loss is one of the most commonly used loss functions in semantic segmentation, used to measure the difference between the predicted pixel classification probability distribution and the true label. For each pixel, the cross-entropy loss is expressed as:

[0103]

[0104] Where C is the total number of categories; y c It is an indicator function of the true label on category c, if the pixel belongs to category c. c The value is 1 if it is 1, otherwise it is 0. The model predicts that the pixel belongs to a certain category. c The probability of.

[0105] To improve segmentation performance for small objects, HRNet can be used in conjunction with the Dice loss. The Dice loss is a loss function based on set similarity, particularly suitable for imbalanced data. The Dice loss is expressed as:

[0106]

[0107] Where: N is the total number of pixels; p i It is the probability that pixel i belongs to a certain category as predicted by the model; g i It is the real label; it is 1 if pixel i belongs to this category, and 0 otherwise.

[0108] Combining the cross-entropy loss with the Dice loss, the total loss is:

[0109] L total =αL CE +βL Dice

[0110] Here, α and β are the weighting coefficients used to adjust the cross-entropy loss and Dice loss in the total loss.

[0111] Integrating a deformable large kernel attention module into the semantic segmentation network HRNet can improve the network's ability to capture and identify defects in small targets.

[0112] Deformable convolution adapts to features of different shapes, and then large kernel attention reinforces key information, such as... Figure 4 (a) and Figure 4 As shown in (b), the deformable large kernel attention module improves the HRNet network's ability to distinguish small targets during inference, thus improving the segmentation effect.

[0113] S4. Use the defect images and image-level labels in the MV Tec industrial defect dataset to iteratively train the semantic segmentation network HRNet obtained in step S3, and use the trained semantic segmentation network HRNet to realize defect detection on the pipe surface.

[0114] S401. Train the generative network Fastflow in the feature segmentation module using the training set, and update the network parameters in each iteration;

[0115] S402. Initialize the semantic segmentation network HRNet, and further adjust and optimize the semantic segmentation network using samples and saliency mask regions generated by the generative network.

[0116] S403. Save the optimal weights of the semantic segmentation network based on the best accuracy of the test set.

[0117] In another embodiment of the present invention, a weakly supervised pipe surface defect detection system based on generative networks is provided. This system can be used to implement the above-mentioned weakly supervised pipe surface defect detection method based on generative networks. Specifically, the weakly supervised pipe surface defect detection system based on generative networks includes a data module, an expansion module, a network module, and a detection module.

[0118] The data module collects surface defect feature information of the pipe to be tested, forming the original dataset;

[0119] The expansion module expands the original dataset obtained from the data module using image enhancement technology, and divides the expanded data into training and testing sets.

[0120] The network module constructs a semantic segmentation network HRNet, which includes a feature extraction module, a feature segmentation module, and a semantic segmentation module, and introduces a deformable large kernel attention mechanism.

[0121] The detection module uses the training and test sets obtained from the expansion module to iteratively train the semantic segmentation network HRNet obtained from the network module, saves the optimal weights of the semantic segmentation network HRNet, initializes the semantic segmentation network HRNet with the optimal weights, and uses the initialized semantic segmentation network HRNet to realize the detection of defects on the surface of the pipe.

[0122] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used in the operation of a weakly supervised pipe surface defect detection method based on generative networks, including:

[0123] The surface defect feature information of the pipe to be inspected is collected to form the original dataset. The original dataset is expanded by image enhancement technology, and the expanded data is divided into training set and test set. A semantic segmentation network HRNet, which includes a feature extraction module, a feature segmentation module and a semantic segmentation module, is constructed, and a deformable big kernel attention mechanism is introduced. The semantic segmentation network HRNet is iteratively trained using the training set and the test set, and the optimal weight of the semantic segmentation network HRNet is saved. The optimal weight is used to initialize the semantic segmentation network HRNet, and the initialized semantic segmentation network HRNet is used to realize the defect detection of the pipe surface.

[0124] In another embodiment of the present invention, a storage medium is also provided, specifically a computer-readable storage medium (memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0125] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the weakly supervised pipe surface defect detection method based on generative networks in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps:

[0126] The surface defect feature information of the pipe to be inspected is collected to form the original dataset. The original dataset is expanded by image enhancement technology, and the expanded data is divided into training set and test set. A semantic segmentation network HRNet, which includes a feature extraction module, a feature segmentation module and a semantic segmentation module, is constructed, and a deformable big kernel attention mechanism is introduced. The semantic segmentation network HRNet is iteratively trained using the training set and the test set, and the optimal weight of the semantic segmentation network HRNet is saved. The optimal weight is used to initialize the semantic segmentation network HRNet, and the initialized semantic segmentation network HRNet is used to realize the defect detection of the pipe surface.

[0127] Please see Figure 6 The terminal device is a computer device. In this embodiment, the computer device 60 includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When executed by the processor 61, the computer program 63 implements the fluid composition calculation method in the reservoir stimulation wellbore of this embodiment. To avoid repetition, these details are not elaborated here. Alternatively, when executed by the processor 61, the computer program 63 implements the functions of each model / unit in the fluid composition calculation system in the reservoir stimulation wellbore of this embodiment. To avoid repetition, these details are not elaborated here.

[0128] Computer device 60 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will understand that... Figure 6 This is merely an example of computer device 60 and does not constitute a limitation on computer device 60. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0129] The processor 61 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0130] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or RAM of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device 60.

[0131] Furthermore, the memory 62 may include both internal storage units of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0132] Please see Figure 7 The terminal device is a chip. In this embodiment, the chip 600 includes a processor 622, which may be one or more, and a memory 632 for storing computer programs executable by the processor 622. The computer program stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processor 622 may be configured to execute the computer program to perform the generalizable monocular absolute depth map estimation method described above.

[0133] Additionally, chip 600 may also include a power supply component 626 and a communication component 650. The power supply component 626 can be configured to perform power management of chip 600, and the communication component 650 can be configured to enable communication of chip 600, such as wired or wireless communication. Furthermore, chip 600 may also include an input / output interface 658. Chip 600 can operate on an operating system stored in memory 632.

[0134] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0135] Example 1:

[0136] The network model was initialized using optimal weights, and its performance was evaluated on the test set. Table 1 shows the Intersection over Union (IoU) ratios (IoU) for various segments on the MV Tec industrial defect dataset using the method of this invention and other semantic segmentation networks.

[0137] Table 1. Performance comparison of semantic segmentation networks on the MV Tec industrial defect dataset.

[0138] Class PSPNet DeepLabV3 HRNet Method of the present invention good 99.49 99.49 99.51 99.48 color 75.95 80.07 76.73 74.78 cut 57.41 56.59 56.95 54.08 hole 61.7 62.21 67.78 66.22 metal_contamination 66.81 65.73 64.95 67.09 thread 44.11 36.22 48.54 54.19 MIoU 67.58 66.72 69.07 69.31

[0139] Experimental results show that, compared with other classic semantic segmentation methods, the method of this invention achieves a mean intersection-over-union (MIoU) of 69.31% on the MV Tec industrial defect dataset, demonstrating strong competitiveness. It achieves a higher mean MIoU with the same number of parameters as PSPNet but fewer than DeepLabv3. Compared with HRNet, the MIoU for defect segmentation of small object categories is improved by 5.65%, and the mean MIoU for all defect categories is improved by 0.24%.

[0140] Example 2:

[0141] The network model was initialized using optimal weights, and the detection performance of the method was comprehensively evaluated on the acquired images of defects on the pipe surface. The area under the curve (AUC) and F1 score (F1-Score) were used to measure defect recognition performance, with an AUC of 95.63% and an F1-Score of 97.27%. The mean intersection-over-union (MIoU) and accuracy (MACC) were used to measure defect classification performance, with an MIoU of 59.47% and a MACC of 69.05%. Figure 5The image shown is a diagram illustrating the detection results of pipe defects using the method of the present invention. This embodiment can effectively detect and classify small target defects on the surface of pipes, improving the automation and accuracy of pipe quality control.

[0142] In summary, the present invention provides a weakly supervised method and system for detecting surface defects in pipes based on generative networks, which has the following advantages:

[0143] 1. Reduce dataset dependencies

[0144] Traditional weakly supervised methods based on classification networks often rely on large datasets to pre-train the classification network; the method of this invention reduces this reliance by using generative networks, enabling efficient defect detection even with limited dataset resources.

[0145] 2. Improve the detection accuracy of small targets

[0146] By integrating a deformable large kernel attention (D-LKA-Attention) module to adapt to features of different shapes and enhancing key information, the method of this invention can effectively improve the detection accuracy and classification performance of semantic segmentation networks for small target defects.

[0147] 3. Enhance the model's generalization ability

[0148] The generalization ability and robustness of the model were enhanced by image augmentation and expansion with a diversified training dataset.

[0149] 4. Meets real-time requirements

[0150] By combining and optimizing different modules in the network model, high-precision defect feature recognition and classification can be achieved while real-time online defect segmentation; thus meeting the real-time requirements for data processing performance in industrial settings.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0153] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0154] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0155] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0156] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0157] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0158] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0159] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0160] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0161] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A weakly supervised method for detecting surface defects in pipes based on generative networks, characterized in that, Includes the following steps: S1. Collect surface defect feature information of the pipe to be inspected to form the original dataset; S2. The original dataset obtained in step S1 is expanded using image enhancement technology, and the expanded data is divided into training set and test set; S3. Construct the semantic segmentation network HRNet and introduce a deformable large kernel attention mechanism. The semantic segmentation network HRNet includes: The feature extraction module uses ResNet18 as the backbone network for feature extraction to extract features from the input image. The feature segmentation module uses Fastflow, a generative network based on a flow model, to quickly and efficiently extract features from the input image and perform preliminary feature segmentation, providing a salient mask region for locating defects. The semantic segmentation module uses the HRNet network to enhance capture capabilities by preserving feature maps and then performs semantic segmentation. S4. Use the training set and test set obtained in step S2 to iteratively train the semantic segmentation network HRNet obtained in step S3, save the optimal weights of the semantic segmentation network HRNet, initialize the semantic segmentation network HRNet with the optimal weights, and use the initialized semantic segmentation network HRNet to realize defect detection on the surface of the pipe.

2. The method for weakly supervised pipe surface defect detection based on generative networks according to claim 1, characterized in that, Step S1 is as follows: The surface defect feature information of the pipe to be detected is filtered by the sliding window method, the background area is removed, and a surface defect slice of size 256×256 is uniformly generated from each pipe surface image. Based on prior knowledge, a corresponding image-level category label is added to each pipe surface image. The category label of each image corresponds to its defect type, which constitutes the original dataset.

3. The method for detecting weakly supervised pipe surface defects based on generative networks according to claim 1, characterized in that, In step S2, 80% of the original dataset is divided into the training set and 20% into the test set.

4. The method for detecting surface defects in pipes based on generative networks according to claim 1, characterized in that, In step S2, the image enhancement techniques include rotation, scaling, and flipping.

5. The method for detecting weakly supervised pipe surface defects based on generative networks according to claim 1, characterized in that, The loss function for the feature extraction module is: in, It is the number of samples in the batch; It is the product of the dimensions of each latent variable; It is the first i The first sample j One implicit variable element; It is the first i Jacobian determinant of each sample.

6. The method for detecting surface defects in pipes based on generative networks according to claim 1, characterized in that, In the feature segmentation module, the specific steps of the generative network Fastflow are as follows: feature map X Treating it as a two-dimensional data distribution, we transform it to a standard normal distribution using a two-dimensional normalization flow. This yields an H×W×D dimensional latent space matrix Z; Calculate the Jacobian determinant of the transformation from X to Z to obtain an H×W dimensional Jacobian matrix. J ; calculate Z The probability density at each position in the matrix yields an H×W dimensional probability matrix. P ; Will J and P Multiplying them yields an H×W dimensional anomaly matrix. A ; Will A and X Concatenating these matrices yields an H×W×(D+1) dimensional output matrix containing both the original features and the anomalous features. Y .

7. The method for detecting weakly supervised pipe surface defects based on generative networks according to claim 1, characterized in that, Loss function of the semantic segmentation network HRNet Specifically: in, and These are the weighting coefficients used to adjust the cross-entropy loss and Dice loss in the total loss. For cross-entropy loss, This is a loss for Dice.

8. The method for detecting weakly supervised pipe surface defects based on generative networks according to claim 1, characterized in that, Step S4 is as follows: The generative network Fastflow in the feature segmentation module is trained using the training set, and the network parameters are updated in each iteration. The semantic segmentation network HRNet is initialized, and the semantic segmentation network is further adjusted and optimized using samples and saliency mask regions generated by the generative network. Save the optimal weights of the semantic segmentation network based on the best accuracy on the test set; The semantic segmentation network HRNet is initialized with optimal weights to detect the surface of pipes and identify the location and type of defects.

9. A weakly supervised pipe surface defect detection system based on generative networks, characterized in that, include: The data module collects surface defect feature information of the pipe to be inspected, forming the original dataset; The augmentation module uses image enhancement technology to augment the original dataset obtained from the data module, and then divides the augmented data into a training set and a test set. The network module constructs a semantic segmentation network HRNet, which includes a feature extraction module, a feature segmentation module, and a semantic segmentation module. A deformable large kernel attention mechanism is introduced. The semantic segmentation network HRNet includes: The feature extraction module uses ResNet18 as the backbone network for feature extraction to extract features from the input image. The feature segmentation module uses Fastflow, a generative network based on a flow model, to quickly and efficiently extract features from the input image and perform preliminary feature segmentation, providing a salient mask region for locating defects. The semantic segmentation module uses the HRNet network to enhance capture capabilities by preserving feature maps and then performs semantic segmentation. The detection module uses the training and test sets obtained from the expansion module to iteratively train the semantic segmentation network HRNet obtained from the network module, saves the optimal weights of the semantic segmentation network HRNet, and initializes semantic segmentation using the optimal weights.

Images