Pipe inner surface defect detection method based on improved YOLOv11

Abstract

The application provides a pipe inner surface defect detection method based on improved YOLOv11, and relates to the field of industrial surface defect detection. The deformable convolution module and the double-layer routing attention mechanism module are constructed, and the feature extraction capability and the key area focusing capability for multi-scale and irregular defects are significantly enhanced. The pipe inner surface defect dataset constructed through the data enhancement strategy effectively improves the generalization capability and the robustness of the model in the real industrial environment such as uneven illumination and complex background, and meets the industrial real-time detection requirement. The application realizes the improvement of the detection precision and the calculation speed, and especially shows a significant advantage in the target recognition task in the small target dense scene and the complex background.

Description

Technical Field

[0001] This invention relates to the field of industrial surface defect detection, and specifically to a method for detecting defects on the inner surface of pipes based on an improved YOLOv11. Background Technology

[0002] Seamless steel pipes are a key basic material in modern industry, and their quality directly determines the reliability and safety of energy transmission, operation of major equipment, and infrastructure construction. In core fields such as oil and gas drilling and extraction, high-pressure boilers, nuclear power, and aerospace, seamless steel pipes are often subjected to extreme environments such as high temperature and high pressure, and must possess high strength, high toughness, and excellent sealing performance. Any tiny defects on the surface can lead to leaks or even pipe ruptures during long-term service, causing serious safety accidents and significant economic losses.

[0003] Currently, the production technology of seamless steel pipes both domestically and internationally has matured. However, during the steel pipe production process, due to various factors such as piercing equipment and processing techniques, different types of defects easily appear on the inner surface, including scratches, roller marks, iron oxide scale, surface inclusions, holes, cracks, and pitting. These defects are difficult to observe and detect in a closed environment, making the technology for detecting defects on the inner surface of steel pipes crucial and challenging. Currently, most companies' quality control measures rely on manual visual inspection, but traditional manual visual inspection is inefficient and lacks sufficient accuracy, failing to meet the requirements of large-scale production. Although steel pipe plants are gradually starting to use visual inspection equipment, a large number of companies still depend on manual visual inspection, relying on quality inspectors to observe with their naked eyes and make subjective judgments based on personal experience to identify various defects. This is highly susceptible to human interference; different inspectors may have inconsistent judgments on the same defect. Moreover, manual inspection is slow, making it difficult to meet the urgent needs of large-scale and efficient production.

[0004] The unique feature of the YOLO (You Only Look Once) series of object detection algorithms lies in transforming the object detection task into a single regression problem. It is an end-to-end algorithm with extremely fast computation speed, and has been widely used in many practical application scenarios such as intelligent monitoring, autonomous driving, and industrial inspection. YOLOv11, launched by Ultralytics in 2024, achieved a significant improvement in detection accuracy. However, YOLOv11 still has room for improvement in the comprehensiveness of feature extraction and feature fusion: YOLOv11 has limited ability to perceive key information in images, and it struggles to accurately capture the contour features of irregular defects; furthermore, multi-scale feature fusion is prone to problems such as loss of features for small targets and classification confusion in complex situations, affecting model performance. Therefore, there is an urgent need to develop an intelligent detection method for defects on the inner surface of pipes, enabling automatic identification and quantitative evaluation of these defects, providing solid technical support for the high-quality and high-efficiency production of seamless steel pipes. Summary of the Invention

[0005] The purpose of this invention is to provide a method for detecting defects on the inner surface of pipes based on an improved YOLOv11. By replacing the C2PSA module and C3k2 in the backbone network of YOLOv11 with a two-layer routing attention mechanism module and a deformable convolution module, respectively, the detection performance is improved.

[0006] The present invention provides a method for detecting defects on the inner surface of pipes based on an improved YOLOv11, comprising the following steps:

[0007] Construct a dataset of defects on the inner surface of pipes, and label and classify the dataset.

[0008] A target detection model for detecting defects on the inner surface of pipes is constructed. The target detection model is trained using the labeled and segmented dataset of defects on the inner surface of pipes to obtain a trained target detection model.

[0009] The system acquires an image of the inner surface of the pipe to be inspected, and outputs defect detection results based on a trained target detection model, including the size and location information of the defects.

[0010] The target detection model is built on the YOLOv11 network. A dual-layer routing attention mechanism module and a deformable convolution module are embedded in the YOLOv11 backbone network. The dual-layer routing attention mechanism module is used to enhance the ability to focus on defect regions. The deformable convolution module is used to improve the feature extraction ability of irregular defects by adaptively adjusting the geometric structure of the convolution kernel.

[0011] The dual-layer routing attention mechanism module includes a first convolutional layer, a segmentation layer, multiple BRA sub-modules, a splicing layer, and a second convolutional layer;

[0012] The BRA submodule is used to process the input feature map and extract the output tensor. The specific method is as follows:

[0013] The feature map of the input BRA submodule is divided into several non-overlapping square regions to obtain the region feature map;

[0014] Calculate the query matrix, key matrix, and value matrix for each region based on the region feature map;

[0015] Calculate the adjacency matrix based on the query matrix and the key matrix;

[0016] The adjacency matrix is ​​filtered, and the top-ranked members of the adjacency matrix are retained in each region. Connect the elements to obtain the index matrix;

[0017] The key matrix and value matrix are filtered and clustered based on the index matrix to obtain the clustered key matrix and clustered value matrix;

[0018] The output tensor is calculated based on the aggregated key matrix and aggregated value matrix.

[0019] Furthermore, the specific method for constructing the data set of defects on the inner surface of the pipe is as follows:

[0020] Several images of the inner surface of the pipe are acquired as an initial dataset, and the images of the inner surface of the pipe include various defect types;

[0021] Data augmentation operations are performed on the initial dataset to obtain the base dataset; the data augmentation operations include vertical flipping, adding Gaussian noise, random brightness adjustment, and random contrast adjustment.

[0022] A conditional generative adversarial network data augmentation strategy with edge guidance was established to expand the basic dataset and obtain a dataset of defects on the inner surface of pipes.

[0023] Furthermore, the edge-guided conditional generative adversarial network data augmentation strategy includes:

[0024] Several images of the inner surface of the pipe taken under ideal uniform light source are collected as source domain images to construct a source domain dataset; several images of the inner surface of the pipe taken under complex lighting conditions are collected as target domain images to construct a target domain dataset. Complex lighting conditions include local reflection, shadow and uneven brightness; a dual-domain dataset is constructed based on the source domain dataset and the target domain dataset, wherein the number of target domain images is less than the number of source domain images.

[0025] For each source domain image in the source domain dataset, the Canny edge detection operator is used to extract the edge gradient map of the defect;

[0026] A conditional generative adversarial network is constructed, which takes the source domain image and the edge gradient map concatenated in the channel dimension as input, and outputs an enhanced image with the lighting style of the target domain based on the conditional generative adversarial network.

[0027] A loss function for the conditional generative adversarial network (GAN) is established, and the GAN is trained to obtain a trained GAN, which is used to generate enhanced images with target domain illumination style.

[0028] Furthermore, the specific method for expanding the basic dataset is as follows:

[0029] The Canny edge detection operator is used to extract the edge gradient map of each image in the basic dataset. Each image in the basic dataset and its corresponding edge gradient map are then input into a trained conditional generative adversarial network to generate an enhanced image with the lighting style of the target domain. The enhanced image and the basic dataset are then merged to form a dataset of defects on the inner surface of pipes.

[0030] Furthermore, the specific method for labeling and classifying the defect dataset on the inner surface of the pipe is as follows:

[0031] The images in the inner surface defect dataset of the pipe are divided into training set, validation set and test set according to the proportion, and the defects are labeled.

[0032] Furthermore, the deformable convolution module includes a first convolution, a segmentation operation, multiple C3k-DCNV2 sub-modules, a concatenation operation, and a second convolution. The output of the first convolution and the output of the segmentation operation are respectively input to the concatenation operation through residual connections.

[0033] The C3k-DCNV2 submodule includes a third convolution, a fourth convolution, multiple BottleNeck-DCNV2 layers, a concatenation operation, and a fifth convolution. The BottleNeck-DCNV2 layer includes a convolution operation and a two-dimensional feature map convolution DCNV2, and the input of the BottleNeck-DCNV2 layer and the output of the two-dimensional feature map convolution DCNV2 are concatenated through residual connections.

[0034] This invention provides a pipe internal surface defect detection method based on an improved YOLOv11, which has the following beneficial effects: In practical applications, this invention exhibits significant advantages in several aspects. Regarding detection accuracy, by constructing a deformable convolutional module and a dual-layer routing attention mechanism module, this invention significantly enhances the feature extraction capability and key area focusing capability for multi-scale, irregular defects. Regarding detection efficiency, this invention nearly doubles the efficiency of the original YOLOv11 model, fully meeting the needs of real-time industrial detection. Regarding practicality and adaptability, the pipe internal surface defect dataset constructed through a systematic data augmentation strategy effectively improves the model's generalization ability and robustness in real industrial environments such as uneven lighting and complex backgrounds. This invention solves the problems of low efficiency, strong subjectivity, and high false negative rate inherent in traditional manual visual inspection and conventional non-destructive testing methods. It achieves high-precision and high-efficiency detection of pipe internal surface defects, enabling timely identification and location of typical defects such as cracks and pits. It provides enterprises with a reliable, traceable, and intelligent quality control solution, possessing significant engineering application value in directly improving product quality control levels, reducing production costs, and enhancing the intelligence level of production lines. Attached Figure Description

[0035] Figure 1 This is a flowchart of the pipe inner surface defect detection method based on the improved YOLOv11 provided in Embodiment 1 of the present invention;

[0036] Figure 2 This is a diagram of the YOLO-DB network structure of the target detection model provided in Embodiment 1 of the present invention;

[0037] Figure 3 This is a structural diagram of the C2BRA dual-layer routing attention mechanism module provided in Embodiment 1 of the present invention;

[0038] Figure 4 This is a structural diagram of the deformable convolutional module C3k2-DCNV2 provided in Embodiment 1 of the present invention, wherein (a) is a structural diagram of the deformable convolutional module C3k2-DCNV2, (b) is a structural diagram of the C3k-DCNV2 submodule, and (c) is a structural diagram of the BottleNeck-DCNV2 layer;

[0039] Figure 5 This is a comparison chart of the mAP of the target detection model YOLO-DB and the YOLOv11 model provided in Embodiment 1 of the present invention;

[0040] Figure 6This is a comparison of defect detection results based on the YOLOv11 model and the trained target detection model YOLO-DB, provided in Embodiment 1 of the present invention. (a) shows the defect detection result obtained by performing defect detection on the inner surface image of the first pipe based on the YOLOv11 model; (b) shows the defect detection result obtained by performing defect detection on the inner surface image of the first pipe based on the trained target detection model YOLO-DB; (c) shows the defect detection result obtained by performing defect detection on the inner surface image of the second pipe based on the YOLOv11 model; (d) shows the defect detection result obtained by performing defect detection on the inner surface image of the second pipe based on the trained target detection model YOLO-DB; (e) shows the defect detection result obtained by performing defect detection on the inner surface image of the third pipe based on the YOLOv11 model; and (f) shows the defect detection result obtained by performing defect detection on the inner surface image of the third pipe based on the trained target detection model YOLO-DB. Detailed Implementation

[0041] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0042] Example 1:

[0043] A method for detecting defects on the inner surface of pipes based on an improved YOLOv11, such as... Figure 1 As shown, it includes:

[0044] Step 1: Construct a dataset of defects on the inner surface of pipes, and label and classify the dataset of defects on the inner surface of pipes;

[0045] Step 1.1: Obtain several images of the inner surface of the pipe as the initial dataset. The images of the inner surface of the pipe include various defect types.

[0046] Step 1.2: Perform data augmentation operations on the initial dataset to obtain the base dataset; data augmentation operations include vertical flipping, adding Gaussian noise, random brightness adjustment, and random contrast adjustment;

[0047] In this embodiment, the SSP2000 dataset of images of the inner surface of cold-drawn pipes from a seamless steel pipe factory is used as the initial dataset. The initial dataset contains eight types of defects: horizontal lines, peeling, pits, pinholes, rust, sand marks, oxide scale, and scratches, with more than 2,000 defect images. The resolution of the original SSP2000 images is 640×360. Considering the low defect rate in the production process of cold-drawn pipes and the characteristics of the inner surface defects of the pipes being prone to blurred edges and varied shapes, some low-quality and blurry defect images need to be filtered out. After filtering, the dataset has 1,200 images remaining. The dataset constructed from these 1,200 defect images is defined as SSP1200. Insufficient data may lead to overfitting, so data augmentation operations are performed. Data augmentation operations include vertical flipping, adding Gaussian noise, random brightness adjustment, and random contrast adjustment. The data-augmented pipe defect images are then merged with the SSP1200 dataset to reconstruct the basic dataset SSP6000.

[0048] Step 1.3: Establish a conditional generative adversarial network data augmentation strategy with edge guidance to expand the basic dataset and obtain a dataset of defects on the inner surface of pipes;

[0049] To further enhance the model's robustness under realistic and complex lighting conditions, this embodiment introduces an edge-guided conditional generative adversarial network (GAN) data augmentation strategy. The specific steps are as follows:

[0050] 1) Construct a two-domain dataset, including a source domain dataset and a target domain dataset;

[0051] A batch of images of the inner surface of pipes taken under ideal uniform light sources were collected as source domain images to construct a source domain dataset. ;

[0052] Several images of the inner surface of the pipe, taken under complex lighting conditions on the actual production line, were collected as target domain images to construct a target domain dataset. Complex lighting conditions include localized reflections, shadows, and uneven brightness;

[0053] The number of images in the target domain is less than the number of images in the source domain;

[0054] 2) For the source domain dataset Each source domain image in The Canny edge detection operator is used to extract the source domain image. Edge gradient map of defects and edge mask ,include:

[0055] Use a Gaussian filter on the source domain image Denoising is performed to obtain the denoised image. The Gaussian function is shown in formula (1) below:

[0056] (1);

[0057] in, Represents the coordinates on a two-dimensional plane as The Gaussian function value of the pixel at that location. The x-coordinate of the pixel is The ordinate of the pixel is Standard deviation;

[0058] The Sobel operator is used to calculate the gradient of the denoised image in the horizontal direction. and the gradient in the vertical direction Further calculate the gradient magnitude and gradient direction, gradient magnitude As shown in formula (2) below:

[0059] (2);

[0060] gradient direction As shown in formula (3) below:

[0061] (3);

[0062] Then, non-maximum suppression is used to preserve local gradient maxima; the final edges are determined through double threshold detection and edge connectivity, resulting in the defect edge gradient map. and edge gradient map Binarization yields the edge mask. ;

[0063] 3) Construct a conditional generative adversarial network, including a generator. and discriminator generator Source domain image and edge gradient map The image stitched along the channel dimension is used as input, and the output is an enhanced image with the target domain lighting style. ,in, For generators.

[0064] Discriminator The input is an enhanced image. And the corresponding edge gradient map, output the enhanced image. The probability that it belongs to the target domain dataset.

[0065] 4) Establish the loss function for the conditional generative adversarial network, including the generator loss function and the discriminator loss function;

[0066] Generator loss function As shown in formula (4) below:

[0067] (4);

[0068] in, It is a random noise vector; It is conditional information, including enhanced images. and source domain image Edge gradient map; The generator is in the condition information The generated samples; The generated samples output by the discriminator The probability of belonging to the target domain dataset; Indicates noise In condition information The following expectations; Let edge-aware loss function be used. These are the edge-sensing weight coefficients; For the perceptual loss function, The weighting coefficients for perceived loss.

[0069] Discriminator loss function As shown in formula (5) below:

[0070] (5);

[0071] in, These are real samples, i.e., images of the target domain; It is the discriminator's conditional information on real samples The probability of the output; It is the joint distribution of the real samples.

[0072] Edge-aware loss function As shown in the following formula (6):

[0073] (6);

[0074] Among them, edge mask Set the value to 1 at the edge of the defect and 0 at the rest. This indicates element-wise multiplication; L1 norm. Edge-aware loss function. The generator is forced to approximate the original image as closely as possible to the edge regions of the defect, thus preserving the geometric contours of the defect.

[0075] Perceptual loss function As shown in formula (7) below:

[0076] (7);

[0077] in, Indicates the generator's first... Feature maps extracted from layers; This represents the square of the L2 norm. (Perceptual loss function) Used to keep the overall content of the image unchanged.

[0078] 5) Train the conditional generative adversarial network to obtain a trained conditional generative adversarial network;

[0079] Using source domain datasets and target domain data Alternating optimization generator and discriminator Minimize the generator loss function And maximize the discriminator loss function This continues until the conditional generative adversarial network converges, resulting in a well-trained conditional generative adversarial network.

[0080] 6) Generate enhanced images based on a trained conditional generative adversarial network;

[0081] Each image and its corresponding edge gradient map from the base dataset SSP6000 are input into a pre-trained conditional generative adversarial network (GAN). Based on the generator... Enhanced images with realistic production line lighting styles are generated in batches. These enhanced images are then merged with the base dataset SSP6000 to form the pipe internal surface defect dataset SSP-GAN. The pipe internal surface defect dataset SSP-GAN is twice the size of the base dataset and contains rich lighting variations, which helps improve the model's generalization ability.

[0082] Step 1.4: Label and classify the dataset of defects on the inner surface of the pipe;

[0083] In practice, Labelme software was used to annotate the SSP-GAN dataset of pipe inner surface defects. The annotated JSON label format was converted into YOLO label format, and the images in the SSP-GAN dataset of pipe inner surface defects were divided into training set, validation set and test set in a ratio of 8:1:1.

[0084] Step 2: Construct the YOLO-DB target detection model for detecting defects on the inner surface of pipes based on the YOLOv11 network;

[0085] Step 2.1: Embed the two-layer routing attention mechanism module C2BRA in the backbone network of YOLOv11, which incorporates a dynamic sparse attention mechanism to enhance computational flexibility and feature recognition.

[0086] In practical implementation, the main technologies of the C2BRA two-layer routing attention mechanism module embedded in YOLOv11 are as follows:

[0087] like Figure 2 and Figure 3 As shown, the dual-layer routing attention mechanism module C2BRA is based on the C2PSA network structure, replacing the PSA sub-module in the C2PSA module structure with the BRA sub-module, including a first convolutional layer, a segmentation layer, multiple BRA sub-modules, a splicing layer and a second convolutional layer.

[0088] The BRA submodule has a two-level routing attention mechanism, specifically including:

[0089] The feature map of the input BRA submodule Divided into A region feature map is obtained by identifying non-overlapping regions. Each region contains 1 eigenvector The height of the feature map of the input BRA submodule is given. The width of the feature map for the input BRA submodule. The number of channels in the feature map of the input BRA submodule. Regional partitioning factor;

[0090] Based on region feature map Calculate the query matrix Key matrix Sum matrix As shown in the following formula (8):

[0091] (8);

[0092] in, , , Query matrix Key matrix Sum matrix The projection weights;

[0093] Based on query matrix Bond matrix Calculate the adjacency matrix As shown in the following formula (9):

[0094] (9);

[0095] in, For query matrix In the The mean of each region, Key matrix In the The mean of each region, Represents the transpose of a matrix;

[0096] For adjacency matrix Perform filtering, retaining the top-ranked members of the adjacency matrix within each region. Connect the elements to obtain the index matrix. As shown in the following formula (10):

[0097] (10);

[0098] in, For top-k filtering operations, the index matrix The The line contains the first Indexes of the k most relevant regions within a region;

[0099] Based on the key matrix Sum matrix Filtering and aggregation are performed to obtain the aggregated key matrix. and clustered value matrix As shown in the following formula (11):

[0100] (11);

[0101] Cluster-based key matrix and clustered value matrix Calculate the output tensor of the BRA submodule As shown in the following formula (12):

[0102] (12);

[0103] in, For token-token attention, This is a depthwise separable convolution with a kernel size of 5 and a stride of 0.

[0104] The computation of the BRA submodule consists of three parts: linear projection, region-to-region routing, and token-token attention. The total computation amount is shown in the following formula (13):

[0105] (13);

[0106] in, The computational cost of linear projection, The computational cost of region-to-region routing. The computational cost of token-token attention;

[0107] If and only if The equation holds true, therefore, the region partitioning factor As shown in formula (14) below:

[0108] (14);

[0109] Regional partitioning factors Scaling the input BRA submodule's feature map, the complexity of the BRA submodule is... The complexity of vanilla attention is Compared with the traditional Transformer self-attention structure, the BRA submodule significantly reduces the computational cost;

[0110] The BRA submodule not only incorporates a dynamic sparse attention mechanism to enhance computational flexibility and feature recognition, but also features a two-level routing attention mechanism. It uses a relatively coarse attention map to select the most relevant regions, and then performs more refined attention calculations within the selected regions.

[0111] Step 2.2: Embed the deformable convolutional module C3k2-DCNV2 in the backbone network of YOLOv11 to achieve dynamic adaptation of the convolutional kernel geometry. It can effectively capture features when facing targets of various shapes and introduces deformation parameters to dynamically adjust the position and shape of sampling points.

[0112] In this embodiment, the C3k2 (C3k=True) module of YOLOv11 is replaced with the C3k2-DCNV2 module. The change of the C3k2-DCNV2 module compared to the C3k2 module is that the C3k submodule in the C3k2 (C3k=True) module of YOLOv11 is replaced with the C3k-DCNV2 submodule. The C3k-DCNV2 submodule replaces the BottleNeck layer in the C3k submodule of YOLOv11 with the BottleNeck-DCNV2 layer. The BottleNeck-DCNV2 layer includes convolution operations and DCNV2, and the input of the BottleNeck-DCNV2 layer and the output of DCNV2 are concatenated through residual connections to obtain the deformable convolution module C3k2-DCNV2, as follows. Figure 4 As shown.

[0113] Deformable Convolution DCNV1 uses a standard-size grid for the input feature map. Sampling is performed, and the sampled results are weighted and summed. (Grid) The size and expansion of the receptive field are defined for a 3×3 convolution kernel. For any position in the feature map of the input deformable convolution DCNV1 Convolutional eigenvalues As shown in the following formula (15):

[0114] (15);

[0115] in, For grid The Middle The element at position 1 for The weight value of the position. The pixel value at the sampling location;

[0116] In the 2D feature map convolution DCNV2, any position in the feature map The convolutional feature value y(P0) is shown in the following formula (16):

[0117] (16);

[0118] in, This is the offset. , For grid The total number of positions in;

[0119] In 2D feature map convolution DCNV2, sampling occurs at irregular and offset positions. Proceed, but the offset The value is usually a decimal, so bilinear interpolation is used to calculate the pixel position of the sampling point after introducing the offset. The sampling point position pixels after introducing the offset Bilinear interpolation kernel function with two-dimensional kernel As shown in formulas (17) and (18) below:

[0120] (17);

[0121] (18);

[0122] in, Indicates the target sampling point in the grid; For integer points participating in the interpolation calculation, represent all spatial locations in the input feature map; For target sampling points x-coordinate Integer points participating in the interpolation calculation x-coordinate For target sampling points The ordinate, Integer points participating in the interpolation calculation The ordinate, Applying a convolutional layer to the input feature map can obtain an offset. The convolutional kernel has the same spatial resolution and magnification as the current convolutional layer. A new feature map of the same size and 2N channels is generated. Each feature point in the generated feature map contains 2N values, which are used to generate the convolutional kernel and offset of the output feature.

[0123] In addition to adding the offset of each sampling point, a weight coefficient is also added. The weight coefficient is used to distinguish whether the introduced region is the region where the target exists. The final convolution feature value y(P0) output by the two-dimensional feature map convolution DCNV2 is as shown in the following formula (19):

[0124] (19);

[0125] in, These are the weighting coefficients;

[0126] Step 3: Train the target detection model using the labeled and segmented inner surface defect dataset of the pipe to obtain the trained target detection model;

[0127] This embodiment conducts ablation experiments on the YOLOv11 model, the two-layer routing attention mechanism module C2BRA, and the deformable convolution module C3k2-DCNV2, and integrates the optimal backbone network into the YOLOv11 detection framework to obtain the trained object detection model YOLO-DB.

[0128] In practice, four models were built: the YOLOv11 model, the YOLO-BRA model with a two-layer routing attention mechanism module, the YOLO-DCNV2 model with a deformable convolutional module C3k2-DCNV2, and the YOLO-DB model with both a two-layer routing attention mechanism module C2BRA and a deformable convolutional module C3k2-DCNV2. SSP-GAN was used to train all four models. Figure 5 The image shows a comparison of the mAP of the object detection model YOLO-DB and the YOLOv11 model under the same parameter settings. After 150 training epochs, the object detection model YOLO-DB shows a significant advantage in mAP value, surpassing the original YOLOv11 model. Ultimately, YOLO-DB is the most accurate and fastest object detection model for defect detection, solving the problem of insufficient ability of the YOLOv11 model to capture feature targets of different scales in images, reducing the loss of feature data for small targets, and accelerating the detection speed of the model.

[0129] Step 4: Output defect detection results based on the trained target detection model YOLO-DB, including the size and location information of the defects;

[0130] This embodiment uses the YOLOv11 model and the trained object detection model YOLO-DB to perform defect detection on the inner surface images of three types of pipes, obtains the defect detection results, and realizes the visualization of defects, such as... Figure 6 As shown, (a) is the defect detection result obtained by performing defect detection on the inner surface image of the first pipe based on the YOLOv11 model, (b) is the defect detection result obtained by performing defect detection on the inner surface image of the first pipe based on the trained target detection model YOLO-DB, (c) is the defect detection result obtained by performing defect detection on the inner surface image of the second pipe based on the YOLOv11 model, (d) is the defect detection result obtained by performing defect detection on the inner surface image of the second pipe based on the trained target detection model YOLO-DB, (e) is the defect detection result obtained by performing defect detection on the inner surface image of the third pipe based on the YOLOv11 model, and (f) is the defect detection result obtained by performing defect detection on the inner surface image of the third pipe based on the trained target detection model YOLO-DB.

[0131] Example 2:

[0132] This embodiment proposes an electronic device, including: one or more processors, and a memory for storing instructions. When the instructions are executed by the one or more processors, the one or more processors execute the improved YOLOv11-based pipe inner surface defect detection method.

[0133] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the pipe inner surface defect detection method based on the improved YOLOv11 as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0134] The processor is used to execute all or part of the steps in the pipe inner surface defect detection method based on the improved YOLOv11 as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0135] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the pipe inner surface defect detection method based on the improved YOLOv11 described in the above embodiments.

[0136] Example 3:

[0137] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0138] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the pipe inner surface defect detection method based on the improved YOLOv11 described in the various embodiments of this application.

[0139] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the improved YOLOv11-based pipe inner surface defect detection method described above.

[0140] Example 4:

[0141] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned pipe inner surface defect detection method based on the improved YOLOv11.

[0142] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0143] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0144] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A method for detecting defects on the inner surface of pipes based on an improved YOLOv11, characterized in that, Includes the following steps: Construct a dataset of defects on the inner surface of pipes, and label and classify the dataset. A target detection model for detecting defects on the inner surface of pipes is constructed. The target detection model is trained using the labeled and segmented dataset of defects on the inner surface of pipes to obtain a trained target detection model. The system acquires an image of the inner surface of the pipe to be inspected, and outputs defect detection results based on a trained target detection model, including the size and location information of the defects. The target detection model is built on the YOLOv11 network, embedding a dual-layer routing attention mechanism module and a deformable convolution module in the YOLOv11 backbone network; the dual-layer routing attention mechanism module is used to enhance the ability to focus on defect regions; the deformable convolution module is used to improve the feature extraction ability of irregular defects by adaptively adjusting the geometric structure of the convolution kernel. The dual-layer routing attention mechanism module includes a first convolutional layer, a segmentation layer, multiple BRA sub-modules, a splicing layer, and a second convolutional layer; The BRA submodule is used to process the input feature map and extract the output tensor. The specific method is as follows: The feature map of the input BRA submodule is divided into several non-overlapping square regions to obtain the region feature map; Calculate the query matrix, key matrix, and value matrix for each region based on the region feature map; Calculate the adjacency matrix based on the query matrix and the key matrix; The adjacency matrix is ​​filtered, and the top-ranked members of the adjacency matrix are retained in each region. Connect the elements to obtain the index matrix; The key matrix and value matrix are filtered and clustered based on the index matrix to obtain the clustered key matrix and clustered value matrix; The output tensor is calculated based on the aggregated key matrix and aggregated value matrix.

2. The method for detecting defects on the inner surface of pipes based on the improved YOLOv11 according to claim 1, characterized in that, The specific method for constructing the data set of defects on the inner surface of pipes is as follows: Several images of the inner surface of the pipe are acquired as an initial dataset, and the images of the inner surface of the pipe include various defect types; Data augmentation operations are performed on the initial dataset to obtain the base dataset; the data augmentation operations include vertical flipping, adding Gaussian noise, random brightness adjustment, and random contrast adjustment. A conditional generative adversarial network data augmentation strategy with edge guidance was established to expand the basic dataset and obtain a dataset of defects on the inner surface of pipes.

3. The method for detecting defects on the inner surface of pipes based on the improved YOLOv11 according to claim 2, characterized in that, The edge-guided conditional generative adversarial network data augmentation strategy includes: Several images of the inner surface of the pipe taken under ideal uniform light source are collected as source domain images to construct a source domain dataset; several images of the inner surface of the pipe taken under complex lighting conditions are collected as target domain images to construct a target domain dataset. Complex lighting conditions include local reflection, shadow and uneven brightness; a dual-domain dataset is constructed based on the source domain dataset and the target domain dataset, wherein the number of target domain images is less than the number of source domain images. For each source domain image in the source domain dataset, the Canny edge detection operator is used to extract the edge gradient map of the defect; A conditional generative adversarial network is constructed, which takes the source domain image and the edge gradient map concatenated in the channel dimension as input, and outputs an enhanced image with the lighting style of the target domain based on the conditional generative adversarial network. A loss function for the conditional generative adversarial network (GAN) is established, and the GAN is trained to obtain a trained GAN, which is used to generate enhanced images with target domain illumination style.

4. The method for detecting defects on the inner surface of pipes based on the improved YOLOv11 according to claim 3, characterized in that, The specific method for expanding the basic dataset is as follows: The Canny edge detection operator is used to extract the edge gradient map of each image in the basic dataset. Each image in the basic dataset and its corresponding edge gradient map are then input into a trained conditional generative adversarial network to generate an enhanced image with the lighting style of the target domain. The enhanced image and the basic dataset are then merged to form a dataset of defects on the inner surface of pipes.

5. The method for detecting defects on the inner surface of pipes based on the improved YOLOv11 according to claim 4, characterized in that, The specific method for labeling and classifying the dataset of defects on the inner surface of pipes is as follows: The images in the inner surface defect dataset of the pipe are divided into training set, validation set and test set according to the proportion, and the defects are labeled.

6. The method for detecting defects on the inner surface of pipes based on the improved YOLOv11 according to claim 1, characterized in that, The deformable convolution module includes a first convolution, a segmentation operation, multiple C3k-DCNV2 sub-modules, a concatenation operation, and a second convolution. The outputs of the first convolution and the segmentation operation are respectively fed into the concatenation operation via residual connections. The C3k-DCNV2 submodule includes a third convolution, a fourth convolution, multiple BottleNeck-DCNV2 layers, a concatenation operation, and a fifth convolution. The BottleNeck-DCNV2 layer includes a convolution operation and a two-dimensional feature map convolution DCNV2, and the input of the BottleNeck-DCNV2 layer and the output of the two-dimensional feature map convolution DCNV2 are concatenated through residual connections.

Images