A sewer internal defect detection method and system based on YOLOv11 improvement

Abstract

The application provides a sewer internal defect detection method and system based on YOLOv11 improvement, the method realizes dynamic weighted fusion of multi-level features through an adaptive feature fusion module, enhances the detection capability of the model on small targets and complex backgrounds, improves the selectivity of channel features through a SENet attention mechanism, and reduces background interference; an MPDIoU loss function is introduced to improve the boundary box regression loss. The improved model is significantly better than the original YOLOv11 in the detection tasks of cracks, obstacles, corrosion, deformation and other sewer defects, and has significant improvement in key indicators such as mAP50 (average precision), Box precision (P) and recall rate (R). The application can realize efficient and accurate detection of multiple types of internal defects of sewers, and provides a better solution for sewer maintenance and management.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, and in particular relates to a method and system for detecting internal defects in sewers based on YOLOv11. Background Technology

[0002] With the acceleration of urbanization and the aging of infrastructure, the maintenance and repair of sewer systems have become increasingly important. Traditional sewer inspection methods mainly rely on manual inspection or the use of simple vision equipment. This approach is not only inefficient and costly, but also struggles to handle complex internal environments, such as dim lighting and poor water quality. In recent years, deep learning-based object detection technology has been widely applied in various fields due to its efficiency and accuracy. However, in specific application scenarios such as underground pipe defect detection, the performance of existing models still has limitations. The internal environment of sewers is usually quite harsh, including but not limited to dim lighting conditions, complex background interference (such as sewage and sediment), and irregular geometries. These factors make it difficult for traditional object detection algorithms to achieve good performance. In addition, there are many types of defects in sewers, ranging from tiny cracks to large-area damage, and the distribution of different types of defects is very sparse, which places higher demands on the detection model. Due to differences in pipe diameter and viewing angle, the size of target objects in the image can vary greatly, which increases the difficulty of detection. Therefore, developing an automated detection system that can accurately identify multiple types of defects in complex environments is particularly important.

[0003] Deep learning, a branch of machine learning, simulates complex data patterns by constructing multi-layered neural networks. Convolutional Neural Networks (CNNs), in particular, have achieved great success in image recognition tasks and have gradually become the mainstream tool in the field of computer vision. The YOLO (You Only Look Once) series of algorithms, as one of the classic representatives of real-time object detection, is renowned for its speed and high accuracy. However, when facing specific domain problems, such as detecting cracks, obstacles, and corrosion in underground pipes, the standard YOLO model may not meet practical needs because these scenarios typically feature low lighting conditions, high noise levels, and diverse defect morphologies. Although the existing YOLOv11 model has demonstrated excellent performance on several public datasets, directly applying it to defect detection inside sewers still presents some challenges: limited feature fusion capabilities, resulting in low detection accuracy for small targets against complex backgrounds; low bounding box regression accuracy, especially when the aspect ratio is the same but the specific dimensions differ; and a lack of modeling for inter-channel dependencies, leading to the loss of some key information and affecting the final detection results. These issues limit the effectiveness of YOLOv11 in practical applications, especially in situations requiring high accuracy and robustness.

[0004] To address these limitations, various improvements have been introduced in recent years to enhance the performance of the YOLOv11 model. First, regarding feature fusion, the traditional YOLOv11 uses a fixed Feature Pyramid Network (FPN), which, while capable of handling multi-scale features, still requires improvement in the detection accuracy of small objects against complex backgrounds. Second, for bounding box regression, the traditional IoU loss function struggles to optimize the relationship between predicted and ground truth bounding boxes, especially when aspect ratios are the same but actual dimensions differ, resulting in unsatisfactory performance. Finally, ordinary CNN structures fail to fully utilize inter-channel correlations, leading to the loss of some crucial information and affecting the final detection results. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method and system for detecting internal defects in sewers based on YOLOv11, which addresses the shortcomings of the existing technology.

[0006] The method includes:

[0007] Step 1: Preprocess the images of the inside of the sewer.

[0008] Step 2: Construct the improved YOLOv11 network structure, extract multi-scale features of the image, and generate feature maps of different scales;

[0009] Step 3: Introduce the channel attention mechanism SENet to enhance the model's attention to important features and generate channel-weighted feature maps at different scales;

[0010] Step 4: Perform adaptive feature fusion on the multi-scale channel weighted feature maps to generate a weighted fused feature map;

[0011] Step 5: Introduce the multi-parameter dynamic optimization intersection-union ratio loss function MPDIoU to optimize the bounding box regression and calculate the total loss; then backpropagate to optimize the training parameters.

[0012] Step 6: Output the final detection results;

[0013] In step 1, to ensure the input image meets the network's input requirements, it is scaled to conform to those requirements, for example, adjusted to a resolution of 640×640. To expand the diversity of the dataset, data augmentation operations are employed, such as random cropping, flipping, and color dithering. Specifically, random cropping alters the cropped area of ​​the image, enhancing the model's ability to recognize different parts of an object; image flipping helps the model learn symmetrical object features; and color dithering simulates different lighting and environmental conditions by adjusting the image's brightness, contrast, and saturation.

[0014] Furthermore, pixel values ​​are normalized to ensure they fall within the [0, 1] interval, guaranteeing a consistent numerical range across different image data. Data standardization involves normalizing the mean and variance of the image to ensure that the distribution of each pixel value is close to zero mean and unit variance. Specifically, the standardization formula is: Among them, among them, For standardized image data, For data augmentation of image data, and These are the mean and standard deviation of the image data, respectively.

[0015] In step 2, the YOLOv11 network structure includes a backbone network, a neck network, and a head network. These networks consist of multiple convolutional layers, downsampling layers, and activation function modules. Based on the YOLOv11 network structure, a processing layer incorporating the SENet (Squeeze-and-Excitation Network) module is added after layers 16, 19, and 22 of the neck network structure, and an adaptive feature fusion module is added to the head network. In the improved network structure, by processing the image layer by layer, layers 16, 20, and 24 generate feature maps of three scales: low-level feature map P3, mid-level feature map P4, and high-level feature map P5. These feature maps represent information at different levels: low-level feature map P3 retains more spatial detail information, such as basic features like edges and textures; mid-level feature map P4 retains some spatial detail information and captures some semantic information, acting as a transition between low-level and high-level features; high-level feature map P5 mainly captures high-level semantic information, such as object category and location, and is suitable for detecting complex semantic patterns and large targets. After an image is input into the network, basic features are first extracted through two or more convolutional operations. Then, the spatial resolution of the feature map is reduced through progressive downsampling and convolutional operations, while the number of channels is increased to enhance the feature representation capability, thereby gradually generating multi-scale feature maps. This design can fully utilize the complementarity of low-level, mid-level, and high-level features, improving the detection accuracy of small targets while strengthening the recognition capability of complex targets.

[0016] In step 3, SENet is an effective channel attention mechanism that can automatically learn the relationship between different channels and enhance attention to key information, thereby improving the model's representation ability.

[0017] In this invention, the network architecture weights are redefined, assigning a continuous value between 0 and 1 as the architecture weight to candidate operations on feature maps (P3, P4, P5) at each scale. These weights represent the importance of each scale and are dynamically adjusted based on performance during training. For example, the weight of layer P3 may be higher when detecting small targets, while the weight of layer P5 will be more important when detecting large targets.

[0018] Architecture weights are a set of parameters used in neural network design to represent the importance of different components or paths. These weights are typically used to control the flow of information within the network structure, determining which parts should be given more importance or ignored.

[0019] In this invention, the three layers of the SENet module are responsible for capturing target information at small, medium, and large scales, respectively. Targets at different scales have different feature distributions; therefore, applying SENet independently at each scale helps the model better focus on the key features of that scale. For each feature layer, a squeezing and activation process is first performed to generate a set of channel weights for the feature map of each layer. These weights are then applied to the corresponding feature map to strengthen features crucial for target detection at a specific scale while weakening less relevant features. This attention-based feature enhancement method helps reduce the impact of background noise and improves the model's target recognition accuracy.

[0020] The compression mentioned refers to compressing the feature map at each spatial location into a global descriptor, which is achieved through Global Average Pooling (GAP). GAP's role is to perform a spatial dimensional averaging calculation on the feature map of each channel, generating a vector of the same length as the number of channels. This vector is considered a comprehensive representation of all spatial information for that channel.

[0021] The aforementioned activation refers to constructing a small neural network using fully connected layers or convolutional layers. This network takes the vector generated during the squeezing stage as input and outputs a vector of the same length, where each element represents the weight of the corresponding channel. This process typically involves two fully connected layers sandwiched by a ReLU activation function and a sigmoid activation function to ensure that the output values ​​are between 0 and 1. These weights reflect the importance of each channel in the overall image.

[0022] Step 3 includes:

[0023] Step 3.1, Preprocess the initial feature map and perform channel separation: Separate the feature map F into c individual feature maps according to the number of channels. Among them, feature map This represents one of the three scale feature maps: P3, P4, and P5. It is a three-dimensional feature map. , Represents the input feature map The three-dimensional real space, The number of channels in a feature map, also known as depth. This represents the height of the feature map, i.e., the number of pixels in the feature map in the vertical direction. The width of the feature map is represented by the number of pixels in the horizontal direction, and n represents a positive integer in the interval [1, c]. This represents a single feature map of the nth channel. It is a two-dimensional feature map. Each channel represents a feature map. A specific feature pattern or information (such as edges, textures, colors, etc.) in the text.

[0024] To address the common issues of low contrast and high noise in sewer environments, an Adaptive Normalization (AN) technique is employed. AN normalizes the individual feature maps of each channel. The pixel values ​​are normalized to the range [0, 1]. Through local contrast enhancement, subtle edge and texture information in the feature map is amplified, ensuring that the final normalization result not only maintains good numerical stability but also highlights key features. The formula is:

[0025] ,

[0026] in, Indicates the feature map The result after normalization and These represent the mean and standard deviation of the channel characteristics, respectively. Represents a small positive number (e.g.) ), used to avoid the denominator being zero;

[0027] Step 3.2: Obtain the global representation of each channel using Global Average Pooling (GAP). Perform channel-by-channel calculations to transform the feature map Compressed into a scalar The formula is:

[0028] ,

[0029] in, This represents the pixel value of channel n at position (i, j). The normalization factor is represented by this formula. Based on this formula, and considering the unique characteristics of the sewer scene, such as its dim lighting, complex environment, and variable target shapes, the improved model considers the influence of the surrounding area when calculating the global representation. It uses a weighted approach to more accurately capture small targets or key parts. The improved formula is:

[0030] ,

[0031] in, This represents the similarity between each pixel in the feature map and its surrounding neighboring pixels. ,in, It is a trainable parameter. Initial value 0.5, and These are spatial distance weight and feature similarity weight, respectively. ,in, and These are the x and y coordinates of the pixel in the neighborhood of position (i, j) for channel n, respectively. It is (i, j) and ( The standard deviation of the distance Representing the exponential function , ,in, Indicates that channel n is at position ( The pixel value of );

[0032] Step 3.3, generate the global representation of the c channels. The features are aggregated to form a high-dimensional feature z of dimension c. Projecting z into a low-dimensional space reduces computational complexity and achieves information compression. A linear transformation is then performed, as shown in the formula: ,in, Represents a dimension reduction matrix. This represents the scaling factor (usually 16), and the dimensionality-reduced feature vector. This indicates the compressed channel relationship. , This represents the channel dimension (i.e., the number of channels); then... By applying the ReLU activation function, nonlinear characteristics are introduced, as shown in the formula. ,in, express The result after processing with the ReLU activation function, It is the ReLU activation function. ;

[0033] Step 3.4 involves remapping the low-dimensional features to high-dimensional channel weights, performing a dimensionality-up transformation. Specifically, this involves transforming the dimensionality-reduced feature vectors... Through an upgraded matrix Map back to the original number of channels This generates the channel weight vector. To improve model flexibility, a multi-scale fusion strategy (MSF) is introduced. MSF allows the model to learn from information at multiple scales (edges, textures, shapes, layouts, etc.), enhancing its understanding of complex scenes. The formula is as follows: ,in, Represents an upgraded matrix. This represents the generated channel weight vector, which is the set of channel weights for all channels; after further normalization, S is applied. The gmoid activation function converts the channel weight vector Limited to Range, formula is The Sigmoid function is defined as follows: , express After S Channel weight vector after processing by the gmoid activation function;

[0034] Step 3.5, perform weight expansion, and... Channel weights for each channel Expanded into a matrix with the same dimensions as the feature map, the formula is as follows: .in, This indicates the channel weights. Expanded to feature map The weight matrix with consistent spatial dimensions, i.e., the extended weight matrix of the nth channel. The dimension is indicated as height. width A matrix of all 1s; considering the differences in importance at different locations, a spatially-aware weighted fusion mechanism (SAWF) is introduced, with the formula as follows: ,in, This represents the nth channel feature map after channel weighting, which incorporates channel weight information;

[0035] Step 3.6: Recombine all the weighted channel feature maps to ensure that the final generated feature map not only contains rich spatial and semantic information, but also maximizes the use of the channel weight information adjusted by the attention mechanism. The formula is as follows: .in, This represents the output feature map generated after fusing the channel weights; that is, the channel-weighted feature map, which includes the input feature map. Spatial and semantic information, as well as channel weights adjusted through the attention mechanism;

[0036] In step 4, the adaptive feature fusion is a technique used to improve the feature extraction capability of neural networks. It allows the model to automatically adjust the weights of feature maps at different scales based on the input data, thereby better capturing target information at different scales. Specifically, the adaptive feature fusion module can spatially filter conflicting information, suppress inconsistencies between features at different scales, and enhance scale invariance.

[0037] Step 4 includes:

[0038] Step 4.1: Perform alignment processing of multi-scale feature maps;

[0039] Step 4.2, generate weights;

[0040] Step 4.3, weighted fusion of feature maps;

[0041] Step 4.4: Perform feature re-optimization.

[0042] Step 4.1 includes:

[0043] Step 4.1.1: Receive the channel-weighted feature maps obtained by processing the feature maps at three scales (P3, P4, and P5) using the SENet attention mechanism. , , .in, This represents the channel-weighted feature map obtained from the low-level feature map P3 after step 3. This represents the channel-weighted feature map obtained from the middle layer feature map P4 after step 3. This represents the channel-weighted feature map obtained from the high-level feature map P5 after step 3;

[0044] Step 4.1.2: To ensure effective fusion of feature maps at different scales, an upsampling operation is used to adjust all received feature maps to a uniform resolution, i.e. Adjust to Consistent resolution, the formula is ,in, express , , The feature map obtained after upsampling, UpSample It is an upsampling operation;

[0045] Step 4.1.3, perform feature alignment. During the alignment process, in order to ensure... Effective fusion of feature maps at three scales requires channel alignment using 1×1 convolutions to address the inconsistency in the number of channels across different scales. Specifically, currently... The feature map has 256 channels. The feature map has 512 channels. The feature map has 1024 channels. The number of channels has been increased to 512. The number of channels is reduced to 512, as shown in the formula:

[0046]

[0047] in, This represents a 1×1 convolution operation. This represents the feature-aligned feature map obtained after dimensionality reduction or dimensionality increase of the feature map channels, where... , , They represent the corresponding Feature maps obtained after feature alignment processing at three different scales.

[0048] Step 4.2 includes:

[0049] Step 4.2.1, Initialization , , Weight parameters of the three feature maps ,in, The weight parameters are initialized to positive values ​​with equal mean (e.g., ...). This indicates that in the initial stage, the contributions of the three feature maps are equal;

[0050] Step 4.2.2 introduces a dynamic weight parameter adjustment mechanism, enabling the network to automatically learn the most suitable weight allocation scheme based on the characteristics of the sewer dataset: ensuring the sum of the weight parameters equals 1, as shown in the formula. , where the function The Softmax function is a commonly used normalization function used to map a set of input values ​​to the range [0, 1] and make their sum equal to 1. This represents the initial weight values ​​for each feature map. , This means that each input value By amplifying using an exponential function, we ensure the value is positive. Represents all input values The sum of exponents, where α, β, and γ are calculated weight parameters, corresponding to... exist The calculation results in the function, in addition, α, β, γ are trainable weight parameters. In each training, the model will automatically learn which scales of features are more important to the current detection task, and thus update these weight parameters for the next training. For example, if the P3 scale features are more important to the object detection task, the network will learn to increase the value of α and decrease the value of β or γ.

[0051] Step 4.3 includes:

[0052] Step 4.3.1: Taking into account the information from each feature map, and introducing the degree of importance given to features at different scales through weight parameters. Specifically, the weight parameters α, β, and γ are correlated with each feature map. , , Element-wise multiplication yields the weighted feature map. The formula is: .

[0053] Step 4.4 includes:

[0054] Step 4.4.1: Adjust the number of channels in the feature map. Improve the feature representation capability through nonlinear transformation, enabling the model to achieve better performance with less computational resources. This applies to the fused feature map. Further optimization is achieved by applying 1×1 convolutional layer operations, as shown in the formula. ,in, express The feature map obtained after a 1×1 convolution operation;

[0055] Step 4.4.2 combines Batch Normalization and ReLU activation to stabilize the feature distribution, accelerate training, and prevent the vanishing gradient problem. The formula is as follows: ,in, Representation of feature map Normalized and activated output feature map; ,in, and Indicates the current feature map The mean and variance of each channel's data. and Initialized to 1 and 0 respectively, these represent the trainable parameters of the network, used for scaling and translation to enhance the model's expressive power; Specifically, The function sets input values ​​less than 0 to 0, while keeping values ​​greater than 0 unchanged.

[0056] Step 4.4.3, process the feature map Further optimization is achieved by performing 3×3 convolution operations to maintain computational efficiency while avoiding overfitting. The formula is as follows: ,in express The feature map obtained after 3×3 convolution operation This represents a 3×3 convolution operation;

[0057] In step 5, after weighted fusion, the fused feature map is fed into the detection head for target category prediction and bounding box regression. The bounding box regression loss function is improved by dynamically optimizing the intersection-union ratio (MPDIoU) loss function using multiple parameters. Specifically, the steps are as follows:

[0058] Step 5.1, feature map branch processing;

[0059] Step 5.2, calculate the classification loss;

[0060] Step 5.3: Calculate the loss function MPDIoU to improve the regression loss;

[0061] Step 5.4, calculate the confidence loss;

[0062] Step 5.5, Loss Optimization and Weight Balancing;

[0063] Step 5.1 includes:

[0064] Step 5.1.1, the detection head will merge the feature maps. The system consists of two branches: a classification branch and a regression branch. The classification branch predicts the target class distribution for each anchor box, and its output is... ,in This refers to the number of categories. The regression branch is used to regress the bounding box parameters for each anchor box, including the center point coordinates. and size ,in, Indicates width, referring to the horizontal span of the bounding box. The height represents the vertical span of the bounding box, and the output of the regression branch is... ;

[0065] Step 5.1.2: The classification branch calculates the class probability distribution of each anchor box using the Softmax function, as shown in the formula. The regression branch directly outputs the bounding box parameters, as shown in the formula. ;

[0066] Step 5.2 includes:

[0067] Step 5.2.1: Calculate the classification loss using cross-picking loss. :

[0068] ,

[0069] in, The number of positive samples. Indicates the first positive sample The true label of a sample in the k-th class Indicates the first positive sample The predicted probability of a sample in the kth class Data source , This represents the number of positive samples.

[0070] Step 5.3 includes:

[0071] Step 5.3.1, calculate the Intersection over Union (IoU):

[0072] ,

[0073] in, Represents the prediction box and real frame The area of ​​the intersection region, Represents the prediction box and real frame The area of ​​the union region, Data source , The data originates from the parameters of the initial real image;

[0074] Step 5.3.2: Calculate the Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box. :

[0075] ,

[0076] in, Represents the prediction box Center coordinates Represents a real box Center coordinates;

[0077] Step 5.3.3: Calculate the minimum distance from the ground truth bounding box keypoints to the predicted bounding box boundaries. :

[0078] ,

[0079] in The first line representing the true bounding box The coordinates of the key points The first prediction box The coordinates of the boundary points;

[0080] Step 5.3.4, calculate MPDIoU loss. The formula is:

[0081] ,

[0082] in, The normalization coefficient is represented by the diagonal length of the smallest bounding rectangle containing both the predicted and ground truth boxes. This indicates the number of key points in the ground truth bounding box. and Indicates the weighting coefficient;

[0083] Step 5.4 includes:

[0084] Step 5.4.1, calculate the confidence loss. :

[0085] ,

[0086] in, This represents the total number of samples. Indicates the first in the total sample True confidence level of a sample Indicates the first in the total sample The prediction confidence level of each sample;

[0087] Step 5.5 includes:

[0088] Step 5.5.1: Taking into account classification, regression, and confidence losses, calculate the total loss. The formula is:

[0089] ,

[0090] in, Indicates the weighting coefficient;

[0091] Step 5.5.2 updates the parameters through backpropagation to maximize detection performance. For each trainable parameter... (e.g., α, β, γ, etc.), calculate the total loss pair The gradient (partial derivative) of . The gradient reflects the extent to which the current parameters affect the total loss, and is used to guide the adjustment of the parameters. Specifically, in the Adam optimization algorithm, the learning rate is used... (Take a value of 0.01) Update the trainable parameters using the following formula: To achieve the goal of updating model parameters, the model is trained multiple times for detecting defects inside sewers. Each training session repeats step 2 (extracting multi-scale features from the image and generating feature maps of different scales), steps 3, 4, and 5 to optimize the parameters and improve the model's detection performance.

[0092] In step 6, after 100 training iterations, the final detection results include data such as average accuracy, target category, bounding box coordinates, and confidence level, which are used for analysis and decision-making in the sewer interior scene.

[0093] This invention also provides a sewer internal defect detection system based on an improved YOLOv11, comprising:

[0094] The data preprocessing module is used to process image data collected inside the sewer, including image enhancement, size normalization, and data annotation.

[0095] The feature extraction and fusion module enhances the model's ability to capture multi-scale features and optimizes the expression of spatial features by introducing the SENet attention mechanism and adaptive feature fusion module.

[0096] The target detection module, based on the improved YOLOv11 model, introduces the MPDIoU loss function to achieve accurate bounding box prediction and target localization;

[0097] The results output module performs post-processing on the detection results, including target classification, defect marking, and confidence score, and outputs them in the form of image overlay and statistical reports.

[0098] This invention effectively improves the performance of the YOLOv11 model in sewer defect detection by introducing an adaptive feature fusion module, an MPDIoU loss function, and a SENet attention mechanism. The adaptive feature fusion module enhances the model's ability to express multi-scale information by dynamically weighting and fusing feature maps of different scales, thus better adapting to the complex environment inside sewers. The MPDIoU loss function improves the accuracy of detection boxes by refining the bounding box regression loss, performing particularly well when handling complex defects. The SENet attention mechanism adaptively adjusts the weights of channel features, highlighting key regions and effectively suppressing interference from irrelevant information. Combining these innovations, this invention significantly improves the model's detection accuracy and robustness for defects such as cracks, corrosion, and obstacles in sewers, overcoming the shortcomings of the traditional YOLOv11 model in complex environments, and providing an efficient and accurate new solution for the field of automatic sewer detection.

[0099] The beneficial effects of this invention are as follows: This invention proposes an efficient detection scheme based on an improved YOLOv11 model for detecting defects inside sewers. This scheme significantly improves the model's detection performance by introducing an adaptive spatial feature fusion module, an MPDIoU loss function, and a SENet attention mechanism. The adaptive feature fusion module can dynamically fuse multi-scale feature maps, enhancing the model's ability to capture diverse features in complex environments, particularly excelling in the dimly lit and structurally complex conditions inside sewers. The MPDIoU loss function optimizes the bounding box regression process, improving target localization accuracy through joint constraints of center point distance and keypoint minimum distance, thus solving the problem of insufficient localization accuracy in traditional methods. The SENet attention mechanism further enhances the expressive power of feature maps, making the model more robust in detecting multiple types of defects. Through the organic combination of these three innovations, this invention not only improves detection accuracy but also maintains high inference speed, exhibiting excellent real-time detection capabilities. It is particularly suitable for efficient and accurate detection of defects inside sewers, possessing high practical value and broad application prospects. Attached Figure Description

[0100] Figure 1 This is a flowchart of the method of the present invention.

[0101] Figure 2 This is a diagram of the overall network structure of the improved YOLOv11 in this invention.

[0102] Figure 3 This is a network structure diagram of the SENet module in this invention.

[0103] Figure 4 This is a network structure diagram of the adaptive feature fusion module in this invention.

[0104] Figure 5 This is a network structure diagram of the detection head module in this invention.

[0105] Figure 6 This is a diagram illustrating the average accuracy mAP50.

[0106] Figure 7 This is a schematic diagram comparing the average accuracy of each training iteration with the original YOLOv11 model. Detailed Implementation

[0107] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention above and in other aspects will become clearer.

[0108] like Figure 1As shown, this embodiment of the invention provides a method for detecting internal defects in sewer systems based on an improved version of YOLOv11, comprising the following steps:

[0109] Step 1, as follows Figure 1 The image of the sewer interior is preprocessed. The size of the input sewer interior image X is usually not fixed, so preprocessing is necessary to adapt it to the network's input requirements. The sewer interior image X is scaled to the target resolution, such as 640×640, and undergoes data augmentation operations (such as random cropping, flipping, color dithering, etc.) to expand the diversity of the dataset. Subsequently, the pixel values ​​are normalized to the [0, 1] interval and standardized to zero mean and unit variance, as shown in the formula... ,in, This refers to standardized image data. This refers to image data after data augmentation. This refers to the mean of the image data. This refers to the standard deviation of the image data. These preprocessing steps can significantly improve the model's generalization ability.

[0110] Step 2, as follows Figure 1 and Figure 2 The YOLOv11 network architecture consists of a backbone network, a neck network, and a head network. These networks include multiple convolutional layers, downsampling layers, and activation function modules. Building upon the YOLOv11 network structure, a processing layer incorporating the SENet module is added after layers 16, 19, and 22 of the neck network, and an adaptive feature fusion module is added to the head network. In this improved network architecture, images are processed layer by layer, with layers 16, 20, and 24 generating feature maps at three scales: P3, P4, and P5. These feature maps represent different levels of information: the low-level feature map P3 retains more spatial details, such as edges and textures; the mid-level feature map P4 retains some spatial details and captures some semantic information, acting as a transition between low- and high-level features; and the high-level feature map P5 primarily captures high-level semantic information, such as object category and location, suitable for detecting complex semantic patterns and large objects. After an image is input into the network, basic features are first extracted through two or more convolutional operations. Then, the spatial resolution of the feature map is reduced through progressive downsampling and convolutional operations, while the number of channels is increased to enhance the feature representation capability, thereby gradually generating multi-scale feature maps. This design can fully utilize the complementarity of low-level, mid-level, and high-level features, improving both the detection accuracy for small targets and the recognition capability for complex targets.

[0111] Step 3, as follows Figure 1 and Figure 3After feature extraction, to further enhance feature representation capabilities, this invention introduces the SENet attention mechanism to weight multi-scale features. SENet uses a channel weighting strategy to make the network pay more attention to key features.

[0112] Step 3 includes:

[0113] Step 3.1, Preprocess the initial feature map and perform channel separation: Input feature map Separate into c individual feature maps based on the number of channels. ,in, This represents one of the three scale feature maps: P3, P4, and P5. It is a three-dimensional feature map. , Represents the input feature map The three-dimensional real space, middle The number of channels in a feature map, also known as depth. This represents the height of the feature map, i.e., the number of pixels in the feature map in the vertical direction. The width of the feature map is represented by the number of pixels in the horizontal direction, and n represents a positive integer in the interval [1, c]. This represents a single feature map of the nth channel. It is a two-dimensional feature map. Each channel represents a feature map. A specific feature pattern or information (such as edges, textures, colors, etc.) in the text.

[0114] To address the common issues of low contrast and high noise in sewer environments, an Adaptive Normalization (AN) technique is employed. AN normalizes the individual feature maps of each channel. The pixel values ​​are normalized to the range [0, 1]. Through local contrast enhancement, subtle edge and texture information in the feature map is amplified, ensuring that the final normalization result not only maintains good numerical stability but also highlights key features. The formula is... ,in, Indicates the feature map The result after normalization and These represent the mean and standard deviation of the channel characteristics, respectively. It represents a small positive number (e.g., 1e−5) to avoid the denominator being zero;

[0115] Step 3.2: Obtain the global representation of each channel using Global Average Pooling (GAP). Perform channel-by-channel calculations to transform the feature map Compressed into a scalar The formula is ,in, This represents the pixel value of channel n at position (i, j). Let represent the normalization factor. Based on this formula, and considering the unique characteristics of the sewer scene, such as its dim lighting, complex environment, and variable target shapes, the improved model considers the influence of the surrounding area when calculating the global representation. It uses a weighted approach to more accurately capture small targets or key parts. The improved formula is: ,in, This represents the similarity between each pixel in the feature map and its surrounding neighboring pixels. ,in, It is a trainable parameter. Initial value 0.5, and These are spatial distance weight and feature similarity weight, respectively. ,in, and These are the x and y coordinates of the pixel in the neighborhood of channel n at position (i, j), respectively. It is (i, j) and ( The standard deviation of the distance Representing the exponential function , ,in, Indicates that channel n is at position ( The pixel value of );

[0116] Step 3.3, generate the global representation of the c channels. The features are aggregated to form a high-dimensional feature z of dimension c. Projecting z into a low-dimensional space reduces computational complexity and achieves information compression. A linear transformation is then performed, as shown in the formula: ,in, Represents a dimension reduction matrix. This represents the scaling factor (usually 16), and the dimensionality-reduced feature vector. This indicates the compressed channel relationship. , This represents the channel dimension (i.e., the number of channels); then... By applying the ReLU activation function, nonlinear characteristics are introduced, as shown in the formula. ,in, express The result after processing with the ReLU activation function, It is the ReLU activation function. ;

[0117] Step 3.4 involves remapping the low-dimensional features to high-dimensional channel weights, performing a dimensionality-up transformation. Specifically, this involves transforming the dimensionality-reduced feature vectors... Through an upgraded matrix Map back to the original number of channels This generates the channel weight vector. To improve model flexibility, a multi-scale fusion strategy (MSF) is introduced. MSF allows the model to learn from information at multiple scales (edges, textures, shapes, layouts, etc.), enhancing its understanding of complex scenes. The formula is as follows: ,in, This represents an upgraded matrix that maps back to the original number of channels. This represents the generated channel weight vector, which is the set of channel weights for all channels; after further normalization, S is applied. The gmoid activation function converts the channel weight vector Limited to Range, formula is The Sigmoid function is defined as follows: ;

[0118] Step 3.5, perform weight expansion, and... Channel weights for each channel Expanded into a matrix with the same dimensions as the feature map, the formula is as follows: .in, This indicates the channel weights. Expanded to feature map The weight matrix with consistent spatial dimensions, i.e., the extended weight matrix of the nth channel. The dimension is indicated as height. width The matrix consists of all 1s. Considering the differences in importance at different positions, a spatial-aware weighted fusion (SAWF) mechanism is introduced. SAWF uses a convolutional neural network to extract spatial information from the original feature map and integrates it into the matrix. In the middle, the formula is ,in, This represents the nth channel feature map after channel weighting, which incorporates channel weight information, where n represents a positive integer in the interval [1, c].

[0119] Step 3.6: Recombine all the weighted channel feature maps to ensure that the final generated feature map not only contains rich spatial and semantic information, but also maximizes the use of the channel weight information adjusted by the attention mechanism. The formula is as follows: .in, This represents the output feature map generated after fusing the channel weights; that is, the channel-weighted feature map, which includes the input feature map. Spatial and semantic information, as well as channel weights adjusted through the attention mechanism;

[0120] Step 4, as follows Figure 1 and Figure 4 In order to fully integrate features at different scales, this invention proposes an adaptive feature fusion module, which avoids information loss and improves the efficiency of multi-scale feature fusion by dynamically adjusting the weights of feature maps at each scale.

[0121] Step 4 includes:

[0122] Step 4.1, Alignment processing of multi-scale feature maps;

[0123] Step 4.2, generate weights;

[0124] Step 4.3, weighted fusion of feature maps;

[0125] Step 4.4, feature re-optimization;

[0126] Step 4.1 includes:

[0127] Step 4.1.1: Receive the channel-weighted feature maps obtained by processing the feature maps at three scales (P3, P4, and P5) using the SENet attention mechanism. , , .in, This represents the channel-weighted feature map obtained from the P3 scale feature map after step 3. This represents the channel-weighted feature map obtained from the P4 scale feature map after step 3. This represents the channel-weighted feature map obtained from the P5 scale feature map after step 3;

[0128] Step 4.1.2: To ensure effective fusion of feature maps at different scales, an upsampling operation is used to adjust all received feature maps to a uniform resolution, i.e. Adjust to Consistent resolution, the formula is ,in, express , , The feature map obtained after upsampling, UpSample It is an upsampling operation;

[0129] Step 4.1.3, perform feature alignment. During the alignment process, in order to ensure... Effective fusion of feature maps at three scales requires channel alignment using 1×1 convolutions to address the inconsistency in the number of channels across different scales. Specifically, currently... The feature map has 256 channels. The feature map has 512 channels. The feature map has 1024 channels. The number of channels has been increased to 512. The number of channels is reduced to 512, as shown in the formula:

[0130]

[0131] in, This represents a 1×1 convolution operation. This represents the feature-aligned feature map obtained after dimensionality reduction or dimensionality increase of the feature map channels, including , , , respectively represent the corresponding Feature maps obtained after feature alignment processing at three different scales;

[0132] Step 4.2 includes:

[0133] Step 4.2.1, Initialization , , Weight parameters of the three feature maps ,in, The weight parameters are initialized to positive values ​​with equal mean (e.g., ...). This indicates that in the initial stage, the contributions of the three feature maps are equal;

[0134] Step 4.2.2 introduces a dynamic weight parameter adjustment mechanism, enabling the network to automatically learn the most suitable weight allocation scheme based on the characteristics of the sewer dataset: ensuring the sum of the weight parameters equals 1, as shown in the formula. , where the function The Softmax function is a commonly used normalization function used to map a set of input values ​​to the range [0, 1] and make their sum equal to 1. This represents the initial weight values ​​for each feature map. , This means that each input value By amplifying using an exponential function, we ensure the value is positive. Represents all input values The sum of exponents, where α, β, and γ are calculated weight parameters, corresponding to... exist The calculation results in the function, in addition, α, β, γ are trainable weight parameters. In each training, the model will automatically learn which scales of features are more important to the current detection task, and thus update these weight parameters for the next training. For example, if the P3 scale features are more important to the object detection task, the network will learn to increase the value of α and decrease the value of β or γ.

[0135] Step 4.3 includes:

[0136] Step 4.3.1: Taking into account the information from each feature map, and introducing the degree of importance given to features at different scales through weight parameters. Specifically, the weight parameters α, β, and γ are correlated with each feature map. , , Element-wise multiplication yields the weighted feature map. The formula is: ;

[0137] Step 4.4 includes:

[0138] Step 4.4.1: Adjust the number of channels in the feature map. Improve the feature representation capability through nonlinear transformation, enabling the model to achieve better performance with less computational resources. This applies to the fused feature map. Further optimization is achieved by applying 1×1 convolutional layer operations, as shown in the formula. ,in, express The feature map obtained after a 1×1 convolution operation;

[0139] Step 4.4.2 combines Batch Normalization and ReLU activation to stabilize the feature distribution, accelerate training, and prevent the vanishing gradient problem. The formula is as follows: ,in, Representation of feature map Normalized and activated output feature map; ,in, and Indicates the current feature map The mean and variance of each channel's data. and Initialized to 1 and 0 respectively, these represent the trainable parameters of the network, used for scaling and translation to enhance the model's expressive power; Specifically, The function sets input values ​​less than 0 to 0, while keeping values ​​greater than 0 unchanged.

[0140] Step 4.4.3, process the feature map Further optimization is achieved by performing 3×3 convolution operations to maintain computational efficiency while avoiding overfitting. The formula is as follows: ,in express The feature map obtained after 3×3 convolution operation This represents a 3×3 convolution operation;

[0141] Step 5, as follows Figure 1 and Figure 5 After feature fusion, the fused feature map is fed into the detection head for target prediction.

[0142] Step 5 includes:

[0143] Step 5.1, feature map branch processing;

[0144] Step 5.2, calculate the classification loss;

[0145] Step 5.3: Calculate the loss function MPDIoU to improve the regression loss;

[0146] Step 5.4, calculate the confidence loss;

[0147] Step 5.5, Loss Optimization and Weight Balancing;

[0148] Step 5.1 includes:

[0149] Step 5.1.1, the detection head will merge the feature maps. The system consists of two branches: a classification branch and a regression branch. The classification branch predicts the target class distribution for each anchor box, and its output is... ,in This refers to the number of categories. The regression branch is used to regress the bounding box parameters for each anchor box, including the center point coordinates. and size ,in, Indicates width, referring to the horizontal span of the bounding box. The height represents the vertical span of the bounding box, and the output of the regression branch is... ;

[0150] Step 5.1.2: The classification branch calculates the class probability distribution of each anchor box using the Softmax function, as shown in the formula. The regression branch directly outputs the bounding box parameters, as shown in the formula. ;

[0151] Step 5.2 includes:

[0152] Step 5.2.1: Calculate the classification loss using cross-picking loss. :

[0153] ,

[0154] in, The number of positive samples. Indicates the first positive sample The true label of a sample in the k-th class Indicates the first positive sample The predicted probability of a sample in the kth class Data source ;

[0155] Step 5.3 includes:

[0156] Step 5.3.1, calculate the Intersection over Union (IoU):

[0157] ,

[0158] in, Represents the prediction box and real frame The area of ​​the intersection region, Represents the prediction box and real frame The area of ​​the union region, Data source , The data originates from the parameters of the initial real image;

[0159] Step 5.3.2: Calculate the Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box. :

[0160] ,

[0161] in, Represents the prediction box Center coordinates Represents a real box Center coordinates;

[0162] Step 5.3.3: Calculate the minimum distance from the ground truth bounding box keypoints to the predicted bounding box boundaries. :

[0163] ,

[0164] in The first line representing the true bounding box The coordinates of the key points The first prediction box The coordinates of the boundary points;

[0165] Step 5.3.4, calculate MPDIoU loss. The formula is:

[0166] ,

[0167] in, The normalization coefficient is represented by the diagonal length of the smallest bounding rectangle containing both the predicted and ground truth boxes. This indicates the number of key points in the ground truth bounding box. and Indicates the weighting coefficient;

[0168] Step 5.4 includes:

[0169] Step 5.4.1, calculate the confidence loss. :

[0170] ,

[0171] in, This represents the total number of samples. Indicates the first in the total sample True confidence level of a sample Indicates the first in the total sample The prediction confidence level of each sample;

[0172] Step 5.5 includes:

[0173] Step 5.5.1: Taking into account classification, regression, and confidence losses, calculate the total loss. The formula is:

[0174] ,

[0175] in, Indicates the weighting coefficient;

[0176] Step 5.5.2 updates the parameters through backpropagation to maximize detection performance. For each trainable parameter... (e.g., α, β, γ, etc.), calculate the total loss pair The gradient (partial derivative) of . The gradient reflects the extent to which the current parameters affect the total loss, and is used to guide the adjustment of the parameters. Specifically, in the Adam optimization algorithm, the learning rate is used... (Take a value of 0.01) Update the trainable parameters using the following formula: To achieve the goal of updating model parameters, the model is trained multiple times for detecting defects inside sewers. Each training session repeats step 2 (extracting multi-scale features from the image and generating feature maps of different scales), steps 3, 4, and 5 to optimize the parameters and improve the model's detection performance.

[0177] Step 6: Output the final detection results;

[0178] Compared with existing technologies, the method of this invention significantly improves detection performance, especially in terms of accuracy and multi-scale target adaptability. It can more accurately detect various targets in sewers, such as cracks, obstacles, corrosion, deformation, rupture, sediment, misalignment, hidden branch pipe connections, foreign object penetration, interface material detachment, and disconnection, demonstrating strong robustness and wide applicability. Under the same experimental conditions (model size m, no pre-trained weights, NVIDIA 4090 graphics card, etc., where m: [0.50, 1.00, 512], m represents medium, 0.50 represents the model depth scaling factor of 0.50, used to control the number of network layers; 1.00 represents the model width scaling factor of 1.00, used to control the number of channels per convolutional layer; 512 represents the maximum number of channels allowed per convolutional layer in the network), and with a training time of 100 epochs, Box P (precision) improved from 79.3% to 83.5%, Box R (recall) improved from 63.1% to 69.7%, mAP50 improved from 70.9% to 78.9%, and mAP50-95 improved from 49.3% to 54.7%. Specifically, step 1 involves constructing an improved YOLOv11 network structure, fusing an adaptive feature fusion module, the MPDIoU loss function, and the SENet attention mechanism. Step 2 involves training the network 100 times on the sewer dataset, using the same experimental environment, including the same model size m, no pre-trained weights, and an NVIDIA 4090 graphics card, to obtain the training results. The average accuracy mAP50 obtained in step 2 is as follows: Figure 6 As shown;

[0179] During training, this invention compared the average accuracy of each training iteration with that of the original YOLOv11 model (using the same experimental environment), such as... Figure 7 As shown, compared to YOLOv11, which improves slowly in the early stages of training and eventually converges to a low mAP50 value (i.e., the lower convergence effect shown by the orange curve), the method of this invention demonstrates significant advantages. The improved model can improve accuracy more quickly and maintain a stable optimization trend in the mid-to-late stages of training, eventually converging to higher validation accuracy.

[0180] This invention provides a method and system for detecting internal defects in sewers based on an improved version of YOLOv11. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for detecting internal defects in sewer systems based on an improved version of YOLOv11, characterized in that, Includes the following steps: Step 1: Preprocess the images of the inside of the sewer. Step 2: Construct the improved YOLOv11 network structure, extract multi-scale features of the image, and generate feature maps of different scales; Step 3: Introduce the channel attention mechanism SENet to enhance the model's attention to important features and generate channel-weighted feature maps at different scales; Step 3 includes: Step 3.1, Preprocess the initial feature map and perform channel separation: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Separate into c individual feature maps based on the number of channels c. Among them, feature map This represents one of the three scale feature maps: P3, P4, and P5. , Represents the input feature map The three-dimensional real space, This represents the number of channels in the feature map. Indicates the height of the feature map. The width of the feature map is represented by n, which represents a positive integer in the interval [1, c]. This represents a single feature map of the nth channel. It is a two-dimensional feature map. ; Adaptive Normalization (AN) is used to transform the individual feature maps of each channel. The pixel values ​​are normalized to the range [0, 1] using the following formula: ， in, Indicates the feature map The result after normalization and These represent the mean and standard deviation of the channel characteristics, respectively. Represent a positive number; Step 3.2: Obtain the global representation of each channel using Global Average Pooling (GAP). Perform channel-by-channel calculations to transform the feature map Compressed into a scalar The formula is: ， in, This represents the pixel value of channel n at position (i, j). Represents the normalization factor. The improved formula is: ， in, This represents the similarity between each pixel in the feature map and its surrounding neighboring pixels. ,in, It is a trainable parameter. and These are spatial distance weight and feature similarity weight, respectively. ,in, and These are the x and y coordinates of the pixel in the neighborhood of position (i, j) for channel n, respectively. It is (i, j) and ( The standard deviation of the distance Represents an exponential function. ,in, Indicates that channel n is at position ( The pixel value of ); Step 3.3, generate the global representation of the c channels. The features are aggregated to form a high-dimensional feature z of dimension c. Projecting z into a low-dimensional space reduces computational complexity and achieves information compression. A linear transformation is then performed, as shown in the formula: ,in, Represents a dimension reduction matrix. Represents the scaling factor, and the dimensionality-reduced feature vector. This indicates the compressed channel relationship. , Indicate the channel dimension; then... By applying the ReLU activation function, nonlinear characteristics are introduced, as shown in the formula. ,in, express The result after processing with the ReLU activation function, It is the ReLU activation function; Step 3.4: Remap the low-dimensional features to high-dimensional channel weights, performing a dimensionality-up transformation: transform the dimensionality-reduced feature vectors... Through an upgraded matrix Map back to the original number of channels This generates the channel weight vector. A multi-scale fusion strategy, MSF, is introduced, with the following formula: ,in, Represents an upgraded matrix. This represents the generated channel weight vector; further normalization is then performed, applying S... The gmoid activation function converts the channel weight vector Limited to Range, formula is The Sigmoid function is defined as follows: , express After S Channel weight vector after processing by the gmoid activation function; Step 3.5, perform weight expansion, and... Channel weights for each channel Expanded into a matrix with the same dimensions as the feature map, the formula is as follows: ,in, This indicates the channel weights. Expanded to feature map The weight matrix with consistent spatial dimensions, i.e., the extended weight matrix of the nth channel. The dimension is indicated as height. width A matrix of all 1s; introducing the spatially aware weighted fusion mechanism SAWF, the formula is as follows: ,in, This represents the feature map of the nth channel after channel weighting; Step 3.6: Recombine all the weighted channel feature maps using the following formula: ,in, This represents the output feature map generated after fusing the channel weights; Step 4: Perform adaptive feature fusion on the multi-scale channel weighted feature maps to generate a weighted fused feature map; Step 5: Introduce the multi-parameter dynamic optimization intersection-union ratio loss function MPDIoU to optimize the bounding box regression and calculate the total loss; then backpropagate to optimize the training parameters. Step 6: Output the final detection results.

2. The method according to claim 1, characterized in that, In step 1, the input image is scaled and then data augmentation is performed; The pixel values ​​are normalized so that they fall within the range of [0, 1].

3. The method according to claim 2, characterized in that, In step 2, the YOLOv11 network structure includes a backbone network, a neck network, and a head network. Based on the YOLOv11 network structure, a processing layer that introduces the SENet module is added after layers 16, 19, and 22 of the neck network structure, namely layers 17, 21, and 25. An adaptive feature fusion module is added to the head network. In the improved network structure, images are processed layer by layer, and feature maps of three scales are generated at layers 16, 19, and 22 respectively: low-level feature map P3, mid-level feature map P4, and high-level feature map P5. After the image is input into the network, basic features are first extracted through two or more convolution operations. Then, the spatial resolution of the feature map is reduced by progressive downsampling and convolution operations, while the number of channels is increased to enhance the feature representation ability, thereby gradually generating multi-scale feature maps.

4. The method according to claim 3, characterized in that, Step 4 includes: Step 4.1: Perform alignment processing of multi-scale feature maps; Step 4.2, generate weights; Step 4.3, weighted fusion of feature maps; Step 4.4: Perform feature re-optimization.

5. The method according to claim 4, characterized in that, Step 4.1 includes: Step 4.1.1: Receive the channel-weighted feature maps obtained by processing the feature maps at three scales (P3, P4, and P5) using the SENet attention mechanism. , , ,in, This represents the channel-weighted feature map obtained from the low-level feature map P3 after step 3. This represents the channel-weighted feature map obtained from the middle layer feature map P4 after step 3. This represents the channel-weighted feature map obtained from the high-level feature map P5 after step 3; Step 4.1.2: Use upsampling to adjust all received feature maps to a uniform resolution, that is... Adjust to Consistent resolution, the formula is ,in, They represent , , The feature map obtained after upsampling, UpSample It is an upsampling operation; Step 4.1.3, perform feature alignment, and... The number of channels has been increased to 512. The number of channels is reduced to 512, and the formula is: ， in, This represents a 1×1 convolution operation. This represents the feature-aligned feature map obtained after dimensionality reduction or dimensionality increase of the feature map channels, where... , , They represent the corresponding Feature maps obtained after feature alignment processing at three different scales.

6. The method according to claim 5, characterized in that, Step 4.2 includes: Step 4.2.1, Initialization , , Weight parameters of the three feature maps ,in, The weight parameters are initialized to positive values ​​with equal mean. Step 4.2.2 introduces a dynamic weight parameter adjustment mechanism, enabling the network to automatically learn the most suitable weight allocation scheme based on the characteristics of the sewer dataset: ensuring the sum of the weight parameters equals 1, as shown in the formula. Where, the Softmax function is the normalization function, and α, β, and γ are the calculated weight parameters, respectively corresponding to... exist The result of calculations within the function.

7. The method according to claim 6, characterized in that, Step 4.3 includes: Step 4.3.1: Connect the weight parameters α, β, and γ with each feature map. , , Element-wise multiplication yields the weighted feature map. The formula is: .

8. The method according to claim 7, characterized in that, Step 4.4 includes: Step 4.4.1, process the fused feature map Further optimization is achieved by applying 1×1 convolutional layer operations, as shown in the formula. ,in, express The feature map obtained after a 1×1 convolution operation; Step 4.4.2 involves combining Batch Normalization and the ReLU activation function for processing, as shown in the formula: ,in, Representation of feature map Normalized and activated output feature map; ,in, and These represent the current feature maps. The mean and variance of each channel's data. and Initialize them to 1 and 0 respectively; ); Step 4.4.3, process the feature map Further optimization is achieved by performing a 3×3 convolution operation, with the following formula: ,in express The feature map obtained after 3×3 convolution operation This represents a 3×3 convolution operation; In step 5, after weighted fusion, the fused feature map is fed into the detection head for target category prediction and bounding box regression. The bounding box regression loss function is improved by dynamically optimizing the intersection-union ratio (MPDIoU) loss function using multiple parameters. Specifically, the steps are as follows: Step 5.1, feature map branch processing; Step 5.2, calculate the classification loss; Step 5.3: Calculate the loss function MPDIoU to improve the regression loss; Step 5.4, calculate the confidence loss; Step 5.5, Loss Optimization and Weight Balancing; Step 5.1 includes: Step 5.1.1, the detection head will merge the feature maps. The system consists of two branches: a classification branch and a regression branch. The classification branch predicts the target class distribution for each anchor box, and its output is... ,in This represents the number of categories; the regression branch is used to regress the bounding box parameters for each anchor box, which include the center point coordinates. and size ,in, Indicates width, referring to the horizontal span of the bounding box. The height represents the vertical span of the bounding box, and the output of the regression branch is... ; Step 5.1.2: The classification branch calculates the class probability distribution of each anchor box using the Softmax function, as shown in the formula. The regression branch directly outputs the bounding box parameters, as shown in the formula. ; Step 5.2 includes: Step 5.2.1: Calculate the classification loss using cross-picking loss. : ， in, The number of positive samples. Indicates the first positive sample The true label of a sample in the k-th class Indicates the first positive sample The predicted probability of a sample in the kth class Data source ; Step 5.3 includes: Step 5.3.1, calculate the Intersection over Union (IoU): ， in, Represents the prediction box and real frame The area of ​​the intersection region, Represents the prediction box and real frame The area of ​​the union region; Step 5.3.2: Calculate the Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box. : ， in, Represents the prediction box Center coordinates Represents a real box Center coordinates; Step 5.3.3: Calculate the minimum distance from the ground truth bounding box keypoints to the predicted bounding box boundaries. : ， in The first line representing the true bounding box The coordinates of the key points Represents the first prediction box The coordinates of the boundary points; Step 5.3.4, calculate MPDIoU loss. The formula is: ， in, Represents the normalization coefficient. This indicates the number of key points in the ground truth bounding box. and Indicates the weighting coefficient; Step 5.4 includes: Step 5.4.1, calculate the confidence loss. : ， in, This represents the total number of samples. Indicates the first in the total sample True confidence level of a sample Indicates the first in the total sample The prediction confidence level of each sample; Step 5.5 includes: Step 5.5.1, calculate the total loss. The formula is: ， in, Indicates the weighting coefficient; Step 5.5.2 updates the parameters through backpropagation to maximize detection performance.

9. A sewer internal defect detection system based on YOLOv11, implemented using the method described in claim 1, characterized in that, include: The data preprocessing module is used to process image data collected inside the sewer, including image enhancement, size normalization, and data annotation. The feature extraction and fusion module enhances the model's ability to capture multi-scale features and optimizes the expression of spatial features by introducing the SENet attention mechanism and adaptive feature fusion module. The target detection module, based on the improved YOLOv11 model, introduces the MPDIoU loss function to achieve accurate bounding box prediction and target localization; The results output module performs post-processing on the detection results, including target classification, defect marking, and confidence score, and outputs them in the form of image overlay and statistical reports.

Images