Methods and systems for identifying typical defects in drainage pipes using machine vision technology

Abstract

This invention relates to the field of visual recognition technology, specifically to a method and system for identifying typical defects in drainage pipes using machine vision technology. The method involves: capturing video of the pipe's interior and extracting frames to obtain raw drainage pipe image data; labeling the images with defect classification tags and defect region contours; constructing a typical defect recognition model for drainage pipes; performing multi-dimensional preprocessing and enhancement on the raw images; generating defect-sensitive features through a multi-scale, multi-channel attention fusion module; outputting relevant results through a parallel recognition and segmentation module; constructing a total loss function and training the model; inputting a pipe image with unknown defects into the trained model to complete automatic recognition, generating a defect contour mask and a recognition report. This invention can achieve automatic and accurate identification of pipe defects, significantly improving detection efficiency and accuracy, reducing labor costs, and is applicable to various typical defect detection scenarios in drainage pipes.

Description

Technical Field

[0001] This invention relates to the field of visual recognition technology, and in particular to a method and system for identifying typical defects in drainage pipes using machine vision technology. Background Technology

[0002] Regular, efficient, and accurate inspection and evaluation of drainage pipes has become a rigid requirement to ensure the safe operation of urban lifelines.

[0003] Existing technologies objectively suffer from the following shortcomings: conventional image preprocessing methods lack specificity for the complex lighting and noise characteristics of pipes, making it difficult to simultaneously enhance local contrast and restore details, and ignoring the spectral correlation between color image channels; conventional single-scale convolution or simple channel attention mechanisms are difficult to model the complex dependencies of defects across scales and channels, resulting in insufficient sensitivity to low-contrast defects; conventional recognition models often ignore the curved surface geometry of the pipe's inner wall and lack explicit modeling of the directional morphology of defects, resulting in low accuracy in the boundary segmentation of irregular defects and susceptibility to interference from complex backgrounds; conventional loss functions lack adaptive adjustment capabilities when faced with severe class imbalance and the coexistence of multi-scale defects, making it difficult to simultaneously optimize classification and segmentation tasks, resulting in insensitivity to small defects and coarse segmentation boundaries.

[0004] Therefore, this invention proposes a method and system for identifying typical defects in drainage pipes using machine vision technology to solve the above problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention develops a method and system for identifying typical defects in drainage pipes using machine vision technology. This invention can achieve automatic and accurate identification of pipe defects, significantly improve detection efficiency and accuracy, reduce labor costs, and is applicable to various typical defect detection scenarios in drainage pipes.

[0006] On the one hand, the technical solution of this invention to solve the technical problem is a method for identifying typical defects in drainage pipes using machine vision technology, comprising the following steps:

[0007] S1. Capture video of the internal environment of the pipeline, extract frame by frame from the continuous video stream to obtain the original drainage pipeline image data, and annotate the acquired image data, marking defect classification labels and the outline of defect areas.

[0008] S2. Construct a typical defect identification model for drainage pipes. The model first performs multi-dimensional preprocessing and enhancement on the collected raw drainage pipe image data. Then, it generates multi-scale channel attention fusion features that are sensitive to defects through a multi-scale multi-channel attention fusion module. Finally, it outputs defect classification and identification results and predicted segmentation heatmaps through a parallel identification and segmentation module that couples defect pattern perception and geometric constraints. Construct a model that conforms to the loss function to calculate the total loss of the model. The model is trained based on the total loss function to obtain a trained model.

[0009] S3. Input the new, unknown defect images of the drainage pipes into the trained model to perform automatic defect identification, output the defect classification and identification results and the predicted segmentation heatmap, then generate a clear defect region contour mask based on the predicted segmentation heatmap, and generate an identification report by combining the defect classification and identification results.

[0010] S1 is as follows:

[0011] First, data is collected using a professional closed-circuit television inspection system or a pipe periscope device. Data collection is carried out under various working conditions, including different drainage pipes, different operating scenarios, and different drainage pipe defects. Frames containing clear defect patterns are selected from the video stream as the original drainage pipe image data, and the corresponding pipe environmental parameters are recorded at the same time.

[0012] Then, the original drainage pipe images are labeled with data. The labeling work is completed by trained professionals using pixel-level fine labeling with the aid of auxiliary tools. The defect classification label is an image-level classification label that clearly specifies one or more typical defect categories present in the collected image data. The contour label of the defect area is a pixel-level segmentation mask that accurately outlines the contour of each defect area in the collected image data. The defect classification label includes four typical defects: crack, fracture, corrosion and deformation.

[0013] S2 is as follows:

[0014] S2.1 Input the original drainage pipe image data and perform multi-dimensional preprocessing and enhancement. Specifically, a preprocessing method based on three-dimensional adaptive wavelet transform and channel-aware illumination correction is adopted. By co-processing spatial multi-scale information and cross-channel illumination components, the image quality is comprehensively improved from both spatial and channel dimensions to generate an enhanced preprocessed image.

[0015] S2.2 Input enhancement preprocessed image, multi-scale channel attention fusion module, captures information under different receptive fields by multi-scale pooling of basic features, and uses attention mechanism based on mutual information metric to dynamically fuse multi-scale and multi-channel features to generate multi-scale channel attention fusion features sensitive to defects;

[0016] S2.3 Input multi-scale channel attention fusion features, adopt a parallel recognition and segmentation module that couples defect pattern perception and geometric constraints, and output defect classification and recognition results and predicted segmentation heatmaps simultaneously by fusing multi-scale semantics and spatial details, explicitly modeling the geometric structure of the pipe inner wall, and utilizing multi-scale contextual information.

[0017] S2.4 Construct a composite loss function that jointly optimizes classification and segmentation tasks while incorporating domain prior knowledge. Employ a joint optimization strategy of multi-task adaptive focusing loss and multi-scale feature consistency regularization. By dynamically adjusting task weights and focusing on difficult samples, while forcing the model to maintain consistent predictions at different scales, the total loss function of the model is obtained.

[0018] S2.5. Divide the data collected in S1 into training set, validation set and test set according to the proportion. Use the training set and validation set to train the model and update the parameters. Save the model parameters with the best performance on the validation set to obtain the trained and best-performing typical defect identification model for drainage pipes. Then validate it on the test set.

[0019] S2.1 is as follows:

[0020] S2.1.1 Based on the linear correlation between each channel of the original drainage pipe image and the local texture richness of each channel image at multiple scales, dynamically calculate the weight coefficients of wavelet domain feature fusion.

[0021] S2.1.2 Perform three-dimensional discrete wavelet transform on the original drainage pipe image to decompose the image at multiple scales and separate the spatial-spectral features of different frequency bands;

[0022] S2.1.3 Based on the global brightness statistics of each channel and the brightness dependence between channels, the intensity of each pixel in each channel is adaptively adjusted through the channel-aware correction function, thereby correcting overexposed and underexposed areas and obtaining the corrected image.

[0023] S2.1.4. The frequency domain features obtained by multi-scale three-dimensional wavelet transform are fused across scales and channels according to dynamic fusion weights, and then added to the weighted multi-channel illumination correction results to generate an enhanced preprocessed image.

[0024] S2.2 is as follows:

[0025] S2.2.1. The enhanced preprocessed image is input into a 3D convolutional layer to extract initial spatial-channel joint features. Specifically, trainable convolutional kernel parameters are used to perform sliding calculations on the enhanced preprocessed image to extract features, resulting in a 3D basic feature tensor with a size of [missing information]. , , , These represent the spatial height, spatial width, and number of channels of the basic feature tensor, respectively.

[0026] S2.2.2, Perform the following on the fundamental feature tensor in the spatial dimension: A set of three-dimensional max pooling operations at different scales, the pooling kernel size and stride are usually adjusted according to... Incrementing to generate a set of multi-scale 3D pooled feature maps;

[0027] S2.2.3 Calculate the mutual information between feature maps at each scale and the global average feature to measure the uniqueness and importance of the information contained in the features at each scale and in each channel, and generate the attention weight tensor through Softmax normalization;

[0028] S2.2.4. The feature maps at each scale are summed element-wise with their corresponding attention weights, and then a composite nonlinear transformation is introduced to enhance the fused features, generating multi-scale channel attention fusion features.

[0029] S2.3 is as follows:

[0030] S2.3.1. A feature aggregation method based on gating mechanism is adopted to fuse high-level semantic features with detailed preprocessed images. The supplementary intensity of spatial detail information is dynamically modulated by channel-aware affinity weights to generate pattern-aware features that are sensitive to both local texture of defects and global context.

[0031] S2.3.2 Construct a geometrically constrained deformable convolutional layer. This deformable convolutional layer predicts the spatial offset constrained by the prior constraints of the cylindrical surface model of the pipeline, so that the sampling points of the convolutional kernel undergo adaptive deformation to obtain the features after geometric deformation adaptation.

[0032] S2.3.3. By constructing a multi-scale context pyramid, the context information of defects at different perceptual scales is captured. The pyramid uses dilated convolutions with different dilation rates to process features in parallel, and multi-scale features are fused by adaptive weights based on channel statistics to obtain features enhanced by multi-scale context information.

[0033] S2.3.4. Construct a lightweight dual-branch parallel head structure based on the fused multi-scale context features. The dual branches include a classification branch and a segmentation branch. The classification branch aggregates the global context and passes it through a fully connected layer to output the predicted probability of each defect category. The segmentation branch generates a segmentation heatmap aligned with the spatial size of the input image through pointwise convolution and upsampling operations.

[0034] S2.4 is as follows:

[0035] S2.4.1 Multi-task adaptive focusing loss includes adaptive weighted classification focusing loss and segmentation focusing loss. The loss weights are adjusted according to the dynamic difficulty of the two tasks in the current batch. The focusing loss of the classification task is the degree of focus of the model on difficult samples in the defect classification task, and the focusing loss of the segmentation task is the degree of focus of the model on difficult-to-segment pixels in the pixel-level segmentation task.

[0036] S4.2.2 Multi-scale feature consistency regularization loss adopts an intermediate supervision-based consistency regularization strategy, which forces consistency between intermediate segmentation results predicted from features at different levels of the pyramid and aligns them with the final segmentation result.

[0037] S4.2.3 The total loss function is a weighted sum of the multi-task adaptive focusing loss and the multi-scale feature consistency regularization loss.

[0038] S2.5 is as follows:

[0039] The training process uses the training set as input and iterates around forward and back propagation. In each iteration, a batch of images is input into the model, which processes them to obtain the predicted defect classification probability and segmentation heatmap. Then, based on the total loss function, the combined error between the current prediction and the true label is calculated. Next, using the back propagation algorithm, the gradient of the total loss with respect to all trainable parameters of the model is calculated. An adaptive moment estimation optimizer is then used to update all trainable parameters of the model based on the calculated gradient.

[0040] During training, the model performance is evaluated periodically on an independent validation set; an iteration stopping condition is set for model training, and training ends only when the iteration stopping condition is met.

[0041] After training, the parameters of the model with the best performance on the validation set are saved as the final model for identifying typical defects in drainage pipes.

[0042] S3 is as follows:

[0043] The original drainage pipe image to be identified is input into the trained drainage pipe typical defect identification model, which outputs a typical defect prediction probability vector and a prediction segmentation heatmap. The category with the highest probability is selected as the final classification and identification result. A probability threshold is set, the prediction segmentation heatmap is binarized, and morphological operations are applied to smooth the boundaries and remove small noise, finally generating a clear defect region contour mask. Combined with the classification and identification results, a structured identification report is generated.

[0044] On the other hand, the present invention also provides a system for identifying typical defects in drainage pipes using machine vision technology, including a module for executing processing instructions for each step of a method for identifying typical defects in drainage pipes using machine vision technology, as follows:

[0045] Data acquisition and annotation module: The module uses a camera to capture video of the inside of the pipe, extracts the original drainage pipe image data frame by frame from the video stream, and annotates the images, including defect classification labels and defect area outlines.

[0046] Model building and training module: Construct a typical defect identification model for drainage pipes, perform multi-dimensional preprocessing and enhancement on the original images, generate sensitive fusion features through a multi-scale multi-channel attention fusion module, and output relevant results through a parallel recognition and segmentation module; construct a total loss function, divide the labeled data into training set, validation set and test set, train and save the optimal model;

[0047] Defect identification and report generation module: Input the pipeline image with unknown defects into the trained model, automatically identify and output the defect classification results and predicted segmentation heatmap, generate the defect region contour mask based on the heatmap, and generate a structured recognition report by combining the classification results.

[0048] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. The above technical solutions have the following advantages or beneficial effects:

[0049] This invention discloses a method and system for identifying typical defects in drainage pipes using machine vision technology. It employs a preprocessing method combining three-dimensional adaptive wavelet transform and channel-aware illumination correction. Through dynamic weight fusion and cross-channel brightness correction, it enhances local contrast while restoring high-frequency details, effectively solving the problem of poor adaptability of traditional global processing methods in complex pipe environments. A multi-scale channel attention fusion module is used, introducing an attention mechanism based on mutual information metric. This dynamically evaluates the uniqueness and importance of features at different scales and in different channels, overcoming the limitation of conventional single-scale convolution in missing low-contrast subtle defects. A parallel recognition and segmentation module coupling defect pattern perception and geometric constraints is constructed. Deformable convolution adaptively fits the curved surface structure of the pipe's inner wall, and a multi-scale context pyramid captures cross-scale features, achieving accurate localization and classification of complex morphological defects. A joint optimization strategy of multi-task adaptive focusing loss and multi-scale feature consistency regularization is adopted. By dynamically adjusting task weights to focus on difficult samples and forcing multi-scale prediction consistency, it effectively alleviates the problems of class imbalance and blurred boundary segmentation. Attached Figure Description

[0050] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0051] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0052] Figure 2 This is an example of a raw image of a drainage pipe.

[0053] Figure 3 This is a pixel-level segmentation mask obtained by annotating the original drainage pipe image.

[0054] Figure 4 This is the image after multi-channel illumination correction.

[0055] Figure 5 Figure (a) shows the multi-channel illumination correction tensor for the R, G, and B channels, as well as the enhanced preprocessed image. Figure (b) shows the multi-channel illumination correction image for the R channel, Figure (c) shows the multi-channel illumination correction image for the G channel, and Figure (d) shows the enhanced preprocessed image.

[0056] Figure 6 This is the predicted segmentation heatmap output by the model. Detailed Implementation

[0057] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0058] Example 1

[0059] like Figure 1 As shown, a method for identifying typical defects in drainage pipes using machine vision technology includes the following steps:

[0060] S1. Capture video of the internal environment of the pipeline, extract frame by frame from the continuous video stream to obtain the original drainage pipeline image data, and annotate the acquired image data, marking defect classification labels and the outline of defect areas.

[0061] S2. Construct a typical defect identification model for drainage pipes. The model first performs multi-dimensional preprocessing and enhancement on the collected raw drainage pipe image data. Then, it generates multi-scale channel attention fusion features that are sensitive to defects through a multi-scale multi-channel attention fusion module. Finally, it outputs defect classification and identification results and predicted segmentation heatmaps through a parallel identification and segmentation module that couples defect pattern perception and geometric constraints. Construct a model that conforms to the loss function to calculate the total loss of the model. The model is trained based on the total loss function to obtain a trained model.

[0062] S3. Input the new, unknown defect images of the drainage pipes into the trained model to perform automatic defect identification, output the defect classification and identification results and the predicted segmentation heatmap, then generate a clear defect region contour mask based on the predicted segmentation heatmap, and generate an identification report by combining the defect classification and identification results.

[0063] In a specific implementation, S1 is as follows:

[0064] We acquire original drainage pipe images that comprehensively reflect the complex environment inside the actual pipes and accurately annotate them to provide a foundation for subsequent model training and evaluation.

[0065] Data acquisition is conducted using professional closed-circuit television (CCTV) inspection systems or pipe periscope equipment under various operating conditions, including drainage pipes of different diameters, materials, and varying degrees of fullness. This covers diverse scenarios such as daytime and nighttime operations, the presence or absence of water flow, and varying degrees of contamination, ensuring that the acquired images reflect real-world challenges such as motion blur, uneven lighting, moisture interference, and contamination obstruction. The acquired continuous video stream is extracted frame by frame, and frames containing clear defect patterns are selected as the original drainage pipe image data, while corresponding pipe environmental parameters are recorded. For example... Figure 2 The image shown is the original image of the drainage pipe.

[0066] The original drainage pipe images were annotated using a pixel-level fine-grained annotation method, performed by trained professionals with the aid of auxiliary tools. The annotation output contained two types of information: first, image-level classification labels, explicitly specifying one or more typical defect categories present in the image; second, pixel-level segmentation masks, precisely outlining the contours of each defect region in the image, such as... Figure 3 As shown, based on Figure 2 The pixel-level segmentation mask of the original drainage pipe image, with the white area representing the eroded area.

[0067] In the specific implementation process, the typical defect categories that are focused on include four types: cracks, fractures, corrosion, and deformation. Finally, the labeled dataset is divided into training set, validation set, and test set according to a preset ratio. The training set is used for learning model parameters, the validation set is used for hyperparameter tuning and monitoring the training process, and the test set is used for final evaluation of model performance.

[0068] In a specific implementation, S2 is as follows:

[0069] S2.1 Multi-dimensional preprocessing and enhancement of drainage pipe image data

[0070] Due to the dim lighting and obstruction caused by sewage and debris inside drainage pipes, the acquired images often suffer from motion blur, uneven illumination, and sensor noise. This results in low contrast and blurred details in defect features. Conventional global histogram equalization or fixed kernel filtering methods cannot effectively restore high-frequency details while enhancing local contrast, and they also ignore the spectral correlation between color image channels. Therefore, this invention employs a preprocessing method based on three-dimensional adaptive wavelet transform and channel-aware illumination correction. By co-processing spatial multi-scale information and cross-channel illumination components, it comprehensively improves image quality from both spatial and channel dimensions. The specific steps are as follows:

[0071] S2.1.1 Calculation of Dynamic Fusion Weights Between Channels

[0072] Based on the linear correlation between channels of the original drainage pipe image and the local texture richness of each channel image at multiple scales, the weight coefficients for wavelet domain feature fusion are dynamically calculated. These coefficients can adaptively enhance information-rich channels and suppress noisy channels, and are expressed as follows:

[0073]

[0074] In the formula, Indicates the first Scale from source channel Merge into target channel The dynamic weighting coefficients are used to guide the contribution ratio of different source channels during multi-scale feature fusion. The larger the weight value, the more significant the contribution of the source channels. For the target channel The more significant the contribution at this scale; This represents the scale index of the wavelet decomposition, with values ​​ranging from 1 to 2. ; This represents the total decomposition scale number, a hyperparameter, with an example value of 3. Indicates the target channel index; for RGB images, These correspond to the red, green, and blue channels, respectively. Indicates the source channel index. ; Image of the original drainage pipe In the A two-dimensional pixel matrix on the channel, with a size of , Image of the original drainage pipe In the A two-dimensional pixel matrix on the channel; Indicates the first Scale for target channel The adjustment factor is a preset hyperparameter used to control the sensitivity of the weights to the correlation and variance product. The example value is 0.5. This represents the Pearson correlation coefficient calculation function, used to quantify the degree of linear correlation between two channel image matrices, with output values ​​ranging from... The closer the absolute value is to 1, the stronger the correlation. Indicates the first A function for calculating the local texture variance of an image at different scales. This function first processes the input image at a scale corresponding to the scale. The wavelet decomposition is performed, and then the local variance is calculated on its high-frequency subband to estimate the richness of texture details of the image at this scale. The larger the variance, the richer the texture. This represents the total number of image channels. For a standard RGB image, ; This represents the natural exponential function.

[0075] S2.1.2 Multi-scale three-dimensional wavelet transform

[0076] A three-dimensional discrete wavelet transform is performed on the original drainage pipe image to decompose the image at multiple scales, separating the spatial-spectral features of different frequency bands. This provides a multi-resolution analysis basis for detail enhancement, specifically targeting the original drainage pipe image. In the Two-dimensional pixel matrix on the channel By adopting the first A three-dimensional discrete wavelet transform operator is used to achieve multi-scale frequency domain decomposition. Specifically, a set of preset three-dimensional wavelet filter banks is used to convolve and downsample the input channel image to obtain the first... Scale of source channel The frequency domain feature tensor obtained by performing a three-dimensional discrete wavelet transform on the image It includes approximate subbands at this scale and high-frequency detail subbands in multiple directions.

[0077] S2.1.3, Multi-channel illumination correction

[0078] To address the issue of non-uniform illumination within the pipeline, based on the global brightness statistics of each channel and the brightness dependency between channels, a channel-aware correction function adaptively adjusts the intensity of each pixel in each channel to correct overexposed and underexposed areas, as shown below:

[0079]

[0080] In the formula, Represents the multi-channel illumination correction tensor In spatial location and channels The pixel value at a given location represents the reflectance intensity that the channel should have at that location after correcting for the effects of non-uniform lighting. This intensity is closer to what was observed under uniform lighting conditions, and helps to recover details in overexposed or underexposed areas. The image after multi-channel illumination correction is shown below. Figure 4 As shown; Indicates the horizontal index of a pixel in the image; Indicates the vertical index of a pixel in the image; Indicates channel The correction intensity weighting coefficient, calculated using a sigmoid function, results in stronger correction enhancement for generally darker channels, expressed as: ; Represents the natural exponential function; Indicates channel The brightness sensitivity coefficient is a preset hyperparameter that controls the influence of the channel's global average brightness on the weighting. The magnitude of the impact, with an example value of 0.1; Image of the original drainage pipe In the passage The global average value of all pixels represents the overall brightness level of the channel; Indicates the location of the original drainage pipe image. and channels Pixel value at; Indicates channel For the channel Illumination dependence coefficient, quantization channel Local brightness changes on the channel The influence weights of the correction amount are trainable parameters; Image of the original drainage pipe In the passage The global average value of all pixels represents the overall brightness level of the channel; Image of the original drainage pipe In the passage The global pixel variance is used to normalize the brightness deviation.

[0081] It should be noted that, This term evaluates the deviation of the current pixel's brightness value in each channel from the global average value of that channel, and calculates a value based on the combined deviation of all channels. The attenuation coefficient within the interval is calculated by the weighted sum of the normalized deviations of the pixel value in each channel. This can measure the overall degree of illumination anomaly experienced by the pixel, and the natural exponential function is used to map this degree of anomaly into the attenuation coefficient, thereby achieving local adaptive illumination correction.

[0082] S2.1.4, Fusing and Generating Preprocessed Images

[0083] The frequency domain features obtained from multi-scale 3D wavelet transform are fused across scales and channels according to dynamic fusion weights, and then added to the weighted multi-channel illumination correction results to generate an enhanced preprocessed image, represented as:

[0084]

[0085] In the formula, This indicates an enhanced preprocessed image, with a size of [size missing]. It is used to fuse and enhance key information recovered and enhanced from the original drainage pipe images; This represents a dynamic weight vector, whose target channel... The corresponding element value is By broadcasting to the frequency domain feature tensor After considering the spatial dimension, the frequency domain feature tensor The corresponding channels are weighted, and then the results of all scales and source channels are summed. This represents element-wise multiplication; This represents a three-dimensional weight tensor, where the channels at each spatial location are... The corresponding element value is It is used to perform channel-level intensity modulation of the multi-channel illumination correction tensor.

[0086] In the specific implementation process, the multi-channel illumination correction tensor of the R, G, and B channels, as well as the enhanced preprocessed image, are used as follows: Figure 5 As shown.

[0087] S2.2, Multi-scale Channel Attention Fusion Module

[0088] Because defects such as minute cracks and pitting corrosion exhibit significant differences across different color channels and spatial scales, conventional single-scale convolution or simple channel attention mechanisms struggle to model these complex cross-scale and cross-channel dependencies, easily leading to missed detections of low-contrast defects. Therefore, this invention employs a multi-scale channel attention fusion module. This module performs multi-scale pooling on basic features to capture information from different receptive fields and utilizes an attention mechanism based on mutual information metrics to dynamically fuse multi-scale and multi-channel features, generating defect-sensitive multi-scale channel attention fusion features. The specific steps are as follows:

[0089] S2.2.1, Extraction of Three-Dimensional Basic Features

[0090] Enhance preprocessed images The input 3D convolutional layer extracts initial spatial-channel joint features, specifically using trainable convolutional kernel parameters to enhance the preprocessed image. Perform sliding computation to extract features and obtain the three-dimensional basic feature tensor. The size is It possesses primary spatial patterns and channel features extracted from preprocessed images;

[0091] in, Represents the fundamental feature tensor The spatial height is smaller than the original drainage pipe image. height ; Represents the fundamental feature tensor The space width is smaller than the original drainage pipe image. width ; Represents the fundamental feature tensor The number of channels is a hyperparameter, with an example value of 64.

[0092] S2.2.2 Generation of Multi-Scale 3D Feature Maps

[0093] Perform on the fundamental feature tensor in the spatial dimension A set of three-dimensional max pooling operations at different scales, the pooling kernel size and stride are usually adjusted according to... Incrementing to generate a set of multi-scale 3D pooled feature maps to simulate the manifestation of defects in different spatial ranges;

[0094] Specifically, definition Indicates the first A three-dimensional pooling feature map of various scales, whose spatial size varies with... Increasing and decreasing, used to capture features under different receptive fields, is the first in the set of 3D pooled feature maps generated by 3D max pooling operation. One element; Indicates the pooling scalar index. ; The total number of scales is a preset hyperparameter, with an example value of 4.

[0095] S2.2.3 Multi-scale channel attention weight calculation

[0096] The mutual information between feature maps at each scale and the global average feature is calculated to measure the uniqueness and importance of the information contained in features at each scale and in each channel. An attention weight tensor is then generated through Softmax normalization, represented as follows:

[0097]

[0098] In the formula, Indicates the first The 3D attention weight tensor corresponding to each scale feature map has a size that is similar to... Similarly, positions with larger weight values ​​indicate that the feature information at that location is more unique and more important; This represents the mutual information estimation function, used to quantify the statistical dependency between two feature tensors. Relative to the global average feature tensor The greater the information increment, the higher the attention weight. Represents the global average feature tensor, obtained by applying all... The average of pooled feature maps at each scale is used as a global reference benchmark for information measurement. The calculation method is expressed as follows: ; Indicates the first The temperature coefficient at each scale is a trainable parameter used to adjust the smoothness of the attention weight distribution; the larger the value, the more uniform the weight distribution. This represents the Softmax function, which is used to normalize input values ​​into a probability distribution.

[0099] S2.2.4 Weighted Fusion and Nonlinear Enhancement

[0100] Adaptive fusion is achieved by element-wise weighted summation of feature maps at each scale and their corresponding attention weights. Subsequently, a composite nonlinear transformation is introduced to enhance the fused features and improve their representational capabilities.

[0101]

[0102] In the formula, This represents a multi-scale channel attention fusion feature, where the number of channels is... It integrates features from different spatial receptive fields, and the contribution at each scale is dynamically weighted by an attention mechanism based on mutual information metric; This represents the hyperbolic tangent activation function, which compresses the input values ​​to a range. Introducing saturated nonlinearity; It represents a sine function, provides a periodic nonlinear transformation, and can be combined with a hyperbolic tangent function to enhance the ability to represent complex features.

[0103] S2.3, Parallel Recognition and Segmentation Module for Coupled Defect Pattern Perception and Geometric Constraints

[0104] In the task of accurately identifying typical defects in drainage pipes, different defect types have diverse and complex spatial morphologies, requiring the model to have effective perception capabilities of structured patterns. Pixel-level accurate segmentation of the defect region is crucial for assessing the severity of the defect, but the complex background inside the pipe makes segmentation prone to boundary blurring. Therefore, this invention employs a parallel identification and segmentation module that couples defect pattern perception with geometric constraints. By fusing multi-scale semantics and spatial details, explicitly modeling the geometric structure of the pipe's inner wall, and utilizing multi-scale contextual information, it simultaneously outputs defect classification results and high-precision segmentation heatmaps. The specific steps are as follows:

[0105] S2.3.1 Feature Aggregation for Defect Pattern Awareness

[0106] To integrate high-level semantic features with detailed preprocessed images, a gating-based feature aggregation method is employed. This method dynamically modulates the supplementary intensity of spatial detail information through channel-aware affinity weights, generating pattern-aware features sensitive to both local texture of defects and global context, represented as follows:

[0107]

[0108] In the formula, The representation is a pattern-aware feature tensor with size . This characterizes the defective features of integrating global semantic understanding with local spatial details; The 3D projection weight tensor represents trainable parameters used to map the stitched high-dimensional features to the target number of channels, with a size of [value missing]. ; Indicates enhanced preprocessed image The number of channels adjusted by the lightweight projection function is a hyperparameter, with an example value of 16. The target number of channels for the pattern-aware feature tensor is a hyperparameter, with an example value of 128. This indicates a splicing operation along the channel dimension; Represents a lightweight projection function, consisting of a set of The convolution kernel implementation is used to enhance preprocessed images. Spatial dimensions downsampled to and its channel number from Adjusted to ; This represents the hyperbolic tangent activation function, which constrains eigenvalues ​​to the interval [0, 1]. To enhance nonlinear expressive power; This represents the element-wise multiplication operator; This represents the Sigmoid activation function, which maps the input to a range. , as a gating weight; The gated weight generation function consists of a global average pooling layer and two fully connected layers, fusing features based on the multi-scale channel attention of the input. The global context information is used to calculate the gated scalar value for each channel, and the output size is [size missing]. .

[0109] S2.3.2 Modeling of Deformable Space with Geometric Constraints

[0110] To adapt to the curved geometry of the pipe's inner wall and the potential directional morphology of defects, a geometrically constrained deformable convolutional layer is constructed. This layer predicts the spatial offset constrained by the pipe's cylindrical surface model, enabling the convolutional kernel sampling points to adaptively deform and better fit the defect region. This is represented as:

[0111]

[0112] In the formula, Indicates the first Deformable convolutional feature maps In spatial location and channels The pixel value at that location represents the features after geometric deformation adaptation, which can more accurately align the irregular shapes of the pipe's inner wall surface and defects. Represented in the pattern-aware feature tensor In the middle, according to the first A deformable convolutional kernel at position Predicted offset Adjusted sampling position Above, the first The characteristic values ​​of each channel; The index represents the deformable convolution kernel. ; This represents the total number of deformable convolution kernels; it is a hyperparameter with an example value of 8. The channel index represents the deformable convolutional feature map. ; This represents the total number of channels in the output of the deformable convolutional layer. It is a hyperparameter with an example value of 128. This represents the radius of the deformable convolution kernel, and the size of the convolution kernel is... , This is a hyperparameter, with an example value of 2; This indicates that in a standard convolution kernel, relative to the center point... Offset index in the direction; This indicates that in a standard convolution kernel, relative to the center point... Offset index in the direction; Representational pattern-aware feature tensor Channel index; Indicates the first Each convolutional kernel is located at... Predicted Directional offset; Indicates the first Each convolutional kernel is located at... Predicted Directional offset; Indicates the first A deformable convolutional kernel at position At this location, connect the input channel. With output channel The weight parameters are trainable parameters.

[0113] In the specific implementation process, and From pattern-aware feature tensors through additional convolutional layers The feature tensor obtained from regression is a pattern-aware feature tensor. As input, output a dimension of The feature map is then used to parse the corresponding features for each position, each convolutional kernel, and each sampling point. This allows the sampling grid of the convolution kernel to adaptively deform according to the local content of the input features, thereby better fitting irregular, directional defect shapes or adapting to the curved surface geometry of the pipe wall, improving the flexibility and accuracy of feature extraction.

[0114] All Deformable convolutional feature maps are concatenated along the channel dimension to form a geometric modeling feature tensor. The size is To simplify the subsequent expression, we will still denote its channel number as . .

[0115] S2.3.3, Multi-scale Context Pyramid and Adaptive Fusion

[0116] To capture the contextual information of defects at different perceptual scales and improve the model's robustness to scale changes, a multi-scale context pyramid is constructed. This pyramid uses dilated convolutions with different dilation rates to process features in parallel, and fuses multi-scale features through adaptive weights based on channel statistics, as shown below:

[0117]

[0118] In the formula, Represents the pyramid fusion feature tensor, with size . The features are enhanced with multi-scale contextual information, enabling the model to simultaneously perceive the local details of the defect and its contextual relationship in a larger range, thereby improving the model's ability to identify defects of different sizes and its robustness to complex backgrounds. This represents the total number of layers in the pyramid, i.e., the number of dilated convolution types used. It is a hyperparameter with an example value of 4. Indicates the index of the pyramid level. ; Indicates the expansion rate The dilation rate of the dilated convolution operation. Set as Incremental, for example, available ; Indicates assignment to the first The 3D channel attention weight tensor of the feature map output by the layer dilated convolution has a size of [size missing]. It performs channel-by-channel multiplication with the corresponding feature map through a broadcast mechanism to adaptively fuse information at different scales based on global statistics. The calculation method is expressed as follows: ; The variance vector is a global statistical descriptive vector composed of the average channel variances of the feature maps at each layer, measuring the variance of the first-order feature map. The overall information richness or activation intensity of the layer feature map represents the information richness of features at each scale. ; Indicates the first The average variance of all channels in the output feature map of the layer dilated convolution is used to measure the overall information richness of the features of that layer. The calculation method is expressed as follows: , This represents the index of the feature map in the spatial height direction; Indicates the first Layer dilated convolution output feature map In the Each channel and spatial location Eigenvalues ​​at; Indicates the first The dilated convolution output feature map at the 1st layer The mean of all spatial locations on each channel; Indicates that for the first The multilayer perceptron learns to generate channel attention weights for fusing features at different scales, enabling the fusion process to adaptively emphasize information-rich scales and suppress scales with redundant information or more noise. This represents the temperature coefficient, which is a trainable parameter used to adjust the smoothness of the weight distribution. Indicates along the dimension Perform Softmax normalization to ensure .

[0119] S2.3.4, Dual-branch parallel recognition and segmentation head

[0120] A lightweight dual-branch parallel head structure is constructed based on the fused multi-scale contextual features to achieve classification of typical defect categories and pixel-level segmentation of defect regions, respectively.

[0121] Classification branch: The classification branch aggregates the global context and passes it through a fully connected layer, outputting the predicted probability for each defect category, represented as:

[0122]

[0123] In the formula, This represents the probability vector for predicting typical defects. ; This represents the total number of defect categories. In one embodiment, it includes four typical defects: cracks, fractures, corrosion, and deformation. The weight matrix of the fully connected classification layer is a trainable parameter of size . ; The bias vector of the fully connected classification layer is a trainable parameter of size . ; This represents the global average pooling operation, which converts the feature tensor... Spatial dimension pooling The output size is ; This represents the Softmax function, which normalizes the input vector into a probability distribution.

[0124] Segmentation Branch: The segmentation branch generates a segmentation heatmap aligned with the spatial dimensions of the input image through pointwise convolution and upsampling operations, represented as:

[0125]

[0126] In the formula, This represents the predicted segmentation heatmap, with a size of [size missing]. Each pixel value represents the probability that the location belongs to a defect area. The input to the model is... Figure 2 The output predicted segmentation heatmap is as follows: Figure 6 As shown; This indicates a bilinear interpolation upsampling operation, which reduces the spatial size of the feature map from... Upsampled to the original drainage pipe image size ; This represents the convolution operator; express The kernel weights of a convolutional layer are trainable parameters used to... The feature mapping of the channel is a single channel with a size of ; express The bias of the convolutional layer is a trainable parameter with a size of [value missing]. .

[0127] S2.4 Loss Function Calculation

[0128] Conventional loss functions often lead to models being insensitive to minor defects or having coarse boundary segmentation when faced with severe class imbalance and the coexistence of multi-scale defects. To efficiently train a drainage pipe defect identification model and ensure its robustness and accuracy in complex scenarios, a composite loss function that can jointly optimize classification and segmentation tasks while incorporating domain prior knowledge is needed. This invention employs a joint optimization strategy of multi-task adaptive focusing loss and multi-scale feature consistency regularization. By dynamically adjusting task weights and focusing on difficult samples, while simultaneously forcing the model to maintain consistent predictions across different scales, the model's performance and stability are improved. The specific steps are as follows:

[0129] S2.4.1 Adaptive Multi-Task Focusing Loss

[0130] The main loss function consists of an adaptively weighted classification-focusing loss and a segmentation-focusing loss. Its weights are adjusted based on the dynamic difficulty of the two tasks within the current batch, and are expressed as follows:

[0131]

[0132] In the formula, The main loss function represents the comprehensive performance of the model on the two core tasks of classification and segmentation, and achieves a balanced optimization of the two tasks through a dynamic weighting mechanism. Indicates the training iteration number. The dynamic weights of the time-of-step classification loss have values ​​in the interval [missing information]. Within this range, the difficulty is dynamically determined by the task difficulty of the current batch, and the calculation method is expressed as follows: This represents the scaling factor, a preset hyperparameter used to control the sensitivity of the weights to the difference in task difficulty. An example value is 2. This represents the average relative error of the classification task within the current batch, used to quantify the difficulty of the classification task. This represents the average absolute error of the segmentation task within the current batch, used to quantify the difficulty of the segmentation task. This represents the focusing loss for the classification task, characterizing the model's focus on difficult examples in defect classification. It aims to improve the model's ability to distinguish easily confused defect categories. The calculation method is expressed as follows: ; This represents the focusing loss for the segmentation task, characterizing the model's focus on hard-to-segment pixels in pixel-level segmentation. It aims to improve the accuracy of segmentation boundaries and the detection rate of small defects. The calculation method is expressed as follows: . Indicates the training batch size; The model represents the first element in the batch. The predicted value for the nth sample is... The probability of a class of defects is predicted from the typical defect prediction probability vector. get; This represents the focusing parameter for the classification task, used to reduce the loss contribution of easily classified samples. An example value is 2. Indicates the first The sample at the th The true classification label for class defects is in one-hot encoded form; This represents a logarithmic function, with the default base being the natural constant. The model represents the first The spatial location of each sample The predicted defect probability value is derived from the predicted segmentation heatmap. get; Indicates the first Each sample is located at The actual segmentation label at the location, with a value of 0 or 1; This represents the weighting coefficient used to balance the loss contribution of foreground defective pixels and background pixels. It can be set according to the proportion of foreground pixels in the dataset, with an example value of 0.25. This represents the focus parameter for the segmentation task, used to adjust the loss contribution of easily classifiable pixels. An example value is 2.

[0133] S2.4.2 Multi-scale feature consistency regularization loss

[0134] To leverage multi-scale contextual pyramid features and enhance the model's robustness to variations in defect scale, a consistency regularization strategy based on intermediate supervision is adopted. This strategy enforces consistency among intermediate segmentation results predicted from features at different pyramid levels and aligns them with the final segmentation result, as follows:

[0135]

[0136] In the formula, The auxiliary regularization loss represents the consistency of multi-scale feature representation within the model and the effectiveness of intermediate supervision. Indicates the constraint number Layer and First The coupling coefficient for predictive consistency in the middle of the layer is a trainable parameter; This represents the Dice similarity coefficient loss function, which measures the overlap between two intermediate segmentation heatmaps. The closer the value is to 1, the higher the consistency. Indicates from the first Layered pyramid features through additional The intermediate segmentation heatmap generated by the convolutional layer and the sigmoid function has a size of [size missing]. , Indicates from the first Layered pyramid features through additional Intermediate segmentation heatmaps generated by convolutional layers and the Sigmoid function; This represents the regularization weight hyperparameter, which is a regularization weight hyperparameter that balances the consistency loss and alignment loss terms. An example value is 0.5. This represents the binary cross-entropy loss function, used to align each intermediate prediction with the final segmentation output at the pixel level.

[0137] S2.4.3, Total Loss Function

[0138] Total loss function The weighted sum of the main loss function and the auxiliary regularization loss, i.e. ,in, It is a hyperparameter that controls the strength of regularization and is used to adjust the auxiliary loss. The contribution to the total loss, with an example value of 0.1.

[0139] S2.5, Update of Typical Defect Identification Model for Drainage Pipelines

[0140] The training process takes the training set prepared in step S1 as input and proceeds in a loop with forward propagation and backward propagation as the core. In each iteration, a batch of enhanced preprocessed images and their corresponding real classification labels and segmentation masks are input into the model. The model sequentially performs parallel recognition and segmentation steps of multi-dimensional preprocessing and enhancement, multi-scale channel attention fusion, and coupled defect pattern perception and geometric constraints to obtain the predicted defect classification probability and segmentation heatmap.

[0141] Then, based on the total loss function, the combined error between the current prediction and the true label is calculated. This total loss function integrates adaptive multi-task focusing loss and multi-scale feature consistency regularization loss, aiming to simultaneously optimize classification accuracy, segmentation precision and enhance the internal consistency of the model.

[0142] Then, using the backpropagation algorithm, the gradient of the total loss with respect to all trainable parameters of the model is calculated. An adaptive moment estimation optimizer is then used to update all trainable parameters of the model based on the calculated gradient, so that the model can produce more accurate predictions in the next iteration.

[0143] During training, model performance is evaluated periodically on independent validation sets, and changes in key metrics such as classification accuracy and segmentation intersection-union ratio are monitored.

[0144] The training iteration stopping conditions of the model are set based on the performance of the validation set, mainly including two situations: one is reaching the preset maximum number of training iterations; the other is that the comprehensive performance index on the validation set no longer improves within several consecutive evaluation periods, indicating that the model has fully learned the patterns in the training data and may begin to overfit. In this case, training is terminated in advance to avoid performance degradation.

[0145] After training, the parameters of the model with the best performance on the validation set are saved as the final model for identifying typical defects in drainage pipes.

[0146] In a specific implementation, S3 is as follows:

[0147] Once the typical defect identification model for drainage pipes is trained, it can be used to automatically identify defects in new and unknown drainage pipe images.

[0148] In the practical application stage, the original drainage pipe image to be identified is first subjected to the same preprocessing operation as in the training stage, that is, a preprocessing method based on three-dimensional adaptive wavelet transform and channel-aware illumination correction is adopted to generate a high-quality enhanced preprocessed image.

[0149] Then, this enhanced preprocessed image is input into the trained recognition model with the optimal parameters loaded. The model sequentially passes through a multi-scale channel attention fusion module to extract features sensitive to defects, and then performs in-depth analysis through a parallel recognition and segmentation module that couples defect pattern perception with geometric constraints.

[0150] The model ultimately outputs two results simultaneously: one is a typical defect prediction probability vector, which contains the probability that the image is judged as various defects such as cracks, fractures, corrosion, and deformation. Specifically, the category with the highest probability is taken as the final classification and recognition result; the other is a prediction segmentation heatmap, which is aligned with the spatial size of the input image and clearly shows the location and range of possible defect areas in the image in the form of pixel probability values.

[0151] To further improve the readability and usability of the results, the segmentation heatmap is usually post-processed, such as by setting a probability threshold to binarize it, and applying morphological operations to smooth the boundaries and remove small noise, ultimately generating a clear defect region contour mask.

[0152] By combining classification results with segmentation masks, the system can generate structured recognition reports that clearly indicate the type, quantity, location, and pixel-level precise contours of defects in the image, providing quantitative and intuitive technical basis for the condition assessment and maintenance decisions of drainage pipelines.

[0153] Example 2

[0154] A system for identifying typical defects in drainage pipes using machine vision technology includes a module for executing processing instructions for each step of a method for identifying typical defects in drainage pipes using machine vision technology, as follows:

[0155] Data acquisition and annotation module: The module uses a camera to capture video of the inside of the pipe, extracts the original drainage pipe image data frame by frame from the video stream, and annotates the images, including defect classification labels and defect area outlines.

[0156] Model building and training module: Construct a typical defect identification model for drainage pipes, perform multi-dimensional preprocessing and enhancement on the original images, generate sensitive fusion features through a multi-scale multi-channel attention fusion module, and output relevant results through a parallel recognition and segmentation module; construct a total loss function, divide the labeled data into training set, validation set and test set, train and save the optimal model;

[0157] Defect identification and report generation module: Input the pipeline image with unknown defects into the trained model, automatically identify and output the defect classification results and predicted segmentation heatmap, generate the defect region contour mask based on the heatmap, and generate a structured recognition report by combining the classification results.

[0158] Although the specific embodiments of the invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the invention. Based on the technical solutions of the invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the invention.

Claims

1. A method for identifying typical defects in drainage pipes using machine vision technology, characterized in that, Includes the following steps: S1. Capture video of the internal environment of the pipeline, extract frame by frame from the continuous video stream to obtain the original drainage pipeline image data, and annotate the acquired image data, marking defect classification labels and the outline of defect areas. S2. Construct a typical defect identification model for drainage pipes. The model first performs multi-dimensional preprocessing and enhancement on the collected raw drainage pipe image data. Then, it generates multi-scale channel attention fusion features that are sensitive to defects through a multi-scale multi-channel attention fusion module. Finally, it outputs defect classification and identification results and predicted segmentation heatmaps through a parallel identification and segmentation module that couples defect pattern perception and geometric constraints. Construct a model that conforms to the loss function to calculate the total loss of the model. The model is trained based on the total loss function to obtain a trained model. S2 is as follows: S2.1 Input the original drainage pipe image data and perform multi-dimensional preprocessing and enhancement. Specifically, a preprocessing method based on three-dimensional adaptive wavelet transform and channel-aware illumination correction is adopted. By co-processing spatial multi-scale information and cross-channel illumination components, the image quality is comprehensively improved from both spatial and channel dimensions to generate an enhanced preprocessed image. S2.2 Input enhancement preprocessed image, multi-scale channel attention fusion module, captures information under different receptive fields by multi-scale pooling of basic features, and uses attention mechanism based on mutual information metric to dynamically fuse multi-scale and multi-channel features to generate multi-scale channel attention fusion features sensitive to defects; S2.3 Input multi-scale channel attention fusion features, adopt a parallel recognition and segmentation module that couples defect pattern perception and geometric constraints, and output defect classification and recognition results and predicted segmentation heatmaps simultaneously by fusing multi-scale semantics and spatial details, explicitly modeling the geometric structure of the pipe inner wall, and utilizing multi-scale contextual information. S2.4 Construct a composite loss function that jointly optimizes classification and segmentation tasks while incorporating domain prior knowledge. Employ a joint optimization strategy of multi-task adaptive focusing loss and multi-scale feature consistency regularization to obtain the model's total loss function. S2.

5. Divide the data collected in S1 into training set, validation set and test set according to the proportion. Use the training set and validation set to train the model and update the parameters. Save the model parameters with the best performance on the validation set to obtain the trained and best-performing typical defect identification model of drainage pipes. Then validate it on the test set. S3. Input the new, unknown defect images of the drainage pipes into the trained model to perform automatic defect identification, output the defect classification and identification results and the predicted segmentation heatmap, then generate a clear defect region contour mask based on the predicted segmentation heatmap, and generate an identification report by combining the defect classification and identification results.

2. The method for identifying typical defects in drainage pipes using machine vision technology according to claim 1, characterized in that, S2.1 is as follows: S2.1.1 Based on the linear correlation between each channel of the original drainage pipe image and the local texture richness of each channel image at multiple scales, dynamically calculate the weight coefficients of wavelet domain feature fusion. S2.1.2 Perform three-dimensional discrete wavelet transform on the original drainage pipe image to decompose the image at multiple scales and separate the spatial-spectral features of different frequency bands; S2.1.3 Based on the global brightness statistics of each channel and the brightness dependence between channels, the intensity of each pixel in each channel is adaptively adjusted through the channel-aware correction function, thereby correcting overexposed and underexposed areas and obtaining the corrected image. S2.1.

4. The frequency domain features obtained by multi-scale three-dimensional wavelet transform are fused across scales and channels according to dynamic fusion weights, and then added to the weighted multi-channel illumination correction results to generate an enhanced preprocessed image.

3. The method for identifying typical defects in drainage pipes using machine vision technology according to claim 1, characterized in that, S2.2 is as follows: S2.2.

1. The enhanced preprocessed image is input into a 3D convolutional layer to extract initial spatial-channel joint features. Specifically, trainable convolutional kernel parameters are used to perform sliding calculations on the enhanced preprocessed image to extract features, resulting in a 3D basic feature tensor with a size of [missing information]. , , , These represent the spatial height, spatial width, and number of channels of the basic feature tensor, respectively. S2.2.2, Perform the following on the fundamental feature tensor in the spatial dimension: A set of three-dimensional max pooling operations at different scales, with the pooling kernel size and stride varying. Incrementing to generate a set of multi-scale 3D pooled feature maps; S2.2.3 Calculate the mutual information between feature maps at each scale and the global average feature to measure the uniqueness and importance of the information contained in the features at each scale and in each channel, and generate the attention weight tensor through Softmax normalization; S2.2.

4. The feature maps at each scale are summed element-wise with their corresponding attention weights, and then a composite nonlinear transformation is introduced to enhance the fused features, generating multi-scale channel attention fusion features.

4. The method for identifying typical defects in drainage pipes using machine vision technology according to claim 1, characterized in that, S2.3 is as follows: S2.3.

1. A feature aggregation method based on gating mechanism is adopted to fuse high-level semantic features with detailed preprocessed images. The supplementary intensity of spatial detail information is dynamically modulated by channel-aware affinity weights to generate pattern-aware features that are sensitive to both local texture of defects and global context. S2.3.2 Construct a geometrically constrained deformable convolutional layer. This deformable convolutional layer predicts the spatial offset constrained by the prior constraints of the cylindrical surface model of the pipeline, so that the sampling points of the convolutional kernel undergo adaptive deformation to obtain the features after geometric deformation adaptation. S2.3.

3. By constructing a multi-scale context pyramid, the context information of defects at different perceptual scales is captured. The pyramid uses dilated convolutions with different dilation rates to process features in parallel, and multi-scale features are fused by adaptive weights based on channel statistics to obtain features enhanced by multi-scale context information. S2.3.

4. Construct a lightweight dual-branch parallel head structure based on the fused multi-scale contextual features. The dual branches include a classification branch and a segmentation branch. The classification branch aggregates the global context and passes it through a fully connected layer to output the predicted probability for each defect category; The segmentation branch generates a segmentation heatmap aligned with the spatial dimensions of the input image through pointwise convolution and upsampling operations.

5. The method for identifying typical defects in drainage pipes using machine vision technology according to claim 1, characterized in that, S2.4 is as follows: S2.4.1 Multi-task adaptive focusing loss includes adaptive weighted classification focusing loss and segmentation focusing loss. The loss weights are adjusted according to the dynamic difficulty of the two tasks in the current batch. The focusing loss of the classification task is the degree of focus of the model on difficult samples in the defect classification task, and the focusing loss of the segmentation task is the degree of focus of the model on difficult-to-segment pixels in the pixel-level segmentation task. S4.2.2 Multi-scale feature consistency regularization loss adopts an intermediate supervision-based consistency regularization strategy, which forces consistency between intermediate segmentation results predicted from features at different levels of the pyramid and aligns them with the final segmentation result. S4.2.3 The total loss function is a weighted sum of the multi-task adaptive focusing loss and the multi-scale feature consistency regularization loss.

6. The method for identifying typical defects in drainage pipes using machine vision technology according to claim 1, characterized in that, S2.5 is as follows: The training process takes the training set as input and proceeds in a loop with forward propagation and backward propagation as the core. In each iteration, a batch of images is input into the model, and the model processes them to obtain the predicted defect classification probability and segmentation heatmap. Then, based on the total loss function, the combined error between the current prediction and the true label is calculated; next, using the backpropagation algorithm, the gradient of the total loss with respect to all trainable parameters of the model is calculated, and the adaptive moment estimation optimizer is used to update all trainable parameters of the model based on the calculated gradient. During training, the model performance is evaluated periodically on an independent validation set; an iteration stopping condition is set for model training, and training ends only when the iteration stopping condition is met. After training, the parameters of the model with the best performance on the validation set are saved as the final model for identifying typical defects in drainage pipes.

7. The method for identifying typical defects in drainage pipes using machine vision technology according to claim 1, characterized in that, S1 is as follows: First, data is collected using a professional closed-circuit television inspection system or a pipe periscope device. Data collection is carried out under various working conditions, including different drainage pipes, different operating scenarios, and different drainage pipe defects. Frames containing clear defect patterns are selected from the video stream as the original drainage pipe image data, and the corresponding pipe environmental parameters are recorded at the same time. Then, the original drainage pipe images are labeled with data. The labeling work is completed by trained professionals using pixel-level fine labeling with the aid of auxiliary tools. The defect classification label is an image-level classification label that clearly specifies one or more typical defect categories present in the collected image data. The contour label of the defect area is a pixel-level segmentation mask that accurately outlines the contour of each defect area in the collected image data. The defect classification label includes four typical defects: crack, fracture, corrosion and deformation.

8. The method for identifying typical defects in drainage pipes using machine vision technology according to claim 1, characterized in that, S3 is as follows: The original drainage pipe image to be identified is input into the trained drainage pipe typical defect identification model, which outputs a typical defect prediction probability vector and a prediction segmentation heatmap. The category with the highest probability is selected as the final classification and identification result. By setting a probability threshold, the predicted segmentation heatmap is binarized, and morphological operations are applied to smooth the boundaries and remove minor noise, ultimately generating a clear defect region contour mask. Combined with the classification and recognition results, a structured recognition report is generated.

9. A system for identifying typical defects in drainage pipes using machine vision technology, characterized in that, The system includes modules for executing the processing instructions for each step of the method for identifying typical defects in drainage pipes using machine vision technology as described in any one of claims 1 to 8, as follows: Data acquisition and annotation module: The module uses a camera to capture video of the inside of the pipe, extracts the original drainage pipe image data frame by frame from the video stream, and annotates the images, including defect classification labels and defect area outlines. Model building and training module: Constructs a typical defect recognition model for drainage pipes, performs multi-dimensional preprocessing and enhancement on the original images, generates sensitive fusion features through a multi-scale and multi-channel attention fusion module, and outputs relevant results through a parallel recognition and segmentation module; Construct a total loss function, divide the labeled data into training, validation and test sets, train and save the optimal model; Defect identification and report generation module: Input the pipeline image with unknown defects into the trained model, automatically identify and output the defect classification results and predicted segmentation heatmap, generate the defect region contour mask based on the heatmap, and generate a structured recognition report by combining the classification results.

Images