A sewer pipe defect detection method based on image recognition

Abstract

The application discloses a kind of drainage pipeline defect detection methods based on image recognition, comprising: step one: collecting drainage pipeline endoscopic video data, locate pipe inner wall ring area and determine geometric parameter;Step two: carry out circumferential reference alignment and equal arc length development processing, generate standardization development chart;Step three: construct multi-channel feature map and input feature extraction network, output embedded feature map;Step four: embedded feature map is input into improved PaDiM model, calculate Mahalanobis distance and generate initial abnormal score chart;Step five: construct local distribution and calculate optimal transport mapping and Jacobian matrix, generate transport curvature anomaly quantity chart;Step six: carry out structured anomaly reconstruction and generate composite abnormal score chart;Step seven: execute threshold segmentation, morphological processing and cross-frame matching, output defect area position and parameter result.The application realizes the high accuracy and stable identification of drainage pipeline defect.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and machine learning technology, and in particular to a method for detecting defects in drainage pipes based on image recognition. Background Technology

[0002] With the continuous expansion of urban drainage systems and the increasing complexity of underground pipe networks, drainage pipes are prone to defects such as cracks, corrosion, disconnection, leakage, and blockage during long-term operation, directly affecting drainage capacity and structural safety. Current technologies for detecting internal defects in drainage pipes typically rely on manual visual inspection or image analysis methods based on endoscopic videos, using simple threshold segmentation, edge detection, or traditional convolutional neural networks for defect identification. However, in practical engineering applications, these methods generally suffer from insufficient stability, high false detection rates, and limited adaptability to complex defects.

[0003] Traditional image processing methods often rely on grayscale changes or local texture differences for judgment, making it difficult to effectively handle interference factors such as uneven lighting inside pipes, stain occlusion, and water reflection. This results in difficulty in accurately distinguishing real defects from surface noise in complex backgrounds. While detection methods based on ordinary convolutional neural networks have improved recognition accuracy to some extent, they mostly adopt an overall feature modeling approach, ignoring the structural regularity of the pipe unfolded diagram in the circumferential and radial directions. They also lack position-related statistical modeling mechanisms, making it difficult to finely depict the normal distribution differences at different spatial locations.

[0004] Therefore, how to provide a method for detecting defects in drainage pipes based on image recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an image recognition-based method for detecting defects in drainage pipes. This invention constructs a spatial location correlation statistical modeling framework based on ring expansion, introduces an improved PaDiM model and a probabilistic transport curvature anomaly mechanism, and jointly characterizes anomalies from two dimensions: statistical offset and local distribution geometric deformation. Through anomaly consistency tensor reconstruction and cross-frame matching strategies, the invention enhances the stability and positioning accuracy of identifying fine cracks and continuous defects, achieving a structurally sensitive and cross-frame consistent intelligent detection effect.

[0006] A method for detecting defects in drainage pipes based on image recognition according to an embodiment of the present invention includes the following steps: Step 1: Collect endoscopic video data of drainage pipes and perform frame extraction processing to obtain an image frame sequence. For each image frame, locate the annular region on the inner wall of the pipe and determine the geometric parameters of the annular region. Step 2: Based on the geometric parameters of the annular region, perform circumferential reference alignment and equal arc length unfolding on the annular region, and perform illumination normalization to generate a normalized unfolded diagram; Step 3: Construct the standardized unfolded map into a multi-channel feature map, and input the multi-channel feature map into the feature extraction network to output the embedded feature map; Step 4: Input the embedded feature map into the improved PaDiM model, calculate the Mahalanobis distance for the embedding vectors at each spatial location, and obtain the initial anomaly score map; Step 5: The improved PaDiM model introduces a probabilistic transport curvature anomaly mechanism. During the anomaly score calculation process, it constructs a local empirical distribution of the spatial location neighborhood embedding vector, solves the optimal transport mapping, calculates the Jacobian matrix, and outputs a transport curvature anomaly map. Step 6: Perform structured anomaly reconstruction processing on the transport curvature anomaly map and the initial anomaly score map under the spatial neighborhood consistency constraint to generate a composite anomaly score map; Step 7: Perform threshold segmentation and morphological processing on the composite anomaly scoring map to obtain the defect connected regions, and perform cross-frame matching processing on the defect connected regions after circumferential alignment to output the defect region location and corresponding parameter results.

[0007] Optionally, step one is as follows: The endoscopic video data is processed by extracting frames at a preset frame rate, and the obtained image frames are processed by unifying resolution and standardizing grayscale to generate the image frames to be located. Gradient calculation is performed on the image frame to be located, and the horizontal gradient component and the vertical gradient component are obtained respectively. The gradient magnitude map is calculated based on the horizontal gradient component and the vertical gradient component, and the gradient magnitude map is subjected to threshold segmentation to obtain a set of candidate edge points. Based on the candidate edge point set, the circular parameters are accumulated and counted. Different combinations of center coordinates and radii are accumulated and counted in the parameter space. The parameter combination with the largest accumulated value is selected as the outer boundary circle parameter of the inner wall of the pipe. The inner boundary circle parameter is determined by offsetting the outer boundary circle parameter inward by a preset pixel distance. A ring-shaped mask region is constructed based on the outer boundary circle parameters and the inner boundary circle parameters. The mask is then cropped on the image frame to be located to extract the image of the inner wall ring region. Gray-scale projection statistics are performed on the image of the inner wall annular region along the circumferential direction. The average gray value of the radial pixels corresponding to each angle is calculated to form a circumferential gray-scale distribution curve. The circumferential gray-scale distribution curve is smoothed, and the circumferential reference starting angle is determined based on the continuity of gray-scale changes. The center coordinates, outer radius, inner radius, and circumferential reference starting angle are integrated to form a set of geometric parameters for the annular region.

[0008] Optionally, step two is as follows: A polar coordinate system is established based on the geometric parameters of the ring region. The center coordinates are used as the origin of the polar coordinates, and the circumferential reference starting angle is used as the zero point of the angle. The angle value range is set to 0 to 360 degrees, and the radius value range is set to the inner radius to the outer radius. The angle range is uniformly discretized according to a preset angle step size, and the radius range is uniformly discretized according to a preset radius step size. The corresponding planar coordinate position is calculated for each discrete angle and discrete radius combination. The pixel gray value is calculated using bilinear interpolation based on the planar coordinate position. The pixel data is arranged in ascending order of angle and ascending order of radius to construct a two-dimensional pixel matrix and form an equal arc length unfolded image. Based on the column index corresponding to the circumferential reference starting angle, the equal arc length unfolded chart is circumferentially rearranged, and the corresponding column index is used as the new starting column to reorganize and arrange the column data; The uniform arc length unfolded image is subjected to illumination normalization processing. The mean gray value of each row of pixels is calculated along the radial direction. The corresponding row of pixels is normalized and adjusted according to the mean gray value of the row. The overall gray value distribution is linearly stretched to generate a normalized unfolded image.

[0009] Optional, step three is as follows: Perform a one-dimensional discrete Fourier transform on each radial row of the standardized unfolded graph along the circumferential direction to obtain the corresponding complex components in the frequency domain. Determine the dominant frequency component based on the frequency corresponding to the maximum amplitude, and extract the phase and amplitude of the dominant frequency component. Calculate the sine and cosine components of the phase value separately, perform a logarithmic transformation on the amplitude, and stack the phase sine component, phase cosine component and amplitude logarithmic component in the channel dimension to construct a three-channel feature map. The three-channel feature map is input into the ResNet-18 fully convolutional feature extraction network after removing the average pooling layer and the fully connected layer. The feature map is calculated layer by layer through the convolutional layer, the batch normalization layer and the residual block, and the feature map of layer 2, layer 3 and layer 4 is output. The feature maps of layer 2, layer 3, and layer 4 are upsampled using bilinear interpolation to ensure that the three feature maps have the same spatial size. They are then concatenated along the channel dimension and compressed through 1×1 convolution to obtain the embedded feature map.

[0010] Optional, step four is as follows: The embedded feature map is input into the improved PaDiM model, which includes an embedded vector index structure, a position-related Gaussian statistical modeling structure, and a Mahalanobis distance calculation structure. Based on the embedding vector index structure, the embedding vector is extracted point by point from the embedding feature map according to the two-dimensional grid coordinates, and a one-to-one correspondence between the two-dimensional grid coordinates and the embedding vector is established. The location-related Gaussian statistical modeling structure performs statistical calculations on the embedding vectors corresponding to each two-dimensional grid coordinate based on normal sample data, and obtains the mean vector and covariance matrix of the embedding vectors corresponding to each two-dimensional grid coordinate, and stores them according to the two-dimensional grid coordinate index. Diagonal regularization is performed on the covariance matrix corresponding to each two-dimensional grid coordinate, and the inverse matrix of the regularized covariance matrix is ​​calculated. Based on the Mahalanobis distance calculation structure, in the anomaly detection stage, the difference vector between the embedded vector and the corresponding mean vector is calculated for each two-dimensional grid coordinate. The difference vector is then multiplied with the corresponding inverse matrix, and the inner product operation is performed between the result vector and the difference vector to obtain the Mahalanobis distance value. A two-dimensional anomaly scoring matrix is ​​constructed by arranging the Mahalanobis distance values ​​corresponding to the coordinates of each two-dimensional grid in spatial order, thus forming an initial anomaly scoring map.

[0011] Optional, step five is as follows: In the improved PaDiM model, a probabilistic transport curvature anomaly mechanism is introduced during the embedding vector anomaly scoring stage. This probabilistic transport curvature anomaly mechanism includes: Taking the embedding vector corresponding to the spatial location in the embedding feature map as the center, extract the set of neighborhood embedding vectors within the spatial neighborhood window of the embedding feature map, assign uniform weights to each embedding vector in the set of neighborhood embedding vectors, and construct a discrete local empirical distribution. Based on the mean vector and covariance matrix corresponding to spatial location, a reference sample set is generated by random sampling through a multivariate Gaussian distribution to construct a reference distribution; Calculate the squared Euclidean distance between the local empirical distribution sample and the reference distribution sample to form a cost matrix. Add an entropy term to the cost matrix and perform iterative row and column normalization to obtain the optimal transportation plan matrix that satisfies the marginal distribution constraints. Based on the optimal transport plan matrix, perform centroid mapping calculation on the reference sample set to construct the optimal transport mapping function; Perform finite difference operations along each feature dimension within the neighborhood window on the optimal transport mapping function, calculate the partial derivatives of each dimension, and construct the Jacobian matrix of the optimal transport mapping. Calculate the difference matrix between the Jacobian matrix and the identity matrix, and calculate the Frobenius norm of the difference matrix to obtain the probability transport curvature anomaly corresponding to the spatial location; A transport curvature anomaly map is constructed by arranging the probabilistic transport curvature anomalies at each spatial location in spatial order.

[0012] Optional, step six specifically includes: At each spatial location, the anomaly score value in the initial anomaly score map and the curvature anomaly value in the transport curvature anomaly map are obtained respectively, and a two-dimensional anomaly vector composed of the anomaly score value and the curvature anomaly value is constructed. Perform a vector outer product operation on the two-dimensional anomaly vector to construct an anomaly consistency tensor, and perform an element-wise averaging operation on the anomaly consistency tensor and the corresponding transpose tensor to obtain a symmetric anomaly tensor. Within a preset spatial neighborhood window, the symmetric anomaly tensor is accumulated in the neighborhood and divided by the number of spatial locations within the neighborhood window to obtain the neighborhood consistent tensor. Perform eigenvalue decomposition on the neighborhood consistency tensor, extract the maximum eigenvalue, and use the maximum eigenvalue as a measure of structural consistency anomalies; The structural consistency anomaly metric is proportionally weighted with the initial anomaly score to obtain the composite anomaly score. A composite anomaly score map is constructed by arranging the composite anomaly scores of each spatial location in spatial order.

[0013] Optionally, step seven is as follows: Obtain the composite anomaly rating map, calculate the mean and standard deviation of all pixels in the composite anomaly rating map, calculate the threshold value based on the product of the mean, standard deviation and coefficient, and perform binarization processing on the composite anomaly rating map according to the threshold value to obtain the binary anomaly map. The morphological opening and closing operations are performed sequentially on the binary outlier map. The morphological opening operation is calculated in the order of erosion followed by dilation, and the morphological closing operation is calculated in the order of dilation followed by erosion, to obtain the binary region map. Perform connected component labeling on the binary region map, calculate the number of pixels, the coordinate range of the minimum bounding rectangle and the coordinates of the centroid of each connected region, and filter connected regions based on the comparison results of the number of pixels and the area threshold to obtain a set of defective connected regions. Between adjacent image frames, the overlapping area of ​​the defective connected regions is calculated based on the unfolded coordinate system after circumferential alignment. The overlapping ratio is obtained by dividing the overlapping area by the union area of ​​the two defective connected regions. The overlapping ratio is compared with the matching threshold to obtain the cross-frame matching result. The defective connected regions corresponding to the cross-frame matching results are merged by region numbering, and the circumferential angle range, radial position range and area parameters of the defective connected regions are calculated. The location of the defective region and the corresponding parameter results are output.

[0014] The beneficial effects of this invention are: This invention constructs a spatial location-related statistical modeling framework based on the unfolded structure of a pipe circumference, and combines an improved PaDiM model with a probabilistic transport curvature anomaly mechanism. Addressing the challenges of uneven illumination, significant structural location differences, and difficulty in characterizing fine-grained defects in endoscopic images of drainage pipes, it proposes a dual anomaly assessment strategy based on location-related Gaussian modeling and localized geometric deformation measurement. This significantly improves the ability to identify complex defects such as microcracks, corrosion edges, and continuous breaks. In the feature construction stage, a joint expression of frequency domain phase and amplitude is introduced, and the sensitivity to the circumferential structural texture of the pipe is enhanced through circumferential discrete Fourier transform and multi-scale feature fusion. In the anomaly scoring stage, probabilistic transport is introduced. The curvature anomaly mechanism constructs a local empirical distribution of the spatial neighborhood embedding vector and solves for the optimal transport mapping with the Gaussian benchmark distribution. It then calculates the Jacobian matrix and curvature metric of the mapping function, jointly characterizing the degree of anomaly from two dimensions: statistical offset and distributional geometric distortion. In the anomaly fusion stage, an anomaly consistency tensor reconstruction mechanism is constructed. Through neighborhood consistency constraints and eigenvalue decomposition, the anomaly intensity in the main structural direction is extracted, suppressing random noise interference and enhancing the structural continuity response. Finally, by combining threshold segmentation, morphological processing, and cross-frame matching strategies after circumferential alignment, stable localization and parameter output of the defect region are achieved, resulting in the technical effect of structural sensitivity, strong spatial consistency, and stable cross-frame identification of defects in drainage pipes. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the overall process of a drainage pipe defect detection method based on image recognition proposed in this invention; Figure 2 This is a schematic diagram of the structure of the improved PaDiM model in this invention; Figure 3 This is a flowchart of the anomaly consistency tensor reconstruction and composite anomaly score generation process in this invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0017] refer to Figures 1-3 A method for detecting defects in drainage pipes based on image recognition includes the following steps: Step 1: Collect endoscopic video data of drainage pipes and perform frame extraction processing to obtain an image frame sequence. For each image frame, locate the annular region on the inner wall of the pipe and determine the geometric parameters of the annular region. Step 2: Based on the geometric parameters of the annular region, perform circumferential reference alignment and equal arc length unfolding on the annular region, and perform illumination normalization to generate a normalized unfolded map; Step 3: Construct a multi-channel feature map from the standardized unfolded map, and input the multi-channel feature map into the feature extraction network to output the embedded feature map; Step 4: Input the embedded feature map into the improved PaDiM model, calculate the Mahalanobis distance for the embedding vectors at each spatial location, and obtain the initial anomaly score map; Step 5: The improved PaDiM model introduces a probabilistic transport curvature anomaly mechanism. During the anomaly score calculation process, a local empirical distribution of the spatial location neighborhood embedding vector is constructed, the optimal transport mapping is solved, the Jacobian matrix is ​​calculated, and the transport curvature anomaly map is output. Step 6: Perform structured anomaly reconstruction processing on the transport curvature anomaly map and the initial anomaly score map under the spatial neighborhood consistency constraint to generate a composite anomaly score map; Step 7: Perform threshold segmentation and morphological processing on the composite anomaly scoring map to obtain the defect connected regions, and perform cross-frame matching processing on the defect connected regions after circumferential alignment to output the defect region location and corresponding parameter results.

[0018] In this embodiment, step one specifically includes: The endoscopic video data is processed by extracting frames at a preset frame rate, and the obtained image frames are processed by unifying resolution and standardizing grayscale to generate the image frames to be located. Gradient calculation is performed on the image frame to be located, and the horizontal gradient component and the vertical gradient component are obtained respectively. The gradient magnitude map is calculated based on the horizontal gradient component and the vertical gradient component, and the gradient magnitude map is subjected to threshold segmentation to obtain a set of candidate edge points. Based on the candidate edge point set, the circular parameters are accumulated and counted. Different combinations of center coordinates and radii are accumulated and counted in the parameter space. The parameter combination with the largest accumulated value is selected as the outer boundary circle parameter of the inner wall of the pipe. The inner boundary circle parameter is determined by offsetting the outer boundary circle parameter inward by a preset pixel distance. A ring-shaped mask region is constructed based on the outer boundary circle parameters and the inner boundary circle parameters. The mask is then cropped on the image frame to be located to extract the image of the inner wall ring region. Gray-scale projection statistics are performed on the image of the inner wall annular region along the circumferential direction. The average gray value of the radial pixels corresponding to each angle is calculated to form a circumferential gray-scale distribution curve. The circumferential gray-scale distribution curve is smoothed, and the circumferential reference starting angle is determined based on the continuity of gray-scale changes. The center coordinates, outer radius, inner radius, and circumferential reference starting angle are integrated to form a set of geometric parameters for the annular region.

[0019] In the process of accumulating and statistically analyzing circular parameters, a three-dimensional parameter space is constructed with the abscissa, ordinate, and radius of the circle as variables. Each candidate edge point is then cumulatively voted on under different radius conditions to form a parameter space counting matrix. The outer boundary circle parameters are determined by filtering for local extrema and suppressing non-maximum values ​​in the counting matrix. The determination of the circumferential reference starting angle is based on the first-order difference stability interval analysis of the grayscale distribution curve. The grayscale difference between adjacent angles is calculated on the smoothed grayscale curve, and angle segments with continuous differences below a preset threshold are selected. The center angle of these segments is chosen as the reference starting angle, thus ensuring the consistency of the unfolded coordinates across different frames.

[0020] In this embodiment, step two specifically includes: A polar coordinate system is established based on the geometric parameters of the ring region. The center coordinates are used as the origin of the polar coordinates, and the circumferential reference starting angle is used as the zero point of the angle. The angle value range is set to 0 to 360 degrees, and the radius value range is set to the inner radius to the outer radius. The angle range is uniformly discretized according to a preset angle step size, and the radius range is uniformly discretized according to a preset radius step size. The corresponding planar coordinate position is calculated for each discrete angle and discrete radius combination. The pixel gray value is calculated using bilinear interpolation based on the planar coordinate position. The pixel data is arranged in ascending order of angle and ascending order of radius to construct a two-dimensional pixel matrix and form an equal arc length unfolded image. Based on the column index corresponding to the circumferential reference starting angle, the equal arc length unfolded chart is circumferentially rearranged, and the corresponding column index is used as the new starting column to reorganize and arrange the column data; The uniform arc length unfolded image is subjected to illumination normalization processing. The mean gray value of each row of pixels is calculated along the radial direction. The corresponding row of pixels is normalized and adjusted according to the mean gray value of the row. The overall gray value distribution is linearly stretched to generate a normalized unfolded image.

[0021] In this implementation, the angle step size is set to 0.5° to 1°, preferably 1°, corresponding to 360 discrete angles; the radius step size is set to 1 pixel to 2 pixels, preferably 1 pixel; bilinear interpolation performs grayscale weighted calculations based on distance between four adjacent pixels corresponding to the target plane coordinates; the column index is obtained by dividing the circumferential reference starting angle by the angle step size and rounding; the row grayscale mean is obtained by summing the grayscale values ​​of all pixels in the same radial row and dividing by the number of pixels; the linear mapping range is set to 0 to 255.

[0022] In this embodiment, step three specifically includes: Perform a one-dimensional discrete Fourier transform on each radial row of the standardized unfolded graph along the circumferential direction to obtain the corresponding complex components in the frequency domain. Determine the dominant frequency component based on the frequency corresponding to the maximum amplitude, and extract the phase and amplitude of the dominant frequency component. Calculate the sine and cosine components of the phase value separately, perform a logarithmic transformation on the amplitude, and stack the phase sine component, phase cosine component and amplitude logarithmic component in the channel dimension to construct a three-channel feature map. The three-channel feature map is input into the ResNet-18 fully convolutional feature extraction network after removing the average pooling layer and the fully connected layer. The feature map is calculated layer by layer through the convolutional layer, the batch normalization layer and the residual block, and the feature map of layer 2, layer 3 and layer 4 is output. The feature maps of layer 2, layer 3, and layer 4 are upsampled using bilinear interpolation to ensure that the three feature maps have the same spatial size. They are then concatenated along the channel dimension and compressed through 1×1 convolution to obtain the embedded feature map.

[0023] In this implementation, a one-dimensional discrete Fourier transform is performed on each radial row of the standardized unfolded image along the circumferential direction, with a frequency resolution set to 1 / 360, corresponding to 360 discrete frequency points; the dominant frequency component is determined by selecting the frequency component with the largest amplitude; the phase values ​​are calculated as sine and cosine values ​​respectively, and the amplitude is processed using a logarithmic transform log(1+amplitude); the ResNet-18 removes the average pooling layer and the fully connected layer, retaining 4 residual stages; the feature maps of layer 2, layer 3, and layer 4 are upsampled to 1 / 4 of the spatial resolution of the unfolded image size through bilinear interpolation; the number of channels after splicing is 256+512+512, which is compressed into 512-dimensional embedded features by 1×1 convolution.

[0024] In this embodiment, step four specifically includes: The embedded feature map is input into the improved PaDiM model, which includes an embedded vector indexing structure, a position-related Gaussian statistical modeling structure, and a Mahalanobis distance calculation structure. Based on the embedding vector index structure, the embedding vector is extracted point by point from the embedding feature map according to the two-dimensional grid coordinates, and a one-to-one correspondence between the two-dimensional grid coordinates and the embedding vector is established. The location-related Gaussian statistical modeling structure performs statistical calculations on the embedding vectors corresponding to each two-dimensional grid coordinate based on normal sample data, and obtains the mean vector and covariance matrix of the embedding vectors corresponding to each two-dimensional grid coordinate, and stores them according to the two-dimensional grid coordinate index. Diagonal regularization is performed on the covariance matrix corresponding to each two-dimensional grid coordinate, and the inverse matrix of the regularized covariance matrix is ​​calculated. Based on the Mahalanobis distance calculation structure, in the anomaly detection stage, the difference vector between the embedded vector and the corresponding mean vector is calculated for each two-dimensional grid coordinate. The difference vector is then multiplied with the corresponding inverse matrix, and the inner product operation is performed between the result vector and the difference vector to obtain the Mahalanobis distance value. A two-dimensional anomaly scoring matrix is ​​constructed by arranging the Mahalanobis distance values ​​corresponding to the coordinates of each two-dimensional grid in spatial order, thus forming an initial anomaly scoring map.

[0025] In this implementation, the size of the embedded feature map is set to H×W×D, where D is the embedding dimension; location-related Gaussian statistical modeling calculates the mean vector and covariance matrix for the set of embedded vectors at each two-dimensional grid coordinate based on normal sample data; the covariance matrix has a dimension of D×D; the diagonal regularization constant is between 0.001 and 0.01; the inverse matrix is ​​calculated by matrix factorization; during the Mahalanobis distance calculation, the difference vector is first transposed, then multiplied with the inverse of the covariance matrix and the vector dot product is performed; the initial anomaly score map maintains the same spatial resolution as the embedded feature map.

[0026] In this embodiment, step five specifically includes: In the improved PaDiM model, a probabilistic transport curvature anomaly mechanism is introduced during the embedding vector anomaly score calculation stage. This probabilistic transport curvature anomaly mechanism includes: Taking the embedding vector corresponding to the spatial location in the embedding feature map as the center, extract the set of neighborhood embedding vectors within the spatial neighborhood window of the embedding feature map, assign uniform weights to each embedding vector in the set of neighborhood embedding vectors, and construct a discrete local empirical distribution. Based on the mean vector and covariance matrix corresponding to spatial location, a reference sample set is generated by random sampling through a multivariate Gaussian distribution to construct a reference distribution; Calculate the squared Euclidean distance between the local empirical distribution sample and the reference distribution sample to form a cost matrix. Add an entropy term to the cost matrix and perform iterative row and column normalization to obtain the optimal transportation plan matrix that satisfies the marginal distribution constraints. Based on the optimal transport plan matrix, perform centroid mapping calculation on the reference sample set to construct the optimal transport mapping function; Perform finite difference operations along each feature dimension within the neighborhood window on the optimal transport mapping function, calculate the partial derivatives of each dimension, and construct the Jacobian matrix of the optimal transport mapping. Calculate the difference matrix between the Jacobian matrix and the identity matrix, and calculate the Frobenius norm of the difference matrix to obtain the probability transport curvature anomaly corresponding to the spatial location; A transport curvature anomaly map is constructed by arranging the probabilistic transport curvature anomalies at each spatial location in spatial order.

[0027] In this implementation, the improved PaDiM model maintains the same overall structure as the original PaDiM model, both establishing location-related Gaussian statistical models based on the spatial locations in the embedded feature maps. For each spatial location, the corresponding mean vector and covariance matrix are stored. The statistical offset between the embedded vector and the corresponding mean vector is used as the basis for anomaly measurement. The Mahalanobis distance is calculated through weighted operations of the inverse covariance matrix to construct an initial anomaly score, thereby realizing an anomaly detection framework that independently models different spatial locations. The improved PaDiM model introduces a probabilistic transport curvature anomaly mechanism in the anomaly scoring calculation stage. While calculating the Mahalanobis distance, it constructs a discrete local empirical distribution of the neighborhood embedding vector for each spatial location and solves the optimal transport mapping with a reference Gaussian distribution generated based on the corresponding mean vector and covariance matrix. Furthermore, it performs finite difference operations on the optimal transport mapping function to construct a Jacobian matrix and calculates the curvature anomaly based on the Jacobian matrix, thus expanding the anomaly measurement from a single-point statistical distance to a local probabilistic structural deformation measurement. By introducing a probabilistic transport curvature anomaly mechanism, the improved PaDiM model enhances its sensitivity to local structural disturbances and geometric deformations while maintaining the advantages of the original position-related statistical modeling. For defects with spatial continuity, such as texture variations, microcracks, and edge damage, it can simultaneously capture statistical offset and distribution geometric distortion features, improve the stability and discriminativeness of anomaly scoring, and enhance the ability to identify fine-grained defects in complex backgrounds.

[0028] In this embodiment, step six specifically includes: At each spatial location, the anomaly score value in the initial anomaly score map and the curvature anomaly value in the transport curvature anomaly map are obtained respectively, and a two-dimensional anomaly vector composed of the anomaly score value and the curvature anomaly value is constructed. Perform a vector outer product operation on the two-dimensional anomaly vector to construct an anomaly consistency tensor, and perform an element-wise averaging operation on the anomaly consistency tensor and the corresponding transpose tensor to obtain a symmetric anomaly tensor. Within a preset spatial neighborhood window, the symmetric anomaly tensor is accumulated in the neighborhood and divided by the number of spatial locations within the neighborhood window to obtain the neighborhood consistent tensor. Perform eigenvalue decomposition on the neighborhood consistency tensor, extract the maximum eigenvalue, and use the maximum eigenvalue as a measure of structural consistency anomalies; The structural consistency anomaly metric is proportionally weighted with the initial anomaly score to obtain the composite anomaly score. A composite anomaly score map is constructed by arranging the composite anomaly scores of each spatial location in spatial order.

[0029] In this implementation, the two-dimensional anomaly vector consists of the initial anomaly score and the probability transport curvature anomaly value; the anomaly consistency tensor is obtained by performing an outer product operation on the two-dimensional anomaly vector to obtain a second-order matrix; the neighborhood window size is set to 3×3 or 5×5; the neighborhood consistency tensor is obtained by summing the elements of the symmetric anomaly tensor in the neighborhood and then dividing by the number of positions in the neighborhood; the eigenvalue decomposition adopts the symmetric matrix eigenvalue decomposition method; the scaling factor ranges from 0.3 to 0.7; the composite anomaly score map maintains the same spatial resolution as the initial anomaly score map.

[0030] In this embodiment, step seven specifically includes: Obtain the composite anomaly rating map, calculate the mean and standard deviation of all pixels in the composite anomaly rating map, calculate the threshold value based on the product of the mean, standard deviation and coefficient, and perform binarization processing on the composite anomaly rating map according to the threshold value to obtain the binary anomaly map. The morphological opening and closing operations are performed sequentially on the binary outlier map. The morphological opening operation is calculated in the order of erosion followed by dilation, and the morphological closing operation is calculated in the order of dilation followed by erosion, to obtain the binary region map. Perform connected component labeling on the binary region map, calculate the number of pixels, the coordinate range of the minimum bounding rectangle and the coordinates of the centroid of each connected region, and filter connected regions based on the comparison results of the number of pixels and the area threshold to obtain a set of defective connected regions. Between adjacent image frames, the overlapping area of ​​the defective connected regions is calculated based on the unfolded coordinate system after circumferential alignment. The overlapping ratio is obtained by dividing the overlapping area by the union area of ​​the two defective connected regions. The overlapping ratio is compared with the matching threshold to obtain the cross-frame matching result. The defective connected regions corresponding to the cross-frame matching results are merged by region numbering, and the circumferential angle range, radial position range and area parameters of the defective connected regions are calculated. The location of the defective region and the corresponding parameter results are output.

[0031] In this implementation, the threshold value is calculated as the mean plus 2 standard deviations; the morphological structural element is a 3×3 rectangular structural element; the area threshold is set to 20 to 50 pixels; the overlap ratio is calculated by dividing the intersection area by the union area; the matching threshold is set to 0.5; the connected component labeling adopts the 8-neighborhood connectivity rule; the circumferential angle range is calculated by multiplying the column index by the angle step size; the radial position range is calculated by multiplying the row index by the radius step size; and the area parameter is obtained by counting the number of pixels in the connected region and multiplying it by the area of ​​a single pixel.

[0032] Example 1: To verify the feasibility of this invention in practice, it was applied to the annual inspection project of a drainage pipe network in a coastal city. The total length of underground drainage pipes in the main urban area of ​​this city exceeds 420 kilometers, with pipe diameters ranging from 300mm to 1200mm. The pipes have generally been in operation for more than 10 years, and some areas suffer from problems such as concrete aging, misaligned joints, and acid corrosion. Traditional inspection methods mainly rely on manual review of videos and image detection software based on simple threshold segmentation for auxiliary analysis. In practical applications, these methods suffer from high false detection rates, insufficient ability to identify minute cracks, and poor stability across frames. Especially when there is water stain reflection, silt adhesion, and uneven lighting inside the pipes, traditional methods often misjudge reflective areas as defects. Furthermore, they have insufficient recognition rate for early cracks less than 2mm in width.

[0033] In this embodiment, a 2.4-kilometer-long, 800-mm-diameter reinforced concrete drainage pipe was selected as the test object. A tracked endoscopic inspection robot was used for video acquisition, with a video resolution of 1920×1080 and a frame rate of 25fps. First, the video data was processed by frame extraction, extracting an image sequence at 5 frames per second, and then performing resolution unification and grayscale standardization on each frame. Subsequently, the annular region of the pipe's inner wall was located using gradient amplitude calculation and circular parameter accumulation statistics. A polar coordinate system was constructed and a uniform arc length unfolded to generate a standardized unfolded map. The unfolded map size was uniformly 512×2048 pixels, where the horizontal axis represents the circumferential angle and the vertical axis represents the radial position. Next, a one-dimensional discrete Fourier transform was performed on each radial row to extract the dominant frequency phase and amplitude information, constructing a three-channel frequency domain feature map. This map was then input into a ResNet-18 network (with the classification layer removed) to extract an embedding feature map with an embedding dimension of 512.

[0034] In the anomaly detection phase, the improved PaDiM model establishes a position-related Gaussian statistical model for each spatial location in the embedded feature map and calculates Mahalanobis distance to form an initial anomaly scoring map. Simultaneously, a probabilistic transport curvature anomaly mechanism is introduced in the anomaly scoring calculation phase. This involves constructing a local empirical distribution of the neighborhood embedding vector and solving for the optimal transport mapping using a reference Gaussian distribution generated based on the mean vector and covariance matrix. The Jacobian matrix and Frobenius norm are then calculated for the mapping function to obtain the transport curvature anomaly map. Subsequently, an anomaly consistency tensor reconstruction mechanism is used to perform consistency constraint processing within a 3×3 spatial neighborhood, generating a composite anomaly scoring map. Finally, through threshold segmentation using the mean plus 2 standard deviations, morphological opening and closing operations on 3×3 structuring elements, and a cross-frame matching rule with an intersection-union ratio of 0.5, the location of the defect region and its corresponding circumferential angle range, radial position range, and area parameters are output.

[0035] To verify the beneficial effects of this invention, three methods were selected for a comparative experiment: the traditional grayscale threshold segmentation method, the standard PaDiM method, and the method of this invention. 312 manually labeled defects were identified, including four categories: cracks, corrosion peeling, interface misalignment, and leakage traces. Among these, 112 were microcracks. The recognition rate was calculated based on defect instances. The experimental results are shown in Table 1.

[0036] Table 1. Performance Comparison of Different Methods in Endoscopic Video Defect Detection of Drainage Pipes

[0037] As shown in Table 1, the present invention achieves an overall recognition rate of 94.8%, an improvement of 8.1 percentage points compared to the standard PaDiM, and a 17.8 percentage point improvement in microcrack recognition rate, while reducing the false detection rate to 4.1%. The average positioning error has been reduced from 8.6 pixels to 4.3 pixels, and cross-frame stability has been improved to 92.6%. In actual engineering verification, the present invention can effectively suppress false detections caused by water stain reflection and form a stable structural response region for continuous cracks, avoiding false detections caused by single-frame fluctuations.

[0038] The results of the above implementation show that the present invention can simultaneously characterize statistical offset and local geometric deformation in complex drainage pipeline inspection scenarios, enhance the ability to identify fine-grained defects, improve detection stability and positioning accuracy, and verify the feasibility and significant technical effects of the present invention in practical engineering applications.

[0039] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting defects in drainage pipes based on image recognition, characterized in that, The steps include the following: Step 1: Collect endoscopic video data of drainage pipes and perform frame extraction processing to obtain an image frame sequence. For each image frame, locate the annular region on the inner wall of the pipe and determine the geometric parameters of the annular region. Step 2: Based on the geometric parameters of the annular region, perform circumferential reference alignment and equal arc length unfolding on the annular region, and perform illumination normalization to generate a normalized unfolded diagram; Step 3: Construct the standardized unfolded map into a multi-channel feature map, and input the multi-channel feature map into the feature extraction network to output the embedded feature map; Step 4: Input the embedded feature map into the improved PaDiM model, calculate the Mahalanobis distance for the embedding vectors at each spatial location, and obtain the initial anomaly score map; Step 5: The improved PaDiM model introduces a probabilistic transport curvature anomaly mechanism. During the anomaly score calculation process, it constructs a local empirical distribution of the spatial location neighborhood embedding vector, solves the optimal transport mapping, calculates the Jacobian matrix, and outputs a transport curvature anomaly map. Step 6: Perform structured anomaly reconstruction processing on the transport curvature anomaly map and the initial anomaly score map under the spatial neighborhood consistency constraint to generate a composite anomaly score map; Step 7: Perform threshold segmentation and morphological processing on the composite anomaly scoring map to obtain the defect connected regions, and perform cross-frame matching processing on the defect connected regions after circumferential alignment to output the defect region location and corresponding parameter results.

2. The method for detecting defects in drainage pipes based on image recognition according to claim 1, characterized in that, Step one specifically involves: The endoscopic video data is processed by extracting frames at a preset frame rate, and the obtained image frames are processed by unifying resolution and standardizing grayscale to generate the image frames to be located. Gradient calculation is performed on the image frame to be located, and the horizontal gradient component and the vertical gradient component are obtained respectively. The gradient magnitude map is calculated based on the horizontal gradient component and the vertical gradient component, and the gradient magnitude map is subjected to threshold segmentation to obtain a set of candidate edge points. Based on the candidate edge point set, the circular parameters are accumulated and counted. Different combinations of center coordinates and radii are accumulated and counted in the parameter space. The parameter combination with the largest accumulated value is selected as the outer boundary circle parameter of the inner wall of the pipe. The inner boundary circle parameter is determined by offsetting the outer boundary circle parameter inward by a preset pixel distance. A ring-shaped mask region is constructed based on the outer boundary circle parameters and the inner boundary circle parameters. The mask is then cropped on the image frame to be located to extract the image of the inner wall ring region. Gray-scale projection statistics are performed on the image of the inner wall annular region along the circumferential direction. The average gray value of the radial pixels corresponding to each angle is calculated to form a circumferential gray-scale distribution curve. The circumferential gray-scale distribution curve is smoothed, and the circumferential reference starting angle is determined based on the continuity of gray-scale changes. The center coordinates, outer radius, inner radius, and circumferential reference starting angle are integrated to form a set of geometric parameters for the annular region.

3. The method for detecting defects in drainage pipes based on image recognition according to claim 1, characterized in that, Step two specifically involves: A polar coordinate system is established based on the geometric parameters of the ring region. The center coordinates are used as the origin of the polar coordinates, and the circumferential reference starting angle is used as the zero point of the angle. The angle value range is set to 0 to 360 degrees, and the radius value range is set to the inner radius to the outer radius. The angle range is uniformly discretized according to a preset angle step size, and the radius range is uniformly discretized according to a preset radius step size. The corresponding planar coordinate position is calculated for each discrete angle and discrete radius combination. The pixel gray value is calculated using bilinear interpolation based on the planar coordinate position. The pixel data is arranged in ascending order of angle and ascending order of radius to construct a two-dimensional pixel matrix and form an equal arc length unfolded image. Based on the column index corresponding to the circumferential reference starting angle, the equal arc length unfolded chart is circumferentially rearranged, and the corresponding column index is used as the new starting column to reorganize and arrange the column data; The uniform arc length unfolded image is subjected to illumination normalization processing. The mean gray value of each row of pixels is calculated along the radial direction. The corresponding row of pixels is normalized and adjusted according to the mean gray value of the row. The overall gray value distribution is linearly stretched to generate a normalized unfolded image.

4. The method for detecting defects in drainage pipes based on image recognition according to claim 1, characterized in that, Step three specifically involves: Perform a one-dimensional discrete Fourier transform on each radial row of the standardized unfolded graph along the circumferential direction to obtain the corresponding complex components in the frequency domain. Determine the dominant frequency component based on the frequency corresponding to the maximum amplitude, and extract the phase and amplitude of the dominant frequency component. Calculate the sine and cosine components of the phase value separately, perform a logarithmic transformation on the amplitude, and stack the phase sine component, phase cosine component and amplitude logarithmic component in the channel dimension to construct a three-channel feature map. The three-channel feature map is input into the ResNet-18 fully convolutional feature extraction network after removing the average pooling layer and the fully connected layer. The feature map is calculated layer by layer through the convolutional layer, the batch normalization layer and the residual block, and the feature map of layer 2, layer 3 and layer 4 is output. The feature maps of layer 2, layer 3, and layer 4 are upsampled using bilinear interpolation to ensure that the three feature maps have the same spatial size. They are then concatenated along the channel dimension and compressed through 1×1 convolution to obtain the embedded feature map.

5. The method for detecting defects in drainage pipes based on image recognition according to claim 1, characterized in that, Step four specifically involves: The embedded feature map is input into the improved PaDiM model, which includes an embedded vector index structure, a position-related Gaussian statistical modeling structure, and a Mahalanobis distance calculation structure. Based on the embedding vector index structure, the embedding vector is extracted point by point from the embedding feature map according to the two-dimensional grid coordinates, and a one-to-one correspondence between the two-dimensional grid coordinates and the embedding vector is established. The location-related Gaussian statistical modeling structure performs statistical calculations on the embedding vectors corresponding to each two-dimensional grid coordinate based on normal sample data, and obtains the mean vector and covariance matrix of the embedding vectors corresponding to each two-dimensional grid coordinate, and stores them according to the two-dimensional grid coordinate index. Diagonal regularization is performed on the covariance matrix corresponding to each two-dimensional grid coordinate, and the inverse matrix of the regularized covariance matrix is ​​calculated. Based on the Mahalanobis distance calculation structure, in the anomaly detection stage, the difference vector between the embedded vector and the corresponding mean vector is calculated for each two-dimensional grid coordinate. The difference vector is then multiplied with the corresponding inverse matrix, and the inner product operation is performed between the result vector and the difference vector to obtain the Mahalanobis distance value. A two-dimensional anomaly scoring matrix is ​​constructed by arranging the Mahalanobis distance values ​​corresponding to the coordinates of each two-dimensional grid in spatial order, thus forming an initial anomaly scoring map.

6. The method for detecting defects in drainage pipes based on image recognition according to claim 1, characterized in that, Step five specifically involves: In the improved PaDiM model, a probabilistic transport curvature anomaly mechanism is introduced during the embedding vector anomaly scoring stage. This probabilistic transport curvature anomaly mechanism includes: Taking the embedding vector corresponding to the spatial location in the embedding feature map as the center, extract the set of neighborhood embedding vectors within the spatial neighborhood window of the embedding feature map, assign uniform weights to each embedding vector in the set of neighborhood embedding vectors, and construct a discrete local empirical distribution. Based on the mean vector and covariance matrix corresponding to spatial location, a reference sample set is generated by random sampling through a multivariate Gaussian distribution to construct a reference distribution; Calculate the squared Euclidean distance between the local empirical distribution sample and the reference distribution sample to form a cost matrix. Add an entropy term to the cost matrix and perform iterative row and column normalization to obtain the optimal transportation plan matrix that satisfies the marginal distribution constraints. Based on the optimal transport plan matrix, perform centroid mapping calculation on the reference sample set to construct the optimal transport mapping function; Perform finite difference operations along each feature dimension within the neighborhood window on the optimal transport mapping function, calculate the partial derivatives of each dimension, and construct the Jacobian matrix of the optimal transport mapping. Calculate the difference matrix between the Jacobian matrix and the identity matrix, and calculate the Frobenius norm of the difference matrix to obtain the probability transport curvature anomaly corresponding to the spatial location; A transport curvature anomaly map is constructed by arranging the probabilistic transport curvature anomalies at each spatial location in spatial order.

7. The method for detecting defects in drainage pipes based on image recognition according to claim 1, characterized in that, Step six specifically involves: At each spatial location, the anomaly score value in the initial anomaly score map and the curvature anomaly value in the transport curvature anomaly map are obtained respectively, and a two-dimensional anomaly vector composed of the anomaly score value and the curvature anomaly value is constructed. Perform a vector outer product operation on the two-dimensional anomaly vector to construct an anomaly consistency tensor, and perform an element-wise averaging operation on the anomaly consistency tensor and the corresponding transpose tensor to obtain a symmetric anomaly tensor. Within a preset spatial neighborhood window, the symmetric anomaly tensor is accumulated in the neighborhood and divided by the number of spatial locations within the neighborhood window to obtain the neighborhood consistent tensor. Perform eigenvalue decomposition on the neighborhood consistency tensor, extract the maximum eigenvalue, and use the maximum eigenvalue as a measure of structural consistency anomalies; The structural consistency anomaly metric is proportionally weighted with the initial anomaly score to obtain the composite anomaly score. A composite anomaly score map is constructed by arranging the composite anomaly scores of each spatial location in spatial order.

8. The method for detecting defects in drainage pipes based on image recognition according to claim 1, characterized in that, Step seven specifically involves: Obtain the composite anomaly rating map, calculate the mean and standard deviation of all pixels in the composite anomaly rating map, calculate the threshold value based on the product of the mean, standard deviation and coefficient, and perform binarization processing on the composite anomaly rating map according to the threshold value to obtain the binary anomaly map. The morphological opening and closing operations are performed sequentially on the binary outlier map. The morphological opening operation is calculated in the order of erosion followed by dilation, and the morphological closing operation is calculated in the order of dilation followed by erosion, to obtain the binary region map. Perform connected component labeling on the binary region map, calculate the number of pixels, the coordinate range of the minimum bounding rectangle and the coordinates of the centroid of each connected region, and filter connected regions based on the comparison results of the number of pixels and the area threshold to obtain a set of defective connected regions. Between adjacent image frames, the overlapping area of ​​the defective connected regions is calculated based on the unfolded coordinate system after circumferential alignment. The overlapping ratio is obtained by dividing the overlapping area by the union area of ​​the two defective connected regions. The overlapping ratio is compared with the matching threshold to obtain the cross-frame matching result. The defective connected regions corresponding to the cross-frame matching results are merged by region numbering, and the circumferential angle range, radial position range and area parameters of the defective connected regions are calculated. The location of the defective region and the corresponding parameter results are output.

Images