Method and system for visual recognition of defects in underground pipe network

Abstract

The application relates to the technical field of visual data processing, and discloses an underground pipe network defect visual identification method and system. The method is characterized in that: firstly, a pipe network defect type is preset, corresponding training image data is collected and expanded, and two deep learning models based on target detection are trained according to the training image data; secondly, an underground pipe network image is acquired, part of the image is split in proportion as training data, and the remaining image is preprocessed to realize defect feature shift, so that input data is obtained; the input data is input into a first model to obtain output data and mark the output data on the image; finally, the input data is input into a second model to obtain verification data according to a preset verification period, a consistency value and a consistency ratio of the two are calculated, and an image extraction ratio is adjusted according to a negative correlation of the ratio. Through the double-model collaborative verification and dynamic ratio adjustment, the accuracy and adaptability of underground pipe network defect identification are improved.

Description

Technical Field

[0001] This application relates to the technical field of visual data processing, and in particular to a method and system for visual identification of defects in underground pipeline networks. Background Technology

[0002] As the core carrier of critical infrastructure such as water supply, drainage, gas, and heating, urban underground pipe networks are directly related to people's livelihood and the normal operation of cities. With the increasing service life of these networks, they are prone to defects such as cracks, disconnections, deformation, and blockages due to factors such as soil corrosion, geological subsidence, and external construction disturbances. If these defects are not detected and repaired in time, they may lead to safety accidents such as media leaks, road collapses, and environmental pollution. Currently, the scale of urban pipe networks continues to expand, and traditional detection methods are insufficient to cover complex pipe network scenarios. Therefore, the demand for efficient and accurate pipe defect detection technology is becoming increasingly urgent, becoming a key requirement for ensuring the safe operation and maintenance of pipe networks.

[0003] Among existing pipeline defect detection methods, image-based manual interpretation is the most common. This method typically uses equipment such as pipeline inspection robots and pipeline periscopes to penetrate deep into the pipeline and capture images or video data of the inner wall. Professional inspectors then export the acquired images one by one, manually observe, identify, and mark them against industry defect judgment standards to determine the type, location coordinates, and severity level of the defects, ultimately generating an inspection report. While this method can achieve a certain level of defect detection and has played an important role in the inspection of small and medium-sized pipeline networks, the overall inspection process relies excessively on manual operation.

[0004] Existing pipeline inspection methods, which rely heavily on manual interpretation, still cannot fully meet the demands for efficient and accurate inspection in actual production. On the one hand, a single pipeline segment in a city can be hundreds of meters long, and the number of images collected for a single segment often reaches hundreds or even thousands of frames. Manually interpreting each frame requires a large number of professional personnel and has a long inspection cycle. Especially in large-scale pipeline network inspection scenarios, human resources are scarce and their utilization efficiency is extremely low, severely restricting the progress of inspection work. On the other hand, manual interpretation is easily affected by subjective factors. The experience level, work focus, and differences in judgment standards of the inspectors can all lead to deviations in the judgment of the same defect. This can not only easily miss subtle defects but also misjudge non-defect features as defects, causing distortion of defect information and adversely affecting subsequent pipeline network repair decisions and safe operation and maintenance. Summary of the Invention

[0005] To achieve automated detection of pipeline defects, this application provides a visual identification method and system for underground pipeline network defects.

[0006] Firstly, this application provides a visual identification method for defects in underground pipeline networks, employing the following technical solution:

[0007] A visual identification method for defects in underground pipe networks includes the following steps:

[0008] Obtain the preset pipeline defect type, collect the corresponding training image data based on the pipeline defect type, expand the training image data, and train the preset first deep learning model and second deep learning model based on target detection based on the expanded training image data.

[0009] Acquire underground pipe network images, extract a portion of the underground pipe network images as training data according to a preset extraction ratio; preprocess the other portion of the underground pipe network images to obtain input data, the preprocessing is used to shift the defect features in the underground pipe network images;

[0010] Input data is fed into the first deep learning model to obtain output data, and the output data is marked on the underground pipe network image;

[0011] Based on a preset verification period, the input data is fed into the second deep learning model to obtain verification data, and the consistency value between the verification data and the output data is calculated. The consistency ratio is calculated based on the consistency value and the preset consistency reference value, and the extraction ratio is adjusted according to the negative correlation of the consistency ratio.

[0012] By adopting the above technical solution, and by pre-setting pipeline defect types and expanding training image data, a rich and diverse training foundation is provided for the dual deep learning model, effectively improving the model's adaptability and generalization ability to pipeline defects under different types and operating conditions. Preprocessing is used to shift image defect features, helping the model to more accurately capture core defect features and reduce interference from non-defect areas. Through the collaborative work of the two models, the second deep learning model verifies the output results of the first model, and the training data extraction ratio is dynamically adjusted by the consistency ratio, achieving adaptive optimization of the training process. This reduces subjective errors and manpower input in manual interpretation, significantly reduces false detections and missed detections, and greatly improves the accuracy of pipeline defect identification.

[0013] Furthermore, the step of collecting corresponding training image data based on pipeline defect types also includes the following sub-steps:

[0014] Training image data has type, size, and illumination attributes;

[0015] The training image data is generated based on the preset first illumination condition and the preset second illumination condition as the content conditions for the corresponding illumination attributes; wherein, the same training image data is divided into the first illumination condition and the second illumination condition at different times, the proportion of the first illumination condition is called the first illumination proportion, the proportion of the second illumination condition is called the second illumination proportion, and the sum of the illumination proportions is one.

[0016] Based on the generated content conditions, a preset first size range and a preset second size range are selected as the data capacity of the training image data for the corresponding size attributes; wherein, the proportion of the first size range is selected as the first size proportion, the proportion of the second size range is selected as the second size proportion, and the sum of the size proportions is one.

[0017] Based on the generated content conditions and data capacity, training image data with corresponding type attributes are generated using a preset first type channel and a preset second type channel; wherein, the proportion using the first type channel is the first type proportion, the proportion using the second type channel is the second type proportion, and the sum of the type proportions is one.

[0018] The first illumination ratio, the first size ratio, and the first type ratio are randomly adjusted within a preset ratio range.

[0019] By adopting the above technical solution, and by clarifying the three core attributes of training image data—type, size, and illumination—the key influencing factors in pipeline defect detection are fully covered by setting dual illumination conditions, dual size ranges, and dual type channels. Combined with a random proportion adjustment mechanism, this ensures the comprehensiveness and balance of training image data in dimensions such as illumination changes, size differences, and type diversity, while avoiding data distribution bias and effectively improving the representativeness and richness of the dataset.

[0020] Furthermore, the step of augmenting the training image data also includes the following sub-steps:

[0021] Obtain a first expansion instruction, rotate the training image data in response to the first expansion instruction, extract the expansion range value and expansion amplitude value from the first expansion instruction, adjust the number of training image data to be rotated according to the positive correlation of the expansion range value, and adjust the rotation angle of the training image data according to the positive correlation of the expansion amplitude value.

[0022] Obtain the second expansion instruction, scale the training image data in response to the second expansion instruction, extract the expansion range value and expansion magnitude value in the second expansion instruction, adjust the amount of scaled training image data according to the positive correlation of the expansion range value, and adjust the scaling factor of the training image data according to the positive correlation of the expansion magnitude value.

[0023] Obtain the third expansion instruction, and in response to the third expansion instruction, flip the training image data. Extract the expansion range value and expansion amplitude value from the third expansion instruction, adjust the number of training image data to be flipped according to the positive correlation of the expansion range value, and select the flipping direction of the training image data according to the expansion amplitude value.

[0024] By adopting the above technical solution, and through three targeted data augmentation methods—rotation, scaling, and flipping—combined with a positive correlation adjustment mechanism between the augmentation range and the augmentation magnitude, the number of augmented samples can be flexibly controlled, and the intensity and dimensionality of the augmentation operation can be precisely adjusted. This efficiently generates a massive amount of diverse training samples, significantly enriching the coverage dimensions of the dataset.

[0025] Furthermore, the method also includes the following steps:

[0026] The illumination types in the underground pipe network image are obtained based on a preset illumination extraction algorithm, and the number of illumination types is calculated.

[0027] If the number of lighting types is less than the preset number of lighting references, the ratio of lighting types is calculated based on the number of lighting types to the number of lighting references, and the number of lighting type categories in the lighting attributes is adjusted according to the negative correlation of the ratio of lighting types.

[0028] The preprocessing steps also include the following sub-steps:

[0029] The image of the underground pipe network is preprocessed using an illumination compensation algorithm, which includes histogram equalization or adaptive histogram equalization.

[0030] An illumination-robust feature extractor and classifier are trained for object detection, and the trained feature extractor and classifier are applied to the first deep learning model and the second deep learning model.

[0031] By adopting the above technical solutions, the illumination type in underground pipe network images is dynamically detected through illumination extraction algorithms, and the number of illumination attribute classifications is adaptively adjusted to ensure that the illumination classification of the dataset matches the actual scene. Combined with illumination compensation preprocessing of histogram equalization or adaptive histogram equalization, the image quality under different illumination conditions is effectively improved, and the interference caused by uneven illumination is weakened. At the same time, feature extractors and classifiers with illumination robustness are integrated into the training of dual deep learning models to enhance the model's feature recognition ability in complex illumination environments and solve the problem of false detection caused by illumination interference.

[0032] Furthermore, the method also includes the following steps:

[0033] The image sharpness type of the underground pipe network is obtained based on a preset feature extraction algorithm, and the number of sharpness types is calculated.

[0034] If the number of sharpness types is less than the preset number of sharpness references, the ratio of sharpness types is calculated based on the number of sharpness types to the number of sharpness references, and the number of size types in the size attribute is adjusted negatively based on the ratio of sharpness types.

[0035] The preprocessing steps also include the following sub-steps:

[0036] Image preprocessing is performed using noise suppression algorithms, including median filtering or Gaussian filtering.

[0037] A noise-robust feature extractor and classifier are trained for object detection, and the trained feature extractor and classifier are applied to the first deep learning model and the second deep learning model.

[0038] By adopting the above technical solutions, the feature extraction algorithm dynamically identifies the image sharpness type and adaptively adjusts the number of size attribute classifications to ensure that the dataset size classification accurately matches the actual image sharpness, avoiding data distribution imbalance. Preprocessing images with noise suppression algorithms such as median filtering and Gaussian filtering effectively filters out interference noise and improves image quality, clearing obstacles for feature extraction. Simultaneously, a noise-robust feature extractor and classifier are integrated into the dual deep learning model training, helping the model stably extract defect features in noisy environments. Combined with potential label smoothing and regularization optimization strategies, overfitting is avoided, solving the problem of false detections caused by noise interference.

[0039] Furthermore, the method also includes the following steps:

[0040] An adaptive image enhancement algorithm is used to preprocess the acquired underground pipe network images by region. The adaptive image enhancement algorithm includes histogram equalization. The processing process is as follows: the image is divided into dark areas, bright areas and transition areas according to the light intensity. Adaptive histogram equalization is used to improve the gray-scale dynamic range in the dark areas. Contrast-limited histogram equalization is used to avoid overexposure in the bright areas. The gray-scale gradient is kept smooth in the transition areas.

[0041] Based on the preprocessed underground pipe network image, the illumination-invariant feature extraction method is used to extract defect features. The illumination-invariant feature extraction method includes: a frequency domain-based method using Fourier transform and wavelet transform to extract the frequency domain amplitude features of the defects; and a color space-based method using HSV color space and Lab color space to extract the color component features of the defects. The defect features include gray-level invariant moments, color moments, and texture features.

[0042] A deep learning model containing a retinal network is selected, wherein the retinal network is configured with a feature pyramid network whose scale covers a set pixel range; pre-trained weights from the ImageNet dataset are loaded using a pre-training method, and the model is fine-tuned through transfer learning; and a generative adversarial network is combined to generate defect samples simulating complex lighting conditions to optimize the training process of the deep learning model.

[0043] Multimodal data from the same viewpoint as visible light images of underground pipe networks are collected. The multimodal data includes infrared images and depth images. After registration processing of the multimodal data, depth features and temperature features are extracted.

[0044] The extracted defect features are fused with multimodal features at the feature level. Specifically, the feature maps are concatenated and weighted summed through convolutional layers to obtain the fused features.

[0045] The fused features are input into the optimized deep learning model to complete the identification of underground pipe network defects, and output the defect type, identification confidence level and the approximate area of ​​the defect in the image.

[0046] By adopting the above technical solutions, the uneven illumination problem is optimized through regional adaptive image enhancement, the defect feature identification is enhanced by combining illumination-invariant feature extraction, and the model is optimized by pre-trained transfer learning, adversarial generative network and multimodal feature fusion. This effectively overcomes the interference of illumination changes on defect identification, significantly reduces the risk of missed detection due to illumination factors, improves the accuracy of defect type judgment and confidence assessment, and enhances the model's adaptability to complex underground environments.

[0047] Furthermore, it also includes the following steps:

[0048] Collect defect images of underground pipe networks. The defect images cover scenarios where the pipes are aged, worn, or corroded, and include underground pipe network images of different pipe diameters, laying environments, and service stages. Construct an initial training dataset based on the defect images.

[0049] The initial training dataset is augmented using data augmentation algorithms to obtain the target training dataset. The data augmentation algorithms include image transformation and noise addition. Image transformation includes rotation, flipping, scaling, cropping, and brightness adjustment. Noise addition includes Gaussian noise addition and salt-and-pepper noise addition.

[0050] Based on the target training dataset, a deep learning model for defect identification is iteratively trained. During the training process, regularization algorithms and ensemble learning are used to optimize the model training process. The regularization algorithms include L1 regularization, L2 regularization, and Dropout regularization. The ensemble learning includes random forest ensemble, gradient boosting tree ensemble, and multi-model voting ensemble. The optimization operation specifically involves introducing a regularization term to suppress overfitting during the model backpropagation process and fusing the output results of multiple base models through ensemble learning.

[0051] A mapping table is provided between preset pipeline parameter combinations and algorithm thresholds and recognition accuracy. The mapping table stores the optimal threshold range and accuracy control range corresponding to different combinations of pipeline types, pipeline materials, and pipeline service life.

[0052] Obtain the pipeline parameters of the actual detection scenario, including pipeline type, pipeline material, and pipeline service life. Query the mapping table based on the pipeline parameters to determine the target algorithm threshold and target recognition accuracy.

[0053] By using the model parameter configuration interface, the corresponding parameters of the deep learning model are adjusted according to the target algorithm threshold and target recognition accuracy to complete the model adaptation.

[0054] By adopting the above technical solutions, a defect dataset covering multiple scenarios is constructed and the sample diversity is expanded through diversified data enhancement. Regularization and ensemble learning are combined to suppress model overfitting. Then, the dynamic adaptation of algorithm threshold and recognition accuracy is achieved through pipeline parameter mapping table. This not only improves the generalization ability and robustness of the model, but also ensures its accurate adaptation in pipeline scenarios with different pipe diameters, materials, and service years, effectively improving the recognition effect under small sample and multiple working conditions.

[0055] Furthermore, it also includes the following steps:

[0056] Acquire images of underground pipe networks containing complex overlapping defects;

[0057] The underground pipe network image is decomposed and enhanced using a multi-scale analysis algorithm. Defect distinguishing features are extracted sequentially at preset small, medium, and large scales. The defect distinguishing features include defect edge contour features, gray-scale distribution features, and texture features.

[0058] The preset neural network and its derived models are used as deep learning models, and the deep learning models have a built-in feature fusion module.

[0059] The defect distinguishing features at small, medium, and large scales are input into the deep learning model, and the feature fusion module performs weighted fusion of the defect distinguishing features at different scales to obtain multi-scale fused features.

[0060] In the process of classifying multi-scale fused features in the deep learning model, transfer learning and defect identification knowledge from other fields are incorporated. The defect identification knowledge from other fields includes knowledge of metal structure corrosion defect identification and knowledge of non-metallic pipe wear defect identification.

[0061] Through the synergistic effect of the deep learning model and the multi-scale analysis algorithm, the system can identify the overlapping defects of complex shapes and defects of different scales in underground pipe networks, and output the defect type and the corresponding regional coordinates of the defect in the image.

[0062] By adopting the above technical solution, multi-scale analysis is used to extract and fuse the distinguishing features of defects at different scales. Transfer learning and cross-domain defect recognition knowledge are incorporated, and deep learning models work together to efficiently solve the problems of overlapping defects in complex shapes and difficulty in identifying defects at different scales, thereby significantly improving the comprehensiveness and accuracy of identifying various complex defects.

[0063] Furthermore, the defect localization process includes the following steps:

[0064] Acquire images of the underground pipe network to be located for defects, and perform noise reduction preprocessing on the underground pipe network images, using median filtering to remove salt-and-pepper noise and Gaussian filtering to remove Gaussian noise;

[0065] The preprocessed underground pipe network image is processed using a preset neural network algorithm, including Faster R-CNN, YOLOv5, and SSD. Through the network's region proposal module or anchor box mechanism, defect candidate features with clear location information are extracted. The location information includes pixel coordinates and region aspect ratio.

[0066] The candidate features of the defects are optimized and screened using contextual information of underground pipeline defects. The contextual information includes the gray-level difference threshold between adjacent pixels or the connectivity judgment condition of adjacent regions. The specific steps are: calculating the gray-level difference between the candidate defect region and the neighboring pixels, and retaining features with a gray-level difference greater than a preset threshold; or, eliminating isolated pseudo-defect features through regional connectivity analysis.

[0067] To facilitate collaborative localization, an auxiliary method is introduced, which includes an image segmentation method and an object detection method: a semantic segmentation algorithm is used to segment the pipe region where the defect is located to eliminate background interference; and an object detection method is used to perform secondary bounding selection on the segmented pipe region to obtain the preliminary boundary box of the defect.

[0068] A preset defect location strategy library is provided, which includes location strategies and threshold ranges corresponding to different scenarios and different defect types.

[0069] Based on the actual detection scenario and defect type, target localization strategies and target thresholds are matched from the strategy library, and the identification strategies and thresholds corresponding to defect localization are adjusted through the model parameter configuration interface;

[0070] Based on the optimized defect features, the output of the auxiliary method, and the adjusted policy threshold, the final bounding box of the defect is determined, and the precise pixel coordinates and positioning deviation range of the defect in the image are output.

[0071] By adopting the above technical solutions, denoising preprocessing and context feature optimization are used to filter out false defects. Combined with image segmentation and target detection to assist in localization, and dynamic adaptation with a scenario-based localization strategy library, the problem of overlapping localization caused by the complex shape of defects is effectively solved. At the same time, the accuracy of defect localization is improved, the precise coordinates and deviation range are clearly output, and background interference and localization error are reduced.

[0072] Secondly, this application provides a visual identification system for defects in underground pipeline networks, employing the following technical solution:

[0073] A visual identification system for underground pipeline defects includes a processor, wherein the processor performs the steps of the visual identification method for underground pipeline defects as described in any of the preceding claims. Attached Figure Description

[0074] Figure 1 This is a step-by-step diagram of a visual identification method for defects in underground pipe networks. Detailed Implementation

[0075] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0076] This application discloses a visual identification method for defects in underground pipeline networks, aiming to address the core pain points of existing underground pipeline inspection methods, such as low efficiency and large subjective errors in manual interpretation, and the susceptibility of algorithms to false detections, missed detections, and overlapping defect identification errors. A full-process solution is built based on deep learning technology. The system consists of two parts: a pipeline robot and an inspection system. The pipeline robot is equipped with a high-definition camera, infrared sensor, depth sensor, and motion control module, enabling autonomous movement within pipelines of DN300-DN1000mm specifications, achieving real-time video transmission and full-area scanning of the pipeline interior. The inspection system integrates the advantages of two core target detection algorithms, YOLOv7 / 8 and EfficientDet. Through algorithm fusion and optimization, it balances detection speed and recognition accuracy. Related operations can be completed through a dedicated APP and web interface, supporting functions such as video upload, real-time recognition, and report generation. The recognition results can intuitively label the defect type, location, and confidence level, clearly presenting the detection effects of defects such as misalignment (confidence level 0.84) and rupture.

[0077] Reference Figure 1 The specific steps are as follows:

[0078] Combining industry standards and actual testing needs, predefined pipeline defect types are identified, specifically including seven core defects: misalignment, rupture, aging, wear, corrosion, disconnection, and leakage. Among these, misalignment and rupture are high-frequency defects, requiring focused optimization of identification accuracy. Based on these defect types, CCTV pipeline inspection robots are used to collect internal pipeline videos. The video coverage includes different pipeline types such as municipal sewage pipes, rainwater pipes, and industrial drainage pipes. Pipe materials include common materials such as reinforced concrete pipes, steel pipes, and plastic pipes, with pipe diameters ranging from DN300 to DN800 mm and service lives ranging from 3 to 30 years, ensuring the representativeness of the data scenarios.

[0079] During the data acquisition process, key factors affecting image quality needed to be systematically analyzed: The light source used was an LED ring light mounted on the pipeline robot, with the illumination intensity adjusted to 500-1500 lux to avoid strong light reflection and shadows in dark areas; the image acquisition device was a 1080P high-definition industrial camera, with a frame rate set to 25fps and an exposure time adjusted to 1 / 500s to ensure image clarity and dynamic capture capability. One image was extracted from every 10 frames of the acquired video, and images with typical defect characteristics (such as misalignment gap ≥5mm, crack length ≥10mm) and no severe blurring or occlusion were selected as raw training data. A total of 12,000 raw images were collected, including 2,800 images of misalignment, 3,200 images of cracks, 1,800 images of aging, 1,500 images of wear, 1,600 images of corrosion, 800 images of disintegration, and 300 images of leakage.

[0080] The original images were manually annotated using the LabelImg annotation tool. The annotation rules were as follows: the core area of ​​the defect was selected by a rectangular bounding box, which had to cover the entire defect area, with an error of no more than 2 pixels; the labels were named in the format of "defect type-pipe diameter-material-service years", such as "misalignment-DN300-reinforced concrete-10 years", to ensure the completeness of the label information. After annotation, the training set, validation set, and test set were randomly divided in a 7:2:1 ratio, i.e., 8400 images for training, 2400 images for validation, and 1200 images for test. During the division, the proportion of each defect type was kept balanced to avoid data distribution bias.

[0081] The training image data includes type, size, and illumination attributes. To ensure the comprehensiveness and balance of the dataset, a generation mechanism of "dual conditions + dual amplitudes + dual channels" is adopted, with specific parameter settings as follows:

[0082] Lighting attribute configuration: Two lighting conditions are preset. The first lighting condition is standard lighting: 800-1000 lux, uniform illumination; the second lighting condition is complex lighting: 300-500 lux dark areas and 1200-1500 lux bright areas coexist. The same training image is used with two lighting conditions in a time-sharing manner. The first lighting ratio is initially set to 60%, and the second lighting ratio is set to 40%. It is randomly adjusted within the preset range of 30%-70% to simulate the actual scene of uneven lighting in underground pipes.

[0083] Size attribute configuration: Select two size ranges based on the generated content conditions. The first size range is for small defects, with the defect area ranging from 50 to 200 pixels. 2 The second size range represents large defects, with the defect area ranging from 500 to 1500 pixels. 2 The first size percentage is initially set to 50%, the second size percentage is set to 50%, and then randomly adjusted within the range of 20%-80% to cover the identification needs of defects of different scales.

[0084] Type attribute configuration: Based on the generated content conditions and data capacity, two types of channels are used to generate data. The first type of channel is the actual collected image, and the second type of channel is the synthetic image (simulating the defect morphology through adversarial generative network). The proportion of the first type is initially set to 80%, and the proportion of the second type is set to 20%, which is randomly adjusted within the range of 60%-90% to supplement rare defect samples.

[0085] To improve the model's generalization ability, three basic augmentation operations—rotation, scaling, and flipping—are performed on the training set images. Advanced augmentation techniques such as brightness adjustment and noise addition are also included. Specific operation parameters and logic are as follows:

[0086] Rotation expansion: In response to the first expansion instruction, the training image data is rotated, the expansion range value and expansion amplitude value in the first expansion instruction are extracted, the number of training image data to be rotated is adjusted according to the expansion range value, and the rotation angle of the training image data is adjusted according to the expansion amplitude value.

[0087] The formula for the number of rotated samples and the expansion range value is: Number of rotated samples N = Expansion range value K, K∈[10,50] and is an integer; the expansion range value directly determines the total number of rotated samples. The larger the range value, the more rotated samples are generated, showing a linear positive correlation.

[0088] When the expansion range value K=10, each original image generates 10 rotation samples, covering 0°, 15°, 30°...135°, 10 angles spaced 15° apart;

[0089] When the expansion range value K=30, 30 rotation samples are generated for each original image, covering 0°, 15°, 30°...435°, 30 angles spaced 15° apart. After exceeding 360°, the images are looped to ensure full angle coverage.

[0090] When the expansion range value K=50, 50 rotation samples are generated for each original image, covering 0°, 15°, 30°...735°, 50 angles spaced 15° apart, and the full-angle adaptability is enhanced through multiple rounds of iteration.

[0091] The adjustment formulas for rotation angle and expansion amplitude are as follows: Rotation angle sequence θ={0°,15°,30°,...,A}, where A is the expansion amplitude value, A∈[0°,360°], and the step size is fixed at 15°; the expansion amplitude value determines the maximum coverage range of the rotation angle. The larger the amplitude value, the more comprehensive the angle coverage, and they are positively correlated.

[0092] When the expansion amplitude value A=90°, the angle sequence is 0°, 15°, 30°, 45°, 60°, 75°, 90°, covering the core angle range of 0°-90°;

[0093] When the expansion amplitude value A=180°, the angle sequence is 0°, 15°, 30°...180°, covering the semi-circular angle;

[0094] When the expansion amplitude value A=360°, the angle sequence is 0°, 15°, 30°...345°, 360°, achieving full 360° coverage without blind spots, adapting to the identification scenario of pipeline defects in any orientation.

[0095] Scaling and Augmentation: In response to the second augmentation instruction, the training image data is scaled. The augmentation range value and augmentation magnitude value in the second augmentation instruction are extracted. The amount of training image data scaled is adjusted according to the positive correlation of the augmentation range value, and the scaling factor of the training image data is adjusted according to the positive correlation of the augmentation magnitude value.

[0096] The formula for adjusting the number of scaled samples and the expansion range value is: the number of scaled samples M = the expansion range value L, where L ∈ [8, 40] and is an integer; the expansion range value directly determines the total number of scaled samples. The larger the range value, the more samples of different scales are generated, showing a linear positive correlation.

[0097] When the expansion range value L=8, each original image generates 8 scaled samples, covering 0.5x, 0.7x, 0.9x, 1.1x, 1.3x, 1.5x, 1.7x, and 1.9x, with 8 scaling factors spaced 0.2x apart;

[0098] When the expansion range value L=20, 20 scaled samples are generated for each original image, covering 0.5x, 0.6x, 0.7x...2.0x, with 20 scaling factors spaced 0.1x apart;

[0099] When the expansion range value L=40, 40 scaled samples are generated for each original image, covering 0.5x, 0.538x, 0.577x...2.0x, and the scale adaptability is enhanced by the encrypted interval.

[0100] The formulas for adjusting the scaling factor and expansion amplitude are as follows: The scaling factor sequence S = {0.5, 0.6, 0.7, ..., B}, where B is the expansion amplitude value, B ∈ [0.5, 2.0], with a fixed step size of 0.1. The expansion amplitude value determines the maximum coverage range of the scaling factor; the larger the amplitude value, the wider the scale coverage, showing a positive correlation. Simultaneously, defect size optimization is combined: small defects (pixel area 50-200px)... 2 In the sequence, prioritize selection of multipliers of 1.5-2.0 for large-size defects (pixel area 500-1500px). 2 Preferably select a multiplier of 0.5-1.0.

[0101] When the expansion amplitude value B=1.0, the scaling factor sequence is 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, which is mainly adapted to the scale reduction training of large-sized defects;

[0102] When the expansion value B=1.5, the scaling factor sequence is 0.5, 0.6...1.5, which takes into account both the reduction of large-sized defects and the moderate amplification of small-sized defects;

[0103] When the expansion amplitude value B=2.0, the scaling factor sequence is 0.5, 0.6...2.0, which focuses on adapting to the magnification scale training of small-sized defects to ensure that the features of minute defects (such as fine cracks) are fully captured.

[0104] Flip and Expand: In response to the third expansion command, the expansion range is set to 5-20 images per original image. The expansion amplitude value corresponds to three directions: horizontal flip, vertical flip, and diagonal flip. The flip direction is selected according to the expansion amplitude value. For example, when the amplitude value is 1, horizontal flip is performed; when the amplitude value is 2, vertical flip is performed; and when the amplitude value is 3, diagonal flip is performed. At least 3 flip samples in 3 directions are generated for each original image.

[0105] Advanced expansion: Add brightness adjustment (brightness variation range of -30 to +30) and noise addition operations, where the mean of Gaussian noise is set to 0 and the variance is set to 0.01-0.05, and the density of salt-and-pepper noise is set to 0.01-0.03, to simulate different light fluctuations and sensor noise interference, further enriching the diversity of the dataset.

[0106] Based on the expanded training dataset, which was ultimately expanded to 100,000 samples, the first and second deep learning models were trained. The specific configurations of the two models are as follows:

[0107] The first deep learning model (main detection model) uses YOLOv8 as its basic architecture, with CSPDarknet53 as the backbone network. The detection head consists of three detection branches at different scales, corresponding to 8x8, 16x16, and 32x32 feature maps, respectively. The input image size is uniformly adjusted to 640x640 pixels, the batch size is set to 16, the initial learning rate is 0.001, and a cosine annealing learning rate scheduling strategy is adopted. The training iterations are 300 rounds. The loss function is CIoU Loss, which is used to optimize the bounding box regression accuracy.

[0108] The second deep learning model (validation model) uses EfficientDet-D2 as its basic architecture, with EfficientNet-B2 as the backbone network. The feature pyramid network (FPN) covers a scale from 4x4 to 256x256 pixels. The input image size is 512x512 pixels, the batch size is set to 8, the initial learning rate is 0.0005, and the number of training iterations is 200. The loss function is FocalLoss, which focuses on solving the problem of imbalanced samples and improving the ability to identify small defects.

[0109] Training optimization strategy: To avoid overfitting, Dropout regularization (dropoutrate=0.5) is introduced into the fully connected layers of both models, and label smoothing technology (smoothing coefficient=0.1) is used. During training, the accuracy and recall of the validation set are monitored in real time. When there is no improvement for 10 consecutive rounds, an early stopping mechanism is triggered to save the optimal model weights.

[0110] The underground pipe network images are collected by the high-definition camera of the pipeline robot. The acquisition parameters are consistent with the training data: 1080P resolution, 25fps frame rate, and 1 / 500s exposure time. After the acquisition is completed, a portion of the images is extracted from the images according to the preset extraction ratio (initially set to 20%) as new training data and added to the training set. The remaining 80% of the images are used as images to be processed and enter the preprocessing process.

[0111] The remaining images to be processed are subjected to a defect feature shifting operation. The specific logic is as follows: the defect edge contour is extracted by an edge detection algorithm (Canny operator, threshold set to 50-150), and the center coordinates (x0, y0) of the defect are calculated; the shifting distance is set according to the defect type, such as shifting a misaligned defect by 3-5 pixels along the pipe axis and a rupture defect by 2-4 pixels along the direction perpendicular to the pipe axis; after shifting, the shape and size of the defect remain unchanged, only the position is adjusted to highlight the core area of ​​the defect and reduce the interference of non-defect areas such as pipe inner wall texture and stains.

[0112] Lighting type detection and classification adjustment: A lighting extraction algorithm (based on grayscale histogram statistics) is used to obtain the lighting type of underground pipe network images, specifically divided into four categories: uniform lighting, dark-dominated, bright-dominated, and alternating light and dark. The number of lighting types is calculated. If the number is less than the preset lighting reference number (set to 4 categories), the number of lighting attribute classifications is adjusted according to the negative correlation between the lighting type ratio (actual number / reference number). The smaller the lighting type ratio, the more uniform the actual lighting type, and the fewer the adjusted lighting attribute classifications, ensuring that the classification accurately matches the lighting diversity of the actual scene and avoiding redundant classification.

[0113] For example, when inspecting straight sections of urban sewage pipes, only three types of illumination are actually detected: uniform illumination, predominantly dark areas, and alternating light and dark areas, due to the lack of strong reflective surfaces on the inner wall of the pipe. At this time, the ratio of illumination types is 3 / 4 = 0.75. Based on the negative correlation adjustment logic, the number of illumination attribute classifications is adjusted to 3, eliminating the redundant classification of predominantly bright areas. In the inspection of old underground rainwater pipes, the inner wall of the pipe is severely corroded and has strong light absorption, so only two types of illumination, predominantly dark areas and alternating light and dark areas, can be detected, with a ratio of 2 / 4 = 0.5. The number of classifications is further simplified to 2, focusing on the low-illumination-related types that actually exist. Even in single-illumination scenarios such as enclosed underground industrial pipelines, only one type of illumination, uniform illumination, is detected, with an illumination type ratio as low as 0.25. Based on the negative correlation adjustment principle, 2 classifications are still retained: uniform illumination and the predominantly dark area type generated through data augmentation. This not only fits the actual scenario but also supplements the model with necessary illumination variation samples to avoid overfitting.

[0114] Illumination compensation preprocessing: Adaptive histogram equalization (CLAHE) is applied to images dominated by dark areas, with a block size of 8x8 pixels and a contrast limit of 2.0 to improve the grayscale dynamic range of defects in dark areas; contrast-limited histogram equalization is applied to images dominated by bright areas to compress the grayscale values ​​of bright areas to the range of 180-220, avoiding overexposure that could lead to loss of defect features; ordinary histogram equalization is applied to images with uniform illumination and alternating light and dark areas, with a grayscale range of 0-255 to optimize the overall contrast.

[0115] Lighting robustness model training: Train a dedicated lighting robustness feature extractor (based on ResNet50 architecture, fine-tuned after pre-training on the ImageNet dataset) and classifier (support vector machine with RBF kernel function) for the object detection task; embed the feature extractor into the backbone network of the dual deep learning model, and use the classifier for secondary determination of defect type to enhance the model's feature recognition ability in complex lighting environments and solve the problem of false detection of misaligned defects caused by lighting interference.

[0116] Image sharpness type detection and classification adjustment: A feature extraction algorithm (based on variance calculation using the Laplacian operator) is used to obtain image sharpness types, which are divided into three categories: sharp (variance ≥ 200), moderately sharp (100 ≤ variance < 200), and blurry (variance < 100). The number of sharpness types is calculated. If the number is less than the preset sharpness reference number (set to 3 categories), the number of size attribute categories is adjusted according to the negative correlation between the sharpness type ratio and the actual image sharpness type. The smaller the ratio, the more uniform the image sharpness type in the actual scene, and the fewer the adjusted size attribute categories. Conversely, a larger ratio retains a more comprehensive range of size categories. The purpose is to avoid data distribution imbalance caused by redundant classification, ensure that the size classification accurately matches the actual sharpness situation, and provide more targeted training data for the model.

[0117] In the inspection of newly built integrated utility tunnels, if the images acquired by CCTV technology are minimally affected by environmental interference, the inner walls of the pipes are clean and unobstructed, and the images cover three clarity types: clear (variance = 280), medium clear (variance = 150), and blurry (variance = 80), then the actual number of clarity types is 3, and the clarity type ratio is 3 / 3 = 1. According to the negative correlation adjustment logic, the number of size attribute categories is maintained at 3 (small size defect, medium size defect, and large size defect), which adapts to the recognition needs of defects of different scales in full-clarity scenarios and helps the model learn the characteristics of defects of different sizes in images of various clarity. During the inspection of old urban sewage pipes, the inner walls of the pipes were severely corroded and covered with silt. The collected images could only detect two types of sharpness: medium clarity (variance = 120) and blurry (variance = 60), with an actual number of two and a ratio of approximately 0.67. In this case, the number of size attribute classifications was adjusted to two (small and medium-sized defects, large-sized defects), eliminating redundant classifications that did not match the actual scene. This allowed the training data to focus on the core size defects in low-resolution scenes, improving the model's adaptability. Even in single-resolution scenes such as enclosed underground industrial pipelines, where fixed laser illumination throughout ensured that all images were sharp (variance = 320), the actual number of sharpness types was only one, with a ratio of only 0.33. Based on the negative correlation adjustment principle, two size classifications (small-sized defects, large-sized defects) were still retained. This not only fits the single-resolution actual scene but also preserves the dimension of size difference through classification. At the same time, data augmentation techniques were used to generate simulated medium-resolution samples to avoid overfitting due to single-resolution issues and ensure robustness in identifying defects of different sizes.

[0118] Noise suppression preprocessing: Images containing salt-and-pepper noise are processed using a 3x3 window median filter. If the noise density is greater than 0.02, a 5x5 window is used. Images containing Gaussian noise are processed using a 3x3 kernel Gaussian filter with a standard deviation of 0.8-1.2, dynamically adjusted according to the noise intensity. Blurred images are processed using a nonlocal mean denoising algorithm with a search window of 7x7 and a similarity window of 5x5, improving image clarity while preserving defect details.

[0119] Noise robustness model training: A noise robustness feature extractor based on wavelet transform is trained. The image is decomposed into low-frequency and high-frequency components using the db4 wavelet. Defect texture features are extracted from the high-frequency components to filter out noise interference. This feature extractor is combined with an illumination robustness classifier and integrated into the training of a dual deep learning model. L2 regularization (regularization coefficient = 0.001) is used to avoid overfitting and further solve the problem of false detection caused by noise interference.

[0120] The preprocessed input data (defect feature shifting + illumination compensation + noise suppression) is input into the first deep learning model. The model outputs the defect type, bounding box coordinates, and recognition confidence (the confidence threshold is initially set to 0.5). The output data is marked on the original underground pipeline image according to the bounding box coordinates. The marking colors are distinguished according to the defect type: red for misalignment, blue for rupture, yellow for aging, green for wear, and purple for corrosion. The confidence value is marked in the upper right corner of the bounding box, presenting the recognition result intuitively.

[0121] Verification cycle setting: The preset verification cycle is every 100 images processed, that is, after the inference and annotation of every 100 images are completed, the verification process of the second deep learning model is started.

[0122] Consistency calculation: Input the input data of these 100 images into the second deep learning model to obtain validation data; calculate the consistency value between the validation data and the output data, specifically including defect type consistency (number of type matches / total number of defects), bounding box overlap rate consistency (number of bounding boxes with IOU ≥ 0.5 / total number of bounding boxes), and confidence consistency (number of confidence differences ≤ 0.1 / total number of defects). The three are weighted and summed to obtain the comprehensive consistency value, with weights of 0.4, 0.4, and 0.2 respectively.

[0123] Extraction ratio adjustment: The preset consistency reference value is 0.85 (set based on the historical best validation results). The consistency ratio is calculated as consistency value / consistency reference value. If the consistency ratio is ≥1.0, it indicates that the model performance is stable, and the extraction ratio is reduced by 5% (minimum to 10%). If 0.8≤consistency ratio<1.0, the extraction ratio remains unchanged. If the consistency ratio<0.8, it indicates that the model is not adaptable to the current scenario, and the extraction ratio is increased by 10% (maximum to 50%) to supplement more training data to optimize the model.

[0124] To further overcome the missed detections caused by changes in lighting and the missed detection of cracks and defects, the image enhancement and feature fusion processes are strengthened. The specific operations are as follows:

[0125] Regional adaptive image enhancement: The image is divided into dark areas (grayscale value < 80), bright areas (grayscale value > 200), and transition areas (80 ≤ grayscale value ≤ 200) according to the illumination intensity. Adaptive histogram equalization (block size 6x6) is used in dark areas, contrast-limited histogram equalization (contrast limit 1.5) is used in bright areas, and linear grayscale adjustment is used in transition areas to maintain a smooth grayscale gradient and avoid sudden changes in brightness between areas.

[0126] Illumination-invariant feature extraction: The frequency domain-based method uses Fourier transform to convert the image to the frequency domain and extracts the frequency domain amplitude features (ignoring phase information and reducing the influence of illumination); the color space-based method converts the RGB image to the HSV color space and extracts the color component features of the Hue channel (color information) and the Saturation channel (saturation information). At the same time, it calculates the gray-level invariant moments (the first four moments), color moments (mean, variance, and slope of the three HSV channels), and texture features (energy, entropy, and contrast based on the gray-level co-occurrence matrix) to ensure that the features are insensitive to illumination changes.

[0127] Model optimization and multimodal fusion: RetinaNet with a Feature Pyramid Network (FPN) was selected as the auxiliary deep learning model. The FPN scale covered 8x8 to 1024x1024 pixels, and pre-trained weights from the ImageNet dataset were loaded. Fine-tuning was performed through transfer learning (learning rate 0.0001, 100 iterations). Generative Adversarial Network (GAN) was combined to generate defect samples simulating complex lighting conditions (such as dark area cracks and bright area corrosion) to expand the training data. Infrared images (wavelength range 8-14μm, temperature resolution 0.1℃) and depth images (accuracy ±2mm, point cloud density 1000 points / cm) were acquired from the same viewpoint as the visible light images. 2 The SIFT feature matching algorithm is used to complete multimodal data registration and extract depth features (defect depth, protrusion height) and temperature features (temperature difference between corroded and normal areas). The illumination-invariant features and multimodal features are concatenated through a convolutional layer (feature map dimensions are superimposed) and weighted summation (illumination-invariant feature weight 0.6, multimodal feature weight 0.4) to obtain fused features. These are input into the optimized model and output the defect type, recognition confidence (threshold adjusted to 0.45), and the approximate area in the image (error ≤ 5 pixels), effectively reducing the risk of missed detection caused by illumination factors.

[0128] To address the adaptation needs of different pipe conditions (aging, wear, corrosion), pipe diameters, and materials, a dataset expansion and model adaptation strategy is designed:

[0129] Multi-scenario dataset construction: Collect defect images covering different working conditions, including cracks in aging pipes, scratches on the inner wall of worn pipes, pitting corrosion and large-area corrosion in corroded pipes, covering pipe diameters of DN300-DN800mm, reinforced concrete pipes / steel pipes / plastic pipes, and pipe scenarios with service life of 3-30 years, and construct an initial training dataset (15,000 images).

[0130] Diverse data augmentation: The initial dataset was expanded to 80,000 target training images using image transformation (rotation, flipping, scaling, cropping, brightness adjustment) and noise addition (Gaussian noise, salt-and-pepper noise). Specifically, scaling (0.8-1.2 times) and brightness adjustment (-20 to +20) were applied to the aging cracks, rotation (0-360 degrees) and cropping (preserving the scratch area) were applied to the wear scratches, and noise addition (Gaussian noise variance 0.03-0.05) and flipping were applied to the corrosion defects to simulate the changes in defect morphology under different working conditions.

[0131] Model training optimization: The deep learning model (YOLOv8) for defect identification is iteratively trained based on the target training dataset. During training, L1 regularization (coefficient 0.0005), L2 regularization (coefficient 0.001), and Dropout regularization (dropoutrate=0.4) are introduced. Regularization terms are introduced during the backpropagation of the model to suppress overfitting. An ensemble learning method is adopted, which integrates random forest (100 decision trees), gradient boosting tree (XGBoost, learning rate 0.1), and multi-model voting ensemble (YOLOv8, EfficientDet-D2, and RetinaNet weights 0.33 each). The final identification result is determined by the majority voting principle, thereby improving the model's generalization ability.

[0132] Dynamic parameter adaptation: A mapping table is preset between pipeline parameter combinations and algorithm thresholds and recognition accuracy. Optimal threshold ranges and accuracy control ranges are pre-stored for different pipeline types (sewage pipes, rainwater pipes, industrial drainage pipes), pipe materials (reinforced concrete pipes, steel pipes, plastic pipes), and pipeline service life (3-10 years, 11-20 years, 21-30 years). For example, the algorithm threshold for the combination "sewage pipe-reinforced concrete pipe-21-30 years" is 0.4, and the recognition accuracy control range is ≥90%. Pipeline parameters from actual detection scenarios are obtained, and the target algorithm threshold and target recognition accuracy are determined by querying the mapping table. The corresponding model parameters are adjusted through the model parameter configuration interface (RESTful API) to complete model adaptation, ensuring stable recognition performance under different working conditions.

[0133] For scenarios involving overlapping defects such as misalignment and breakage, a multi-scale analysis and cross-domain knowledge fusion identification process is designed:

[0134] Multi-scale image processing: Acquire images of underground pipe networks with complex overlapping defects, and use a multi-scale analysis algorithm (Gaussian pyramid decomposition) to perform multi-scale decomposition and feature enhancement on the images. The preset small scale is 8x8 pixels, the medium scale is 32x32 pixels, and the large scale is 128x128 pixels. For each scale of the image, edge enhancement (Sobel operator) and threshold segmentation (Otsu algorithm) are performed, and defect edge contour features (such as the continuity of cracks and the gap width of misalignments), gray-level distribution features (such as the gray-level mean and variance of the corrosion area) and texture features (such as the texture direction of the wear area) are extracted in sequence.

[0135] Multi-scale feature fusion: A neural network with a built-in feature fusion module (based on the ResNet101 architecture) is selected as the deep learning model, and the defect discrimination features of small, medium and large scales are input into the model; the feature fusion module performs weighted fusion of features of different scales, with the weights allocated as 0.3 for small scale, 0.4 for medium scale and 0.3 for large scale, highlighting the core role of medium scale features; during the fusion process, a focus attention mechanism (CBAM) is used to automatically focus on key features of overlapping areas (such as the boundary differences between adjacent defects) and suppress redundant information.

[0136] Cross-domain knowledge fusion: When the model classifies multi-scale fused features, it incorporates transfer learning and cross-domain defect recognition knowledge; the transfer learning uses model weights pre-trained on metal structure corrosion detection datasets and non-metallic pipe wear detection datasets, which are then fine-tuned and transferred to the underground pipe network defect recognition task; the cross-domain knowledge specifically includes the "corrosion area ratio threshold (≥5% is judged as corrosion)" for metal structure corrosion defects and the "scratch depth threshold (≥1mm is judged as wear)" for non-metallic pipe wear defects, which help the model distinguish the types and boundaries of overlapping defects.

[0137] Final recognition output: Through the synergy of deep learning models and multi-scale analysis algorithms, the recognition of overlapping defects in complex shapes and defects at different scales is completed, and the defect type (such as "misalignment + crack overlap") and the corresponding region coordinates of the defect in the image (boundary box coordinate error ≤ 3 pixels) are output, successfully solving the defect overlap problem shown in the figure.

[0138] The defect location process has undergone multiple optimization steps to ensure accuracy and reliability. The specific steps are as follows:

[0139] Preprocessing for location: Obtain the underground pipeline network image to be located for defect location. First, use a 3x3 window median filter to remove salt-and-pepper noise, and then use a 3x3 kernel Gaussian filter with a standard deviation of 1.0 to remove Gaussian noise, thus completing the denoising preprocessing. Then, perform grayscale normalization (0-255) and histogram equalization on the denoised image to improve the contrast between the defect and the background.

[0140] Defect candidate feature extraction: Three neural network algorithms with high localization accuracy, Faster R-CNN, YOLOv5, and SSD, are used. The specific configurations are as follows: the anchor box size of Faster R-CNN is set to 128x128, 256x256, and 512x512, and the sliding window stride of the Region Proposal Module (RPN) is set to 16 pixels; YOLOv5 has 3 detector heads and 9 anchor boxes; the feature map scale of SSD is 8x8 to 1024x1024, and the default number of boxes in each feature map is 6. Through the region proposal module or anchor box mechanism of the above algorithms, defect candidate features containing clear location information (pixel coordinates x, y and region aspect ratio w / h) are extracted, and the number of candidate features is controlled between 50-200 per image.

[0141] Candidate Feature Optimization and Screening: Candidate features are optimized and screened using contextual information about underground pipeline defects. This contextual information includes the gray-level difference threshold between adjacent pixels (set to 30) and connectivity criteria for adjacent regions (connected region area ≥ 20px). 2 ); Specific operations: Calculate the grayscale difference between the candidate defect region and its neighboring pixels, retain features with a grayscale difference greater than 30, and eliminate pseudo-defects with small grayscale differences; through region connectivity analysis, eliminate defects with an area less than 20px. 2 Isolated pseudo-defect features were identified, and the number of candidate features was reduced to 20-50 per image after screening.

[0142] Collaborative localization: Image segmentation and object detection are introduced as auxiliary methods. The U-Net semantic segmentation algorithm (VGG16 encoder and deconvolution layer decoder) is used to segment the pipeline region where the defect is located, eliminating background interference such as soil and water accumulation. The object detection method (EfficientDet-D1) is used to perform secondary bounding box selection on the segmented pipeline region to obtain the preliminary bounding box of the defect. The bounding box overlap ratio (IOU) threshold is set to 0.3 to avoid repeated bounding box selection.

[0143] Dynamic adaptation of positioning strategies: A pre-set defect positioning strategy library contains positioning strategies and threshold ranges corresponding to different scenarios (pipe bends, straight pipe sections, and interfaces) and different defect types (pitting corrosion, cracks, and large-area corrosion). For example, a dense anchor frame strategy (anchor frame density is 1.5 times that of straight pipe sections) is used for pipe bends, long strip anchor frames (width-to-height ratio 1:5) are used for crack defects, and square anchor frames (width-to-height ratio 1:1) are used for large-area corrosion. Based on the actual detection scenario and defect type, the target positioning strategy and target threshold are matched from the strategy library, and the recognition strategy and threshold are adjusted through the model parameter configuration interface.

[0144] Final localization output: Based on the optimized defect features, the output results of the auxiliary method, and the adjusted policy threshold, non-maximum suppression (Soft-NMS, confidence decay coefficient 0.5) is used to filter the final bounding box, and the precise pixel coordinates of the defect in the image (top left corner X / Y, bottom right corner X / Y, error ≤ 2 pixels) and the localization deviation range (≤ 3 pixels) are output, which effectively solves the localization overlap problem caused by the complex shape of the defect and reduces background interference and localization error.

[0145] Through the detailed implementation process described above, the visual identification method for underground pipeline defects proposed in this application can achieve intelligent and high-precision detection of pipeline defects, reduce false detection rate, reduce missed detection rate, achieve pixel-level positioning accuracy, significantly reduce manual input, and provide reliable data support for the operation, maintenance and repair of underground pipelines.

[0146] This application also discloses a visual identification system for underground pipeline defects, including a processor, wherein the processor executes the steps of the visual identification method for underground pipeline defects as described in any of the above embodiments.

[0147] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A visual identification method for defects in underground pipe networks, characterized in that, Includes the following steps: Obtain the preset pipeline defect type, collect the corresponding training image data based on the pipeline defect type, expand the training image data, and train the preset first deep learning model and second deep learning model based on target detection based on the expanded training image data. Acquire underground pipe network images, extract a portion of the underground pipe network images according to a preset extraction ratio as training data; preprocess the other portion of the underground pipe network images to obtain input data, the preprocessing is used to shift the defect features in the underground pipe network images; Input data is fed into the first deep learning model to obtain output data, and the output data is marked on the underground pipe network image; Based on a preset verification period, the input data is fed into the second deep learning model to obtain verification data, and the consistency value between the verification data and the output data is calculated. The consistency ratio is calculated based on the consistency value and the preset consistency reference value, and the extraction ratio is adjusted according to the negative correlation of the consistency ratio. An adaptive image enhancement algorithm is used to preprocess the acquired underground pipe network images by region. The adaptive image enhancement algorithm includes histogram equalization. The processing process is as follows: the image is divided into dark areas, bright areas and transition areas according to the light intensity. Adaptive histogram equalization is used to improve the gray-scale dynamic range in the dark areas. Contrast-limited histogram equalization is used to avoid overexposure in the bright areas. The gray-scale gradient is kept smooth in the transition areas. Based on the preprocessed underground pipe network image, the illumination-invariant feature extraction method is used to extract defect features. The illumination-invariant feature extraction method includes: a frequency domain-based method using Fourier transform and wavelet transform to extract the frequency domain amplitude features of the defects; and a color space-based method using HSV color space and Lab color space to extract the color component features of the defects. The defect features include gray-level invariant moments, color moments, and texture features. A deep learning model containing a retinal network is selected, wherein the retinal network is configured with a feature pyramid network whose scale covers a set pixel range; pre-trained weights from the ImageNet dataset are loaded using a pre-training method, and the model is fine-tuned through transfer learning; and a generative adversarial network is combined to generate defect samples simulating complex lighting conditions to optimize the training process of the deep learning model. Multimodal data from the same viewpoint as visible light images of underground pipe networks are collected. The multimodal data includes infrared images and depth images. After registration processing of the multimodal data, depth features and temperature features are extracted. The extracted defect features are fused with multimodal features at the feature level. Specifically, the feature maps are concatenated and weighted summed through convolutional layers to obtain the fused features. The fused features are input into the optimized deep learning model to complete the identification of underground pipe network defects, and output the defect type, identification confidence level and the approximate area of ​​the defect in the image.

2. The visual identification method for underground pipeline defects according to claim 1, characterized in that, The step of collecting corresponding training image data based on pipeline defect types also includes the following sub-steps: Training image data has type, size, and illumination attributes; The training image data is generated based on the preset first illumination condition and the preset second illumination condition as the content conditions for the corresponding illumination attributes; wherein, the same training image data is divided into the first illumination condition and the second illumination condition at different times, the proportion of the first illumination condition is called the first illumination proportion, the proportion of the second illumination condition is called the second illumination proportion, and the sum of the illumination proportions is one. Based on the generated content conditions, a preset first size range and a preset second size range are selected as the data capacity of the training image data for the corresponding size attributes; wherein, the proportion of the first size range is selected as the first size proportion, the proportion of the second size range is selected as the second size proportion, and the sum of the size proportions is one. Based on the generated content conditions and data capacity, training image data with corresponding type attributes are generated using a preset first type channel and a preset second type channel; wherein, the proportion using the first type channel is the first type proportion, the proportion using the second type channel is the second type proportion, and the sum of the type proportions is one. The first illumination ratio, the first size ratio, and the first type ratio are randomly adjusted within a preset ratio range.

3. The visual identification method for underground pipeline network defects according to claim 1, characterized in that, The step of augmenting the training image data also includes the following sub-steps: Obtain a first expansion instruction, rotate the training image data in response to the first expansion instruction, extract the expansion range value and expansion amplitude value from the first expansion instruction, adjust the number of training image data to be rotated according to the positive correlation of the expansion range value, and adjust the rotation angle of the training image data according to the positive correlation of the expansion amplitude value. Obtain the second expansion instruction, scale the training image data in response to the second expansion instruction, extract the expansion range value and expansion magnitude value in the second expansion instruction, adjust the amount of scaled training image data according to the positive correlation of the expansion range value, and adjust the scaling factor of the training image data according to the positive correlation of the expansion magnitude value. Obtain the third expansion instruction, and in response to the third expansion instruction, flip the training image data. Extract the expansion range value and expansion amplitude value from the third expansion instruction, adjust the number of training image data to be flipped according to the positive correlation of the expansion range value, and select the flipping direction of the training image data according to the expansion amplitude value.

4. The method for visual identification of defects in underground pipe networks according to claim 1 or 2, characterized in that, The method also includes the following steps: The illumination types in the underground pipe network image are obtained based on a preset illumination extraction algorithm, and the number of illumination types is calculated. If the number of lighting types is less than the preset number of lighting references, the ratio of lighting types is calculated based on the number of lighting types to the number of lighting references, and the number of lighting type categories in the lighting attributes is adjusted according to the negative correlation of the ratio of lighting types. The preprocessing steps also include the following sub-steps: The image of the underground pipe network is preprocessed using an illumination compensation algorithm, which includes histogram equalization or adaptive histogram equalization. An illumination-robust feature extractor and classifier are trained for object detection, and the trained feature extractor and classifier are applied to the first deep learning model and the second deep learning model.

5. The method for visual identification of defects in underground pipe networks according to claim 1 or 2, characterized in that, The method also includes the following steps: The image sharpness type of the underground pipe network is obtained based on a preset feature extraction algorithm, and the number of sharpness types is calculated. If the number of sharpness types is less than the preset number of sharpness references, the ratio of sharpness types is calculated based on the number of sharpness types to the number of sharpness references, and the number of size type categories in the size attribute is adjusted negatively based on the ratio of sharpness types. The preprocessing steps also include the following sub-steps: Image preprocessing is performed using noise suppression algorithms, including median filtering or Gaussian filtering. A noise-robust feature extractor and classifier are trained for object detection, and the trained feature extractor and classifier are applied to the first deep learning model and the second deep learning model.

6. The method for visual identification of defects in underground pipe networks according to claim 1 or 3, characterized in that, It also includes the following steps: Collect defect images of underground pipe networks. The defect images cover scenarios where the pipes are aged, worn, or corroded, and include underground pipe network images of different pipe diameters, laying environments, and service stages. Construct an initial training dataset based on the defect images. The initial training dataset is augmented using data augmentation algorithms to obtain the target training dataset. The data augmentation algorithms include image transformation and noise addition. Image transformation includes rotation, flipping, scaling, cropping, and brightness adjustment. Noise addition includes Gaussian noise addition and salt-and-pepper noise addition. Based on the target training dataset, a deep learning model for defect identification is iteratively trained. During the training process, regularization algorithms and ensemble learning are used to optimize the model training process. The regularization algorithms include L1 regularization, L2 regularization, and Dropout regularization. The ensemble learning includes random forest ensemble, gradient boosting tree ensemble, and multi-model voting ensemble. The optimization operation specifically involves introducing a regularization term to suppress overfitting during the model backpropagation process and fusing the output results of multiple base models through ensemble learning. A mapping table is provided between preset pipeline parameter combinations and algorithm thresholds and recognition accuracy. The mapping table stores the optimal threshold range and accuracy control range corresponding to different combinations of pipeline types, pipeline materials, and pipeline service life. Obtain the pipeline parameters of the actual detection scenario, including pipeline type, pipeline material, and pipeline service life. Query the mapping table based on the pipeline parameters to determine the target algorithm threshold and target recognition accuracy. By using the model parameter configuration interface, the corresponding parameters of the deep learning model are adjusted according to the target algorithm threshold and target recognition accuracy to complete model adaptation.

7. The method for visual identification of defects in underground pipe networks according to claim 1, characterized in that, It also includes the following steps: Acquire images of underground pipe networks containing complex overlapping defects; The underground pipe network image is decomposed and enhanced using a multi-scale analysis algorithm. Defect distinguishing features are extracted sequentially at preset small, medium, and large scales. The defect distinguishing features include defect edge contour features, gray-scale distribution features, and texture features. The preset neural network and its derived models are used as deep learning models, and the deep learning models have a built-in feature fusion module. The defect distinguishing features at small, medium, and large scales are input into the deep learning model, and the feature fusion module performs weighted fusion of the defect distinguishing features at different scales to obtain multi-scale fused features. In the process of classifying multi-scale fused features in the deep learning model, transfer learning and defect identification knowledge from other fields are incorporated. The defect identification knowledge from other fields includes knowledge of metal structure corrosion defect identification and knowledge of non-metallic pipe wear defect identification. Through the synergistic effect of the deep learning model and the multi-scale analysis algorithm, the system can identify the overlapping defects of complex shapes and defects of different scales in underground pipe networks, and output the defect type and the corresponding regional coordinates of the defect in the image.

8. The method for visual identification of defects in underground pipe networks according to claim 1, characterized in that, The defect localization process includes the following steps: Acquire images of the underground pipe network to be located for defects, and perform noise reduction preprocessing on the underground pipe network images, using median filtering to remove salt-and-pepper noise and Gaussian filtering to remove Gaussian noise; The preprocessed underground pipe network image is processed using a preset neural network algorithm, including Faster R-CNN, YOLOv5, and SSD. Through the network's region proposal module or anchor box mechanism, defect candidate features with clear location information are extracted. The location information includes pixel coordinates and region aspect ratio. The candidate features of the defects are optimized and screened using contextual information of underground pipeline defects. The contextual information includes the gray-level difference threshold between adjacent pixels or the connectivity judgment condition of adjacent regions. The specific steps are: calculating the gray-level difference between the candidate defect region and the neighboring pixels, and retaining features with a gray-level difference greater than a preset threshold; or, eliminating isolated pseudo-defect features through regional connectivity analysis. To facilitate collaborative localization, an auxiliary method is introduced, which includes an image segmentation method and an object detection method: a semantic segmentation algorithm is used to segment the pipe region where the defect is located to eliminate background interference; and an object detection method is used to perform secondary bounding selection on the segmented pipe region to obtain the preliminary boundary box of the defect. A preset defect location strategy library is provided, which includes location strategies and threshold ranges corresponding to different scenarios and different defect types. Based on the actual detection scenario and defect type, target localization strategies and target thresholds are matched from the strategy library, and the identification strategies and thresholds corresponding to defect localization are adjusted through the model parameter configuration interface; Based on the optimized defect features, the output of the auxiliary method, and the adjusted policy threshold, the final bounding box of the defect is determined, and the precise pixel coordinates and positioning deviation range of the defect in the image are output.

9. A visual identification system for defects in underground pipe networks, characterized in that, Includes a processor, wherein the steps of the visual identification method for underground pipeline defects as described in any one of claims 1-8 are executed.

Images