A method and system for locating a fault in an underground pipeline of an underground pipe network

Abstract

This application provides a method and system for locating faults in underground pipeline networks, comprising: acquiring visual data of the pipeline interior using a mobile device; performing environmentally adaptive processing on the visual data to obtain processed visual data; separating the target area from the processed visual data to obtain target area information; identifying and classifying anomalies in the visual data based on the target area information to obtain identified defect information; associating the position of the identified defect information in a two-dimensional image with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial location information of the defect; acquiring the geometric attribute information of the defect based on the identified defect information; and outputting a distribution representation containing the three-dimensional spatial location information and geometric attribute information of the defect. This application enables trenchless, high-precision, and intelligent detection and location of faults inside underground pipelines.

Description

Technical Field

[0001] This application relates to the field of image detection technology, and more specifically, to a method and system for locating underground pipeline faults in underground pipe networks. Background Technology

[0002] Currently, the detection and location of underground pipeline faults mainly rely on manual segmented excavation or the insertion of contact sensor probes into the pipeline. Manual excavation not only requires significant manpower and resources and is inefficient, but also causes secondary damage to the road surface and surrounding environment, disrupting the normal operation of the city. Traditional sensor probe detection methods are limited to single-point measurements, making it difficult to comprehensively and continuously obtain complete information about the pipeline's internal condition. For localized features such as cracks, their location accuracy is poor, and they cannot visually demonstrate the specific shape and size of defects. These existing detection methods generally suffer from limited detection coverage, inaccurate fault identification, low levels of intelligence, and the potential to cause other hazards, making them insufficient to meet the demands of modern large-scale pipeline systems for efficient and precise operation and maintenance management. Summary of the Invention

[0003] This application discloses a method and system for locating underground pipeline faults in underground pipe networks, in order to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, this application discloses a method for locating underground pipeline faults in an underground pipe network, comprising:

[0006] Acquire visual data of the pipeline interior using a mobile device;

[0007] The visual data is subjected to environmental adaptation processing to obtain the processed visual data;

[0008] The processed visual data is then subjected to target region separation to obtain target region information;

[0009] Based on target area information, anomalies in visual data are identified and categorized to obtain the identified defect information.

[0010] The location of the identified defect information in the two-dimensional image is correlated with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial location information of the defect.

[0011] Based on the identified defect information, the geometric attribute information of the defect is obtained;

[0012] The output includes a distribution representation of the defect's three-dimensional spatial location and geometric attribute information.

[0013] Secondly, this application also discloses an underground pipeline fault location system for underground pipe networks, comprising:

[0014] The visual data acquisition module is used to acquire visual data inside the pipeline via a mobile device;

[0015] The environmental adaptability processing module is used to perform environmental adaptability processing on visual data to obtain processed visual data.

[0016] The target region separation module is used to separate the target regions from the processed visual data to obtain target region information;

[0017] The anomaly identification and classification module is used to identify and classify anomalies in visual data based on target area information, and obtain the identified defect information.

[0018] The three-dimensional spatial location association module is used to associate the position of the identified defect information in the two-dimensional image with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial location information of the defect.

[0019] The geometric attribute acquisition module is used to acquire the geometric attribute information of defects based on the identified defect information;

[0020] The distribution representation output module is used to output a distribution representation that includes the three-dimensional spatial location information and geometric attribute information of the defects.

[0021] Compared with the prior art, this application has at least the following beneficial effects:

[0022] This application enables comprehensive and continuous monitoring of the internal condition of underground pipelines, providing precise three-dimensional spatial location and detailed geometric attributes of defects. It significantly improves the efficiency, accuracy, and intelligence of fault detection, providing strong technical support for the refined operation, maintenance, and management of urban underground pipeline networks, and avoiding potential safety accidents and economic losses. Attached Figure Description

[0023] Figure 1 A flowchart illustrating a method for locating underground pipeline faults in an underground pipe network, as provided in this application;

[0024] Figure 2 This application provides a structural schematic diagram of an underground pipeline fault location system for an underground pipe network. Detailed Implementation

[0025] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0026] like Figure 1As shown in the embodiment of this application, a method for locating underground pipeline faults in an underground pipe network includes:

[0027] Acquire visual data of the pipeline interior using a mobile device;

[0028] The visual data is subjected to environmental adaptation processing to obtain the processed visual data;

[0029] The processed visual data is then subjected to target region separation to obtain target region information;

[0030] Based on target area information, anomalies in visual data are identified and categorized to obtain the identified defect information.

[0031] The location of the identified defect information in the two-dimensional image is correlated with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial location information of the defect.

[0032] Based on the identified defect information, the geometric attribute information of the defect is obtained;

[0033] The output includes a distribution representation of the defect's three-dimensional spatial location and geometric attribute information.

[0034] To make the technical solution of this application easier and clearer to understand, the key terms and implementation environment involved will be explained in detail below.

[0035] Mobile devices refer to robots, crawlers, or drones that can move autonomously or semi-autonomously inside pipelines. Their main function is to carry visual sensors and motion sensing units to inspect and collect data inside pipelines.

[0036] Visual data typically refers to sequences of images or videos captured by cameras or other optical sensors mounted on mobile devices. This data contains visual information about the inside of pipelines, such as pipe wall conditions, connections, deposits, and potential defects.

[0037] Environmental adaptation processing refers to the preprocessing of raw visual data to eliminate or reduce the impact of complex pipeline environmental factors such as uneven lighting, water mist, dirt, and reflection on image quality, thereby ensuring the accuracy of subsequent image analysis.

[0038] Target region separation refers to identifying and extracting key regions related to the pipeline structure from the processed visual data, such as pipe walls, interfaces, and branch pipes, so that the analysis can focus on these regions.

[0039] Defect information refers to anomalies inside pipelines identified through image analysis, such as cracks, perforations, corrosion, blockages, and misalignments. This information typically includes the type, location, size, and severity of the defect.

[0040] Three-dimensional motion information refers to the three-dimensional spatial data such as the position, attitude, and trajectory of a mobile device during its movement inside a pipeline. It is usually provided by an inertial measurement unit (IMU), odometry, or SLAM (Simultaneous Localization and Mapping) system.

[0041] Geometric attribute information refers to the physical size and shape characteristics of defects, such as the length, width, and depth of cracks, the diameter of perforations, and the area of ​​corroded regions.

[0042] Distribution representation refers to presenting the three-dimensional spatial location and geometric attribute information of defects in a visual or structured data form, such as a three-dimensional point cloud map, defect distribution map, report table, etc., so that users can intuitively understand the overall health status inside the pipeline.

[0043] The implementation environment of this application is usually inside underground pipelines, which may have complex environmental conditions such as insufficient light, humidity, narrowness, and curvature. These conditions place high demands on visual data acquisition and processing.

[0044] This application provides a method for locating underground pipeline faults in underground pipe networks, the main features and implementation of which are as follows:

[0045] First, visual data about the pipeline's interior is acquired via a mobile device. This device can be equipped with a high-resolution visible light camera to continuously capture images or record video during pipeline inspections. Alternatively, the mobile device can also carry an infrared camera to acquire thermal imaging data in low-light or foggy environments. Furthermore, the mobile device can integrate a multispectral or hyperspectral imager to obtain richer information about material characteristics. For example, when inspecting metal pipelines, a visible light camera can clearly capture macroscopic defects in the pipe wall, while an infrared camera may help detect temperature anomalies that are difficult to see with the naked eye, indicating potential leaks or corrosion.

[0046] Secondly, the visual data undergoes environmentally adaptive processing to obtain processed visual data. One approach is to use image enhancement algorithms, such as histogram equalization, gamma correction, or the Retinex algorithm, to improve the brightness, contrast, and color balance of the image and eliminate the effects of uneven lighting. Another approach is to utilize dehazing algorithms or underwater image restoration techniques to remove water mist, turbidity, or reflective interference from the visual data. For example, when a mobile device operates in a pipeline filled with turbid water, the directly acquired visual data may be blurry. In this case, underwater image restoration algorithms can be applied to estimate the optical parameters of the water body and perform backscattering compensation, thereby obtaining processed visual data with significantly improved clarity.

[0047] Next, the processed visual data is used to separate the target region to obtain target region information. One approach is to use image segmentation algorithms based on edge detection or region growing to separate structural regions such as pipe walls and interfaces from the background. Another approach is to use deep learning models, such as U-Net or Mask R-CNN, to perform semantic segmentation on the pipe structure in the visual data and accurately identify the target region. For example, in the processed visual data, the Canny edge detection algorithm can be used to identify the clear outline of the pipe wall, thereby distinguishing the pipe wall region from cavities or other debris inside the pipe and obtaining accurate target region information.

[0048] Then, based on the target area information, anomalies in the visual data are identified and categorized to obtain the identified defect information. One approach is to use traditional image processing methods, such as texture analysis, shape matching, or color analysis, to identify typical defects such as cracks, perforations, and corrosion spots. Another approach is to utilize deep learning models such as convolutional neural networks (CNNs) to learn and classify image features within the target area, automatically identifying and categorizing various types of defects. For example, in the separated pipe wall target area, a deep learning model can be trained to distinguish between fine hairline cracks, circular or irregularly shaped perforations, and large areas of rust, and to label the type and confidence level of each defect.

[0049] Subsequently, the location of the identified defect information in the 2D image is correlated with the 3D motion information of the mobile device to obtain the 3D spatial location information of the defect. One implementation method is to use the mobile device's odometer data and inertial measurement unit (IMU) data to estimate its 3D pose in the pipeline through dead reckoning, and then convert the pixel coordinates of the defect in the 2D image to 3D spatial coordinates using camera intrinsic and extrinsic parameters. Another implementation method is to use visual SLAM technology to construct a 3D map inside the pipeline in real time, simultaneously locate the pose of the mobile device, and then directly map the defect information onto the 3D map. For example, when a crack is identified in a frame of an image, the 3D pose data (including position and attitude) of the mobile device at that moment can be queried based on the capture time of the frame. Then, using a camera projection model, the pixel coordinates of the crack in the image are back-projected into the 3D space inside the pipeline, thereby obtaining the precise 3D spatial location of the crack.

[0050] Next, based on the identified defect information, the geometric attribute information of the defect is obtained. One approach is to use image measurement technology to calculate the pixel size of the defect in the image and, combined with camera calibration parameters and the 3D position information of the defect, estimate the actual physical size of the defect, such as its length, width, or area. Another approach is to use 3D reconstruction technology to perform local 3D reconstruction of the defect area, directly obtaining the 3D geometric model of the defect, thereby accurately measuring its geometric attributes. For example, for an identified perforation defect, its diameter (in pixels) in a 2D image can be measured using image processing algorithms, and then, combined with the camera focal length, pixel size, and distance from the perforation to the camera, the actual physical diameter of the perforation can be calculated.

[0051] Finally, the output includes a distribution representation of the defect's 3D spatial location and geometric attributes. One approach is to output this information as a structured data table, containing fields such as defect ID, type, 3D coordinates (X, Y, Z), length, width, and area. Another approach is to generate a 3D visualization model, overlaying the 3D point cloud or mesh model inside the pipeline with the defect's 3D location and geometric attributes, and using different colors or markers to distinguish different types of defects. For example, all identified defects and their 3D location and geometric attribute information can be integrated into an interactive 3D pipeline model. Users can rotate and zoom the model to view the overall defect distribution inside the pipeline and click on a specific defect to obtain its detailed geometric attribute data.

[0052] The overall working principle of this application lies in achieving comprehensive and accurate location and assessment of internal defects in underground pipelines by integrating multi-source sensing data and advanced image processing technology. First, the mobile device continuously collects high-resolution visual data during its inspection inside the pipeline; this data forms the basis for all subsequent analyses. Due to the complex and variable environment inside pipelines, the raw visual data often suffers from noise, blurring, and uneven lighting. Therefore, environmental adaptability processing is a crucial step, ensuring the accuracy and robustness of subsequent image analysis. The processed visual data, through target region separation technology, focuses the analysis on the pipeline structure itself, avoiding interference from irrelevant background information and improving the efficiency of defect identification.

[0053] Within the target area, the anomaly identification and classification module utilizes advanced image recognition algorithms to automatically and accurately detect and classify various types of defects. This significantly reduces the burden of manual interpretation and improves detection consistency. However, simply identifying the location of a defect in a two-dimensional image is insufficient. To achieve precise fault localization, this two-dimensional information must be transferred to three-dimensional space. Therefore, associating the two-dimensional image location of the defect with the three-dimensional motion information of the moving device is a crucial step in achieving three-dimensional spatial localization. This association process requires precise pose information of the moving device to ensure that the actual spatial location of the defect inside the pipeline can be accurately calculated.

[0054] After obtaining the three-dimensional spatial location of the defect, its geometric attributes are also needed to better assess its severity and impact on pipeline operation. These geometric attributes, such as the length and width of a crack or the diameter of a perforation, provide a quantitative basis for subsequent maintenance decisions. Finally, all this information is integrated and output in a distributed representation, providing users with not only an intuitive defect distribution map but also detailed defect attribute data. This allows pipeline managers to comprehensively understand the pipeline's health status and develop scientific maintenance plans. The entire process forms a closed loop; from data acquisition to final fault location and assessment, the various modules work closely together to solve the problems of low efficiency, poor accuracy, and incomplete information inherent in traditional detection methods.

[0055] This application introduces a non-contact image detection-based method to achieve comprehensive and detailed detection of internal pipeline defects. First, by acquiring visual data of the pipeline interior using a mobile device, continuous and wide-area coverage of the pipeline's internal environment is achieved, overcoming the limitations of traditional single-point measurements. Second, environmentally adaptive processing of the visual data effectively addresses the impact of complex internal pipeline environments (such as insufficient lighting, water mist, and dirt) on image quality, ensuring the accuracy of subsequent analysis—a feat difficult to achieve with traditional methods.

[0056] Furthermore, this application, through steps such as target area separation, anomaly identification and classification, utilizes advanced image processing and deep learning technologies to automatically and accurately identify and classify various types of defects, greatly improving the intelligence and efficiency of detection. Compared with traditional manual visual inspection, this application avoids errors caused by subjective judgment and can detect minute defects that are difficult to detect with the naked eye. Most importantly, this application correlates the position of the defect in the two-dimensional image with the three-dimensional motion information of the moving device, thereby obtaining the precise three-dimensional spatial location information of the defect and further obtaining the geometric attribute information of the defect. This technological breakthrough significantly improves the positioning accuracy of defects and provides quantitative size data of defects, providing unprecedented detailed basis for pipeline maintenance and management. The final output, a distributed representation containing the three-dimensional spatial location and geometric attribute information of defects, provides pipeline managers with an intuitive and comprehensive view of pipeline health status, helping to formulate more scientific and efficient maintenance strategies. Therefore, this application represents a significant improvement and enhancement over existing technologies in terms of detection efficiency, positioning accuracy, information comprehensiveness, and intelligence level.

[0057] In some embodiments, the step of associating the location of the identified defect information in a two-dimensional image with the three-dimensional motion information of the mobile device includes:

[0058] Acquire the micro-vibration response of the pipeline structure captured by the micro-vibration sensing unit of the mobile device;

[0059] Analyze the micro-vibration response of pipeline structures to identify external vibration characteristics;

[0060] Based on the external vibration characteristics, the motion sensing data of the mobile device is adjusted to obtain the adjusted motion sensing data;

[0061] The adjusted motion perception data is fused with the visual perception data of the mobile device to obtain the three-dimensional pose of the mobile device;

[0062] Based on the three-dimensional pose of the mobile device, the position of the identified defect information in the two-dimensional image is correlated with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial position information of the defect.

[0063] Specifically, acquiring the micro-vibration response of the pipeline structure captured by the micro-vibration sensing unit of the mobile device refers to the use of micro-vibration sensing units integrated within the mobile device, such as accelerometers, gyroscopes, or specialized vibration sensors, to monitor and record in real time the micro-vibrations generated by the pipeline structure as the mobile device passes by. These vibration responses contain information about the interaction between the pipeline structure and the external environment, such as vibrations caused by external vehicle traffic or geological activity.

[0064] Analyzing the micro-vibration response of pipeline structures and identifying external vibration characteristics can be understood as extracting characteristic patterns related to external vibration sources by performing signal processing on the collected micro-vibration response data, such as time-domain or frequency-domain analysis. These characteristic patterns can help distinguish between vibrations caused by the movement of the mobile device itself and vibrations caused by the external environment.

[0065] Based on external vibration characteristics, the motion sensing data of the mobile device is adjusted to obtain adjusted motion sensing data. Specifically, once external vibration characteristics are identified, the system compensates for or corrects the original motion sensing data of the mobile device (such as inertial measurement unit (IMU) data) based on these characteristics. The purpose is to eliminate or reduce the interference of external vibration on the motion sensing data, thereby improving the accuracy of the motion sensing data.

[0066] Fusing adjusted motion-sensing data with visual perception data from the mobile device to obtain the device's 3D pose involves multi-sensor fusion of motion-sensing data (compensated for external vibration) and visual perception data acquired by the device via a camera. For example, algorithms such as Kalman filtering, extended Kalman filtering, or particle filtering can be used to combine the advantages of both data sources to obtain a more accurate and robust 3D pose (including position and orientation) for the mobile device. Visual perception data provides rich environmental texture and feature point information, while motion-sensing data provides continuous motion information; fusion of the two can effectively compensate for the limitations of a single sensor in specific environments.

[0067] Therefore, based on the three-dimensional pose of the mobile device, the position of the identified defect information in the two-dimensional image is associated with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial position information of the defect. This means that after obtaining the high-precision three-dimensional pose of the mobile device, the pixel coordinates of the defect identified in the two-dimensional image can be back-projected into the three-dimensional space through the camera intrinsic parameters and the precise three-dimensional pose of the mobile device, thereby calculating the precise three-dimensional spatial position of the defect inside the pipeline.

[0068] This application introduces a micro-vibration sensing unit to capture the minute vibration responses of pipeline structures. These micro-vibration responses are analyzed to identify external vibration characteristics, thereby distinguishing between interference caused by the external environment (such as traffic, construction, etc.) and the motion of the mobile device itself. It is precisely because these external vibrations can be identified and quantified that the system can make targeted adjustments and compensations to the raw motion sensing data of the mobile device, effectively reducing the impact of external interference on the accuracy of the motion sensing data. Based on this, the adjusted motion sensing data is fused with visual sensing data, leveraging the localization advantages of visual data in textured areas and the continuity advantages of motion data in short-term motion. Through multi-sensor fusion algorithms, such as Kalman filtering, they can mutually verify and complement each other, thus obtaining a more accurate and robust 3D pose of the mobile device than a single data source. Finally, based on this high-precision 3D pose, the defect location in the 2D image can be more accurately mapped to 3D space, significantly improving the localization accuracy of the defect's 3D spatial location.

[0069] Through the above technical solution, this application effectively overcomes the problem of decreased positioning accuracy caused by external interference to the motion sensing data of mobile devices. By introducing micro-vibration sensing and external vibration feature recognition, the system can actively identify and compensate for the influence of the external environment on the motion sensing data, making the motion sensing data purer and more accurate. Furthermore, by fusing the adjusted motion sensing data with visual sensing data, the complementary advantages of different sensors are fully utilized, significantly improving the three-dimensional pose estimation accuracy of mobile devices in complex underground pipeline environments. This high-precision three-dimensional pose is key to achieving accurate three-dimensional spatial positioning of defects, thus making the fault location results more reliable and providing a solid data foundation for subsequent repair and maintenance work.

[0070] In some embodiments, the steps of analyzing the micro-vibration response of the pipeline structure and identifying external vibration characteristics specifically include:

[0071] The micro-vibration response of the pipeline structure is segmented to obtain the segmented micro-vibration response.

[0072] Spectral analysis was performed on the segmented micro-vibration response to obtain the frequency distribution and energy spectrum.

[0073] Identify multiple energy peaks in the frequency distribution and energy spectrum;

[0074] Based on the preset external vibration characteristic information, the energy peak value is matched to obtain the matching result;

[0075] Analyze the evolution of energy peak values ​​within different time windows;

[0076] Based on the matching results and evolution patterns, external vibration characteristics are identified.

[0077] Specifically, segmenting the micro-vibration response of pipeline structures involves dividing continuously acquired micro-vibration response data into several independent short-time segments according to a preset time length or number of data points. The purpose is to decompose complex continuous signals into more easily analyzed local segments for subsequent refined processing, such as short-time Fourier transform. Spectral analysis of the segmented micro-vibration response can be understood as converting the time-domain micro-vibration response data to the frequency domain using signal processing techniques such as Fourier transform, thereby obtaining its frequency distribution and energy spectrum. The frequency distribution reveals the various frequency components and their intensities contained in the signal, while the energy spectrum quantifies the energy carried by different frequency components. Its purpose is to reveal the intrinsic frequency characteristics of the micro-vibration response, providing crucial frequency domain information for identifying external vibration sources. In practical applications, identifying multiple energy peaks in the frequency distribution and energy spectrum specifically refers to locating specific frequency points or frequency ranges in the spectral analysis results where the energy is significantly higher than the surrounding frequencies. These energy peaks typically correspond to the main vibration source or resonant frequency. Its purpose is to extract the most representative vibration features from the signal as the basis for subsequent matching with preset external vibration features.

[0078] Matching energy peaks based on pre-defined external vibration characteristics involves comparing the identified energy peaks with a pre-established database of external vibration characteristics. This database may contain typical frequency ranges, energy intensity, and durations of different external vibration sources (e.g., vehicle traffic, construction activities, pump station operation). Matching allows for a preliminary determination of whether the currently detected vibration matches a known external vibration source. Furthermore, analyzing the evolution of energy peaks within different time windows involves tracking the changes in parameters such as frequency, amplitude, and duration of specific energy peaks over time through continuous segmented processing and spectral analysis. For example, a stable external vibration source may exhibit continuous peaks with relatively constant frequency and energy, while a transient impact may manifest as a brief, high-energy peak. The aim is to further verify and differentiate external vibration characteristics through dynamic analysis, improving identification accuracy, such as distinguishing between continuous vibration and occasional noise. Finally, identifying external vibration characteristics based on the matching results and evolution patterns involves comprehensively considering the degree of matching between the energy peaks and pre-defined characteristics, as well as the dynamic characteristics of these peaks over time, to make a final judgment about the external vibration characteristics. For example, if an energy peak closely matches the preset characteristics of vehicle traffic and exhibits periodic appearance and disappearance over time, it can be identified with high confidence as an external vibration caused by vehicle traffic.

[0079] This application addresses the shortcomings of single or coarse analysis methods in identifying external vibration characteristics by introducing a series of refined steps, including segmentation processing, spectrum analysis, energy peak identification, matching with preset features, and evolution law analysis. Specifically, segmentation processing decomposes complex continuous signals into manageable short-time segments, laying the foundation for subsequent frequency domain analysis. Spectrum analysis converts time-domain signals into frequency-domain information, revealing external vibration characteristics hidden in noise. Through frequency distribution and energy spectrum, the fingerprints of different vibration sources can be intuitively observed. Identifying energy peaks is a crucial step in extracting these fingerprints, focusing on the most energetic frequency components in the signal to filter out most background noise. Subsequently, matching these identified energy peaks with preset external vibration characteristic information allows for a preliminary determination of the vibration source type. More importantly, by analyzing the evolution law of energy peaks within different time windows, the nature of the vibration source can be further verified and differentiated, such as distinguishing between persistent interference and instantaneous impact, or different types of periodic vibrations. It is precisely because of these multi-dimensional and multi-stage analyses that the identification of external vibration characteristics no longer relies solely on a single instantaneous signal feature, but comprehensively considers its frequency, energy, duration, and dynamic changes, thereby significantly improving the accuracy and reliability of identification, effectively avoiding misjudgment and omission, and providing a solid foundation for the precise adjustment of subsequent motion sensing data.

[0080] Through the above technical solution, this application enables more accurate and robust identification of external vibration characteristics in the micro-vibration response of pipeline structures. Compared to solutions that only perform simple analysis, this application, by introducing segmented processing, spectrum analysis, energy peak identification, feature matching, and evolution law analysis, can effectively filter out noise interference, accurately distinguish different types of external vibration sources, and dynamically track their changes. This refined identification capability significantly reduces the risk of misjudging or missing external vibration characteristics, thereby ensuring the accuracy of subsequent adjustments to the motion sensing data of the mobile device. As a result, the accuracy of the three-dimensional pose estimation of the mobile device is greatly improved, ultimately making the three-dimensional spatial location information of underground pipeline defects more accurate and reliable, providing more solid data support for fault location.

[0081] In some embodiments, the step of adjusting the motion sensing data of the mobile device based on external vibration characteristics to obtain adjusted motion sensing data includes:

[0082] Obtain the confidence level of external vibration characteristics;

[0083] Based on the confidence level, the method of processing the motion sensing data of the mobile device is adjusted to obtain the preliminarily adjusted motion sensing data;

[0084] Continuous monitoring of the pipeline structure's response to minute vibrations;

[0085] Based on the continuously monitored micro-vibration response of the pipeline structure, the external vibration characteristics were verified, and the verification results were obtained; and

[0086] Based on the verification results, the initially adjusted motion perception data was adjusted again to obtain the adjusted motion perception data.

[0087] Specifically, the confidence level for acquiring external vibration characteristics refers to the quantitative assessment of the reliability or certainty of external vibration characteristics identified by analyzing the minute vibration responses of pipeline structures. For example, this confidence level can be determined based on the similarity of feature matching, statistical significance, or the temporal stability of the characteristics. Adjusting the processing method of the motion sensing data of the mobile device according to the confidence level to obtain preliminarily adjusted motion sensing data can be understood as dynamically selecting or modifying the compensation algorithm or parameters of the motion sensing data based on the reliability of the external vibration characteristics. For example, when the confidence level is high, a more aggressive compensation strategy can be adopted; when the confidence level is low, a more conservative strategy may be adopted, or even no significant adjustments may be made temporarily to avoid introducing unnecessary errors.

[0088] Continuous monitoring of the micro-vibration response of pipeline structures refers to the uninterrupted collection and real-time analysis of vibration data within the pipeline using a micro-vibration sensing unit on a mobile device. The aim is to promptly capture changes in the external vibration environment. Specifically, verifying external vibration characteristics based on the continuously monitored micro-vibration response involves comparing the real-time monitored micro-vibration response with previously identified external vibration characteristics to confirm whether the characteristic still exists, whether its intensity and frequency remain consistent, or whether new external vibration characteristics have appeared. The verification result can be a Boolean value (valid / invalid) or a confidence score. Further, based on the verification result, the initially adjusted motion sensing data is readjusted to obtain adjusted motion sensing data. This involves revising the initial motion sensing data adjustment based on the verification result. For example, if the verification result indicates that the external vibration characteristic has become ineffective or has changed significantly, the previous initial adjustment can be weakened, modified, or canceled; if the verification result confirms that the external vibration characteristic is still effective, the current adjustment strategy can be maintained or further optimized.

[0089] This application addresses the limitations of adjusting motion sensing data solely based on initially identified external vibration features in complex and variable underground pipeline environments by introducing a confidence assessment and continuous verification mechanism for external vibration characteristics. Specifically, firstly, by acquiring the confidence level of external vibration features, the initial adjustment of motion sensing data can be adaptively adjusted based on the reliability of the features. Higher confidence allows for more aggressive adjustments, while lower confidence leads to a more cautious approach, avoiding over- or erroneous adjustments due to uncertain features. Secondly, by continuously monitoring the minute vibration responses of the pipeline structure, this approach can grasp changes in the environmental vibration state in real time. Based on this, the external vibration features are verified, ensuring the real-time nature and accuracy of the identified vibration characteristics. Finally, the initially adjusted motion sensing data is readjusted based on the verification results, forming a closed-loop feedback mechanism. This allows the adjustment process to dynamically adapt to environmental changes, significantly improving the accuracy and robustness of the adjustments.

[0090] Through the above technical solutions, this application can effectively improve the accuracy and reliability of motion sensing data adjustment for mobile devices. By introducing confidence assessment, blind adjustments are avoided, reducing the error risk caused by the uncertainty of external vibration feature identification. The continuous monitoring and verification mechanism ensures the real-time and adaptability of the adjustment strategy, enabling the system to dynamically respond to the complex and ever-changing vibration environment inside the pipeline. As a result, the final adjusted motion sensing data has higher accuracy, providing a more reliable input for subsequent three-dimensional pose fusion, thereby improving the overall accuracy and stability of underground pipeline fault location.

[0091] In some embodiments, the step of fusing the adjusted motion perception data with the visual perception data of the mobile device to obtain the three-dimensional pose of the mobile device includes:

[0092] To obtain the texture richness, the number of feature points in the visual perception data, and the feature point tracking stability of the visual perception data;

[0093] The reliability of visual perception data is evaluated based on texture richness, number of feature points, and feature point tracking stability.

[0094] Acquire noise levels in motion sensing data, drift trends in motion sensing data, and confidence levels of external vibration compensation;

[0095] The accuracy of motion sensing data is evaluated based on noise level, drift trend, and confidence level of external vibration compensation.

[0096] Based on the reliability of visual perception data and the accuracy of motion perception data, the fusion weights of visual perception data and motion perception data are dynamically allocated.

[0097] Based on the fusion weights, visual perception data and motion perception data are fused to obtain the three-dimensional pose of the mobile device;

[0098] Based on the three-dimensional pose, predict the visual observation and motion perception data at the next moment to obtain the predicted value;

[0099] The actual collected visual perception data and motion perception data are compared with the predicted values ​​to obtain the observation residuals;

[0100] When the observation residuals exceed a preset threshold, the fusion weight of the corresponding data source is reduced.

[0101] Among these metrics, texture richness in visual perception data refers to quantifying the detail and variability contained in an image. High texture richness typically implies more identifiable features, aiding visual localization. Feature point count refers to the number of unique points detected in a visual image that can be used for tracking and matching; a higher count generally indicates more abundant visual information. Feature point tracking stability refers to the consistency and accuracy with which feature points are successfully tracked across consecutive frames; higher stability indicates more reliable visual data. Based on these metrics, the usability and confidence of visual perception data at the current moment can be comprehensively evaluated.

[0102] The noise level in motion sensing data refers to the magnitude of random error in the output data of a motion sensor (e.g., an inertial measurement unit); lower noise indicates higher accuracy. Drift trend refers to the gradually accumulating deviation between the measured value and the true value after a motion sensor has been operating for a long time; smaller drift indicates higher accuracy. The confidence level of external vibration compensation refers to the effectiveness and accuracy of compensating for external vibrations; higher confidence level indicates less influence of external interference on the motion sensing data, resulting in higher accuracy. Based on these indicators, the accuracy of motion sensing data at the current moment can be quantified.

[0103] Dynamically allocating fusion weights for visual and motion perception data based on their reliability and accuracy means adjusting their relative weights in the fusion algorithm according to real-time evaluations. For example, when visual data has high reliability but motion data has low accuracy, the weight of visual data is increased; conversely, the weight of motion data is increased. This can be achieved through adaptive Kalman filtering, particle filtering, or learning-based fusion methods. Based on these fusion weights, the fusion of visual and motion perception data yields the 3D pose of the mobile device. This can be achieved using multi-sensor fusion algorithms, such as extended Kalman filtering, unscented Kalman filtering, or graph optimization-based methods, to combine the weighted visual and motion data and estimate the mobile device's position and orientation in 3D space.

[0104] Predicting visual observations and motion sensing data for the next moment based on 3D pose involves using the currently estimated 3D pose and motion model to predict the pose a mobile device might reach in the next moment, and thereby inferring the visual image features and motion sensor readings for that moment. The actual acquired visual and motion sensing data are compared with the predicted values ​​to obtain observation residuals, which are the differences between the actual acquired visual and motion data and the expected values ​​obtained based on the prediction model. The magnitude of the residuals reflects the consistency between the prediction model and the actual observations, as well as the quality of the data source. When the observation residuals exceed a preset threshold, the fusion weight of the corresponding data source is reduced. This means that when the observation residual of a data source (visual or motion) significantly exceeds the preset threshold, it indicates that the data source may have an anomaly or degraded in quality. In this case, its weight in the fusion should be reduced to minimize its negative impact on the overall pose estimation. This is a feedback control mechanism to ensure the robustness of the fusion process.

[0105] This application addresses the issue of insufficient pose estimation accuracy in complex and variable environments by introducing a real-time evaluation mechanism for the quality of visual and motion-sensing data. Specifically, by acquiring the texture richness, feature point quantity, and tracking stability of the visual data, the current reliability of the visual data can be comprehensively evaluated. Simultaneously, by analyzing the noise level, drift trend, and confidence level of external vibration compensation in the motion-sensing data, the real-time accuracy of the motion data can be accurately determined. Based on these real-time evaluation results, the system can dynamically adjust the weights of visual and motion-sensing data during the fusion process, ensuring that higher-quality data sources are always prioritized under different environmental conditions, thereby optimizing 3D pose estimation. Furthermore, by using the current 3D pose to predict the observation data at the next moment and comparing the actual acquired data with the predicted value to generate observation residuals, this scheme establishes an effective feedback loop. When the observation residuals exceed a preset threshold, the system can promptly identify potential anomalies or quality degradation in the data source and correspondingly reduce the fusion weight of that data source, further enhancing the adaptability and robustness of pose estimation and effectively avoiding a decrease in overall positioning accuracy due to fluctuations in the quality of a single data source.

[0106] Through the above technical solution, this application can significantly improve the accuracy and robustness of 3D pose estimation for mobile devices in complex underground pipeline environments. Compared to the above-mentioned scheme that fuses adjusted motion perception data with the visual perception data of the mobile device to obtain the 3D pose of the mobile device, this application introduces a dynamic weight allocation and a feedback adjustment mechanism based on observation residuals, enabling the system to adapt to changes in the quality of visual and motion data in real time. This not only avoids the accumulation of pose estimation errors caused by fixed fusion weights, but also reduces the impact of a decline in the quality of a certain data source in a timely manner, thereby ensuring the correlation accuracy of the 3D spatial location information of defects. This provides more reliable basic data for subsequent acquisition of defect geometric attributes and distribution representation output, thus improving the overall accuracy and reliability of underground pipeline fault location.

[0107] In some embodiments, the step of reducing the fusion weight of the corresponding data source when the observation residual exceeds a preset threshold includes:

[0108] Analyze the observation residuals to determine whether they are caused by sudden environmental changes, and obtain the judgment results.

[0109] When the judgment result is an environmental mutation, the fusion weight of the corresponding data source is maintained, the environmental adaptation processing flow is triggered, and the environmental recovery monitoring mechanism is started at the same time.

[0110] When the judgment result indicates a decline in the quality of the data source itself, the fusion weight of the corresponding data source is adjusted based on the magnitude of the observation residuals, the duration of the observation residuals, and the historical reliability of the data source.

[0111] Specifically, the observation residual refers to the value obtained by the system by comparing the predicted value with the actual collected visual perception data and motion perception data. This residual reflects the uncertainty or error in the system's estimation of the mobile device's pose at the current moment.

[0112] Analyzing observation residuals and determining whether they are caused by abrupt environmental changes aims to distinguish the root causes of increased residuals. For example, by analyzing the temporal trends and spatial distribution characteristics of observation residuals, combined with the identification of abnormal regions in visual perception data (such as drastic changes in brightness or sudden disappearance of texture) and the detection of instantaneous changes in motion perception data (such as abrupt changes in acceleration or angular velocity), a comprehensive judgment can be made as to whether the residuals are related to sudden changes in the environment.

[0113] When the judgment result indicates an environmental mutation, the fusion weight of the corresponding data source is maintained. This is to avoid incorrectly penalizing the inherent quality of the data source due to temporary environmental changes. Simultaneously, an environmental adaptation process is triggered, such as activating processing modules for specific environmental conditions like strong light, weak light, or blurriness, enabling the system to quickly adapt to new environmental conditions. Furthermore, an environmental recovery monitoring mechanism is initiated. This mechanism continuously monitors environmental characteristic values ​​to determine whether the environment has returned to a normal state, allowing for timely exit from the adaptive processing process once environmental recovery is achieved.

[0114] When the judgment indicates a decline in the quality of the data source itself, the fusion weights need to be finely adjusted based on the magnitude and duration of the observed residuals, as well as the historical reliability of the data source. A larger magnitude of the observed residuals indicates a more severe decline in data source quality, and the weight reduction should be greater; a longer duration indicates a more persistent problem, and the weight reduction should be more significant; the historical reliability of the data source serves as an important reference. For data sources with a good historical performance, a certain degree of tolerance can be given when problems arise, avoiding over-adjustment.

[0115] This application addresses the limitations of indiscriminately reducing data source fusion weights when observation residuals exceed thresholds by introducing an intelligent judgment mechanism for the causes of observation residuals. Specifically, when the system detects an increase in observation residuals, it no longer blindly reduces the data source weights but first analyzes the nature of the residuals. If the residuals are caused by sudden environmental changes, such as suddenly entering a dark area or encountering strong reflections, the data source itself may still be reliable; its output may simply become temporarily unreliable in the current environment. In this case, the data source fusion weights are maintained, and an environmental adaptive processing flow is initiated, allowing the system to proactively adjust its processing strategy to adapt to the new environment. Simultaneously, an environmental recovery monitoring mechanism ensures timely exit from adaptive processing after environmental recovery, thus avoiding misjudgment of the data source and unnecessary weight reduction. Conversely, if the residuals are determined to be caused by a decline in the quality of the data source itself, such as sensor malfunction or performance drift, the fusion weights are adjusted selectively based on the severity and duration of the residuals, as well as the historical performance of the data source, to reduce its negative impact on overall pose estimation. This differentiated processing mechanism ensures that the adjustment of fusion weights is more reasonable and accurate, thereby improving the system's robustness and adaptability in complex and ever-changing environments.

[0116] Through the above technical solution, this application can effectively distinguish whether the observation residuals are caused by sudden environmental changes or by a decline in the quality of the data source itself, thereby avoiding erroneous penalties for reliable data sources when there are temporary environmental changes. As a result, the system can more intelligently manage the data fusion process, improving the accuracy and robustness of pose estimation in complex and variable underground pipeline environments. Furthermore, by triggering environmental adaptive processing procedures and environmental recovery monitoring mechanisms, this application enables the system to possess stronger environmental adaptability, allowing it to quickly respond and restore normal operation, significantly improving the reliability and efficiency of underground pipeline fault location.

[0117] In some embodiments, the step of determining whether the above-mentioned observation residual is caused by a sudden change in the environment and obtaining the determination result includes:

[0118] By analyzing the temporal trend and spatial distribution characteristics of the observation residuals, the trend and spatial distribution characteristics of the observation residuals are obtained.

[0119] Abnormal region identification is performed on visual perception data to determine whether there are regions in the visual perception data that do not conform to the pipeline structure, have specific shapes or brightness changes, and obtain abnormal region information of visual perception data.

[0120] Instantaneous change detection is performed on motion sensing data to determine whether there are sudden changes in acceleration or angular velocity that exceed the normal range of motion, thereby obtaining instantaneous change information of motion sensing data;

[0121] Based on the changing trend and spatial distribution characteristics of the observation residuals, the abnormal area information of the visual perception data, and the instantaneous change information of the motion perception data, it is determined whether the observation residuals are caused by sudden environmental changes, and the judgment result is obtained.

[0122] Specifically, the temporal trend of observed residuals refers to the evolution of residual values ​​over time, such as whether it is an instantaneous spike, a persistent shift, or a periodic fluctuation; the spatial distribution characteristics refer to the distribution pattern of residuals on the image or sensor array, such as local concentration, global dispersion, or a specific geometric shape. Analyzing these characteristics helps in the preliminary judgment of the nature of anomalies. Among these, anomaly region identification in visual perception data aims to detect whether there are areas in the image that do not conform to the normal internal structure of the pipeline, appear suddenly or disappear suddenly, or have abnormal shapes, colors, or brightness changes. For example, when a mobile device enters an area suddenly flooded, the visual data will show large areas of blurring or color changes, which are clearly different from the normal texture of the pipeline's inner wall. This identification can be achieved using techniques such as image segmentation, feature extraction, and matching. Furthermore, instantaneous change detection in motion perception data refers to monitoring data from sensors such as accelerometers and gyroscopes to determine whether there are sudden accelerations, decelerations, or changes in angular velocity beyond the normal range of motion. For example, when a mobile device suddenly collides with an obstacle or passes through a sharp bend, the motion perception data will show instantaneous large fluctuations. This detection can be performed using methods such as setting thresholds, filtering, or model-based prediction residual analysis. Ultimately, by comprehensively analyzing the changing trends and spatial distribution characteristics of the observation residuals, the abnormal area information of the visual perception data, and the instantaneous change information of the motion perception data, the root cause of the observation residuals can be determined more accurately. For example, if the observation residuals exhibit large instantaneous fluctuations, while the visual data shows large-area blurring and the motion data shows severe jitter, it is likely that the mobile device has encountered a sudden environmental change, such as suddenly entering a stream of water or getting stuck by an obstacle. Conversely, if the observation residuals show a continuous, small increase, and neither the visual nor the motion data shows obvious abnormalities, it may indicate that the quality of the data source itself is slowly declining.

[0123] This application overcomes the limitations of relying on a single data source or feature for judgment by employing a multi-dimensional, multi-source data fusion analysis approach. Specifically, by analyzing the temporal and spatial characteristics of observation residuals, the overall situation of anomalies can be grasped at a macro level; by identifying anomalous areas in visual perception data, visual changes in the environment, such as sudden changes in illumination, changes in the medium, or the appearance of obstacles, can be directly perceived; and by detecting instantaneous changes in motion perception data, physical impacts or drastic changes in the motion state of moving devices can be captured. It is precisely by comprehensively considering these different types and dimensions of information that the system can more comprehensively and accurately determine whether the observation residuals are caused by sudden environmental changes or by a decline in the quality of the data source itself. This comprehensive judgment mechanism avoids misjudgment and provides a reliable basis for subsequently adopting the correct fusion weight adjustment strategy.

[0124] The above technical solution significantly improves the accuracy and robustness of the system in determining the causes of observation residuals. Compared to relying solely on a single residual index, this solution combines the inherent variation patterns of the observation residuals, abnormal characteristics of the visual environment, and changes in the motion state of the mobile device, forming a more comprehensive judgment basis. Therefore, the system can effectively distinguish between two different types of anomalies: sudden environmental changes and deterioration in data source quality, thus avoiding inappropriate fusion weight adjustment strategies due to misjudgment. For example, in the event of sudden environmental changes, it avoids unnecessarily reducing the data source weight, ensuring the continuity and effectiveness of data fusion; in the event of deterioration in data source quality, it can promptly identify and adjust the weights, preventing the negative impact of low-quality data on overall positioning accuracy. This is of great significance for improving the adaptability and reliability of underground pipeline fault location methods.

[0125] In some embodiments, the steps of maintaining the fusion weight of the corresponding data source, triggering the environmental adaptation processing flow, and simultaneously initiating the environmental recovery monitoring mechanism when the determination result is an environmental mutation include:

[0126] Obtain information about the type of environmental mutation;

[0127] Based on the type of environmental mutation, activate the corresponding environmental adaptation processing module;

[0128] An environmental recovery monitoring mechanism is activated, which continuously monitors environmental characteristic values ​​to determine whether the environment has returned to a normal state.

[0129] During the operation of the environmental adaptation process, new information on environmental mutations is continuously acquired;

[0130] When new environmental mutation information is obtained, adjust the currently running environmental adaptation processing module or start a new environmental adaptation processing module according to the new environmental mutation information.

[0131] When the environmental recovery monitoring mechanism determines that the environment has returned to normal, it exits the environmental adaptation processing flow and restores the fusion weight of the corresponding data source.

[0132] Specifically, acquiring information on the type of environmental mutation refers to the system's further analysis of the specific nature of the mutation after it is identified, such as sudden changes in lighting, water mist, pipe wall contamination, or structural deformation. This type information can be determined through in-depth analysis of abnormal patterns in visual and motion perception data, or by combining it with a pre-set environmental model. The purpose is to provide a basis for subsequently initiating targeted environmental adaptation measures.

[0133] Activating the corresponding environmental adaptation processing module based on the type of environmental mutation can be understood as the system selecting and activating the most suitable processing strategy for the current environmental conditions from a pre-set processing module library based on the identified type of environmental mutation. For example, if a change in illumination is identified, the image brightness / contrast adaptive adjustment module may be activated; if water mist is identified, the defogging processing module may be activated. The purpose is to ensure the targeted and efficient nature of environmental adaptation processing.

[0134] The activation of the environmental recovery monitoring mechanism refers to the parallel operation of an independent monitoring mechanism during the initiation of the environmental adaptation processing flow. This mechanism continuously collects and analyzes environmental characteristic values, such as image clarity, the number of feature points, and the stability of motion data, to determine whether the environment has recovered from a sudden change to a normal or acceptable state. Its purpose is to provide a reliable basis for judging the exit of the environmental adaptation processing flow and the restoration of data source fusion weights.

[0135] During the operation of an environmentally adaptive processing flow, continuously acquiring new information on environmental mutations means that the system does not process environmental mutations only once, but maintains sensitivity to environmental changes throughout the process. This can be achieved by periodically reassessing environmental characteristics or monitoring the changing trends of observed residuals. The aim is to address the dynamic and complex nature of environmental mutations and avoid processing failure due to secondary environmental changes.

[0136] When new environmental mutation information is acquired, the system adjusts the currently running environmental adaptive processing module or activates a new one based on this information. This means that if the environment changes again or new mutations occur during the adaptive processing process, the system can respond promptly. For example, if severe fouling is detected on the inner wall of a pipe during defogging, the system can adjust the defogging parameters or simultaneously activate the fouling area identification and compensation module. The purpose is to improve the system's robustness and adaptability to complex and changing environments.

[0137] Finally, when the environmental recovery monitoring mechanism determines that the environment has returned to normal, it exits the environmental adaptation processing flow and restores the fusion weights of the corresponding data sources. This means that once the environmental recovery monitoring mechanism confirms that the environment is no longer in a state of abrupt change, the system will stop running the environmental adaptation processing module and restore the fusion weights of the data sources to normal levels according to the previously set strategy. The purpose is to avoid unnecessary processing overhead and ensure that information from each data source can be fully utilized when the environment is normal.

[0138] This application effectively addresses the potential lack of adaptability of basic solutions when facing complex, ever-changing, or continuously evolving environmental mutations by introducing mechanisms for acquiring information on environmental mutation types, targeted module activation, continuous environmental recovery monitoring, and dynamic response and module adjustment to new environmental mutation information during the processing flow. Specifically, by acquiring information on the type of environmental mutation, the system can accurately select the most suitable processing module, avoiding blind or improper processing. The introduction of the environmental recovery monitoring mechanism ensures that the environmentally adaptive processing flow can exit promptly and accurately after environmental recovery, avoiding resource waste. More importantly, continuously acquiring and responding to new environmental mutation information during the operation of the environmentally adaptive processing flow allows the system to dynamically adjust or activate new processing modules, thereby maintaining its robustness and processing effectiveness even when the environment continues to deteriorate or undergoes complex mutations, ensuring the continuity and accuracy of underground pipeline fault location.

[0139] Through the above technical solution, this application can significantly improve the adaptability and robustness of underground pipeline fault location methods in complex and ever-changing environments. Specifically, this solution avoids a one-size-fits-all approach by finely classifying environmental mutations and activating corresponding processing modules, thus improving processing efficiency and effectiveness. Simultaneously, the introduced environmental recovery monitoring mechanism enables the system to promptly cease adaptive processing after environmental recovery, optimizing resource utilization. Crucially, during the environmental adaptive processing flow, the system can continuously sense and respond to new environmental mutations, dynamically adjusting its processing strategy. This allows the method to maintain high accuracy and reliability in the face of continuously evolving or compounding harsh environments, effectively reducing location errors and system failure risks caused by environmental changes, thereby improving the overall accuracy and efficiency of fault location.

[0140] In some embodiments, the step of activating the corresponding environmental adaptation processing module based on the type information of the environmental mutation includes:

[0141] From the pre-defined processing module library, select environmental adaptive processing modules related to environmental mutation type information; extract visual perception data and motion perception data inside the current pipeline; perform feature extraction on the visual perception data and motion perception data to obtain real-time environmental features; compare the real-time environmental features with the typical environmental features corresponding to each selected processing module to obtain the matching degree; select the environmental adaptive processing module with the highest matching degree; configure its internal parameters according to the requirements of the selected environmental adaptive processing module.

[0142] Specifically, selecting environmentally adaptive processing modules related to environmental mutation type information from a pre-defined processing module library means that this library pre-stores various processing algorithms or models designed for different environmental mutation situations (such as drastic changes in light intensity, water mist, sediment accumulation, and pipe wall deformation). The selection process can be based on predefined mapping rules. For example, if the environmental mutation type information indicates strong light, then image enhancement or deglare removal modules for strong light conditions can be selected. Alternatively, a machine learning classifier can be used to intelligently search and match the module library based on the environmental mutation type information.

[0143] Extracting visual and motion perception data from within the pipeline refers to the system immediately acquiring the latest real-time data stream from the visual and motion sensors of the mobile device after a sudden environmental change. This data provides firsthand information reflecting the actual environmental conditions within the pipeline and the motion status of the mobile device.

[0144] Feature extraction is performed on visual perception data and motion perception data to obtain real-time environmental features. Specifically, for visual perception data, features such as average brightness, contrast, texture complexity, color distribution, and blurriness can be extracted; for motion perception data, features such as instantaneous acceleration, angular velocity, vibration frequency, and rate of change of motion trajectory can be extracted. These extracted features together constitute a multi-dimensional real-time environmental feature vector, used to quantitatively describe the specific state of the current environment.

[0145] The matching degree is obtained by comparing the real-time environmental features with the typical environmental features corresponding to each selected processing module. Each environmental adaptive processing module is optimized for specific typical environmental features during its design. The matching process can evaluate the matching degree by calculating the Euclidean distance, cosine similarity, or other metrics between the real-time environmental feature vector and the typical environmental feature vector of each module, or by using a pre-trained classification model to determine which module's typical environmental features are closest to the real-time environmental features.

[0146] The selection of the environmental adaptability processing module with the highest matching degree ensures that the chosen module is most effective in responding to the currently detected environmental abrupt changes. For example, if real-time environmental features indicate that the current environment has high ambiguity and low contrast, and has the highest matching degree with the typical environmental features of the water mist processing module, then that module is selected.

[0147] Configure the internal parameters of the selected environmentally adaptive processing module according to its requirements. Parameter configuration may include adjusting the algorithm's threshold, model weights, filter parameters, gain coefficients, etc., to ensure that the selected module achieves optimal performance under the current specific environmental conditions. For example, for an image enhancement module, its gamma correction coefficients or histogram equalization parameters can be dynamically adjusted based on the current illumination intensity and contrast.

[0148] This application first selects relevant environmental adaptation processing modules from a pre-defined processing module library based on the type of environmental mutation, ensuring the accuracy of the processing direction. Then, by extracting visual and motion perception data from within the pipeline in real time and performing feature extraction, a precise quantitative description of the current environment is obtained. Furthermore, these real-time environmental features are compared with the typical environmental features corresponding to each selected processing module, and the module with the highest matching degree is selected, thus ensuring a high degree of fit between the selected module and the current environmental conditions. Finally, the internal parameters of the selected module are configured according to its requirements, enabling environmental adaptation processing to be performed in an optimized manner, effectively addressing the challenges brought about by environmental mutations.

[0149] Through the above technical solution, this application enables intelligent selection and fine-grained configuration of environmental adaptability processing modules. Compared to simply activating preset modules based on the type of environmental mutation, this solution ensures the relevance and effectiveness of the selected modules by matching real-time environmental characteristics with typical module characteristics, avoiding inefficient or erroneous processing that may result from mismatched modules. Furthermore, the dynamic configuration of internal module parameters further enhances the accuracy and robustness of processing, thereby significantly improving the accuracy and reliability of fault location in complex and variable underground pipeline environments.

[0150] like Figure 2 As shown in the illustration, this application also discloses an underground pipeline fault location system for underground pipe networks, comprising:

[0151] Visual data acquisition module 1 is used to acquire visual data inside the pipeline via a mobile device;

[0152] Environmental adaptation processing module 2 is used to perform environmental adaptation processing on visual data to obtain processed visual data;

[0153] The target region separation module 3 is used to separate the target regions from the processed visual data to obtain target region information.

[0154] The anomaly identification and classification module 4 is used to identify and classify anomalies in visual data based on target area information, and obtain the identified defect information.

[0155] The three-dimensional spatial position association module 5 is used to associate the position of the identified defect information in the two-dimensional image with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial position information of the defect.

[0156] The geometric attribute acquisition module 6 is used to acquire the geometric attribute information of the defects based on the identified defect information;

[0157] The distribution representation output module 7 is used to output a distribution representation that includes the three-dimensional spatial location information and geometric attribute information of the defects.

[0158] The system provided in this application embodiment uses image processing and spatial positioning technology to achieve accurate, efficient, and non-contact detection and location of internal defects in underground pipelines.

[0159] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for locating underground pipeline faults in an underground pipe network, characterized in that, Includes the following steps: Acquire visual data of the pipeline interior using a mobile device; The visual data is subjected to environmental adaptation processing to obtain the processed visual data; The processed visual data is then subjected to target region separation to obtain target region information; Based on target area information, anomalies in visual data are identified and categorized to obtain the identified defect information. The location of the identified defect information in the two-dimensional image is correlated with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial location information of the defect. Based on the identified defect information, the geometric attribute information of the defect is obtained; The output includes a distribution representation of the defect's three-dimensional spatial location and geometric attribute information.

2. The method for locating underground pipeline faults in an underground pipe network according to claim 1, characterized in that, The step of associating the location of the identified defect information in the two-dimensional image with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial location information of the defect includes: Acquire the micro-vibration response of the pipeline structure captured by the micro-vibration sensing unit of the mobile device; Analyze the micro-vibration response of pipeline structures to identify external vibration characteristics; Based on the external vibration characteristics, the motion sensing data of the mobile device is adjusted to obtain the adjusted motion sensing data; The adjusted motion perception data is fused with the visual perception data of the mobile device to obtain the three-dimensional pose of the mobile device; Based on the three-dimensional pose of the mobile device, the position of the identified defect information in the two-dimensional image is correlated with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial position information of the defect.

3. The method for locating underground pipeline faults in an underground pipe network according to claim 2, characterized in that, The steps for analyzing the micro-vibration response of the pipeline structure and identifying external vibration characteristics include: The micro-vibration response of the pipeline structure is segmented to obtain the segmented micro-vibration response. Spectral analysis was performed on the segmented micro-vibration response to obtain the frequency distribution and energy spectrum. Identify multiple energy peaks in the frequency distribution and energy spectrum; Based on the preset external vibration characteristic information, the energy peak is matched to obtain the matching result; Analyze the evolution of energy peak values ​​within different time windows; Based on the matching results and evolution patterns, external vibration characteristics are identified.

4. The method for locating underground pipeline faults in an underground pipe network according to claim 2, characterized in that, The step of adjusting the motion sensing data of the mobile device based on external vibration characteristics to obtain adjusted motion sensing data includes: Obtain the confidence level of external vibration characteristics; Based on the confidence level, the method of processing the motion sensing data of the mobile device is adjusted to obtain the preliminarily adjusted motion sensing data; Continuous monitoring of the pipeline structure's response to minute vibrations; The external vibration characteristics were verified based on the continuously monitored micro-vibration response of the pipeline structure, and the verification results were obtained. Based on the verification results, the initially adjusted motion perception data was adjusted again to obtain the adjusted motion perception data.

5. The method for locating underground pipeline faults in an underground pipe network according to claim 2, characterized in that, The step of fusing the adjusted motion perception data with the visual perception data of the mobile device to obtain the three-dimensional pose of the mobile device includes: To obtain the texture richness, the number of feature points in the visual perception data, and the feature point tracking stability of the visual perception data; The reliability of visual perception data is evaluated based on texture richness, number of feature points, and feature point tracking stability. Acquire noise levels in motion sensing data, drift trends in motion sensing data, and confidence levels of external vibration compensation; The accuracy of motion sensing data is evaluated based on noise level, drift trend, and confidence level of external vibration compensation. Based on the reliability of visual perception data and the accuracy of motion perception data, the fusion weights of visual perception data and motion perception data are dynamically allocated. Based on the fusion weights, visual perception data and motion perception data are fused to obtain the three-dimensional pose of the mobile device; Based on the three-dimensional pose of the mobile device, the visual observation and motion perception data of the next moment are predicted to obtain the predicted value; The actual collected visual perception data and motion perception data are compared with the predicted values ​​to obtain the observation residuals; When the observation residuals exceed a preset threshold, the fusion weight of the corresponding data source is reduced.

6. The method for locating underground pipeline faults in an underground pipe network according to claim 5, characterized in that, The step of reducing the fusion weight of the corresponding data source when the observation residual exceeds a preset threshold includes: Analyze the observation residuals to determine whether they are caused by sudden environmental changes, and obtain the judgment results. When the judgment result is an environmental mutation, the fusion weight of the corresponding data source is maintained, the environmental adaptation processing flow is triggered, and the environmental recovery monitoring mechanism is started at the same time. When the judgment result indicates a decline in the quality of the data source itself, the fusion weight of the corresponding data source is adjusted based on the magnitude of the observation residuals, the duration of the observation residuals, and the historical reliability of the data source.

7. The method for locating underground pipeline faults in an underground pipe network according to claim 6, characterized in that, The steps for analyzing the observation residuals and determining whether the observation residuals are caused by sudden environmental changes to obtain the determination result include: By analyzing the temporal trend and spatial distribution characteristics of the observation residuals, the trend and spatial distribution characteristics of the observation residuals are obtained. Abnormal region identification is performed on visual perception data to determine whether there are regions in the visual perception data that do not conform to the pipeline structure, have specific shapes or brightness changes, and obtain abnormal region information of visual perception data. Instantaneous change detection is performed on motion sensing data to determine whether there are sudden changes in acceleration or angular velocity that exceed the normal range of motion, thereby obtaining instantaneous change information of motion sensing data; Based on the changing trend and spatial distribution characteristics of the observation residuals, the abnormal area information of the visual perception data, and the instantaneous change information of the motion perception data, it is determined whether the observation residuals are caused by sudden environmental changes, and the judgment result is obtained.

8. The method for locating underground pipeline faults in an underground pipe network according to claim 6, characterized in that, The steps of maintaining the fusion weight of the corresponding data source and triggering the environmental adaptation processing flow and initiating the environmental recovery monitoring mechanism when the judgment result is an environmental mutation include: Obtain information about the type of environmental mutation; Based on the type of environmental mutation, activate the corresponding environmental adaptation processing module; An environmental recovery monitoring mechanism is activated, which continuously monitors environmental characteristic values ​​to determine whether the environment has returned to a normal state. During the operation of the environmental adaptation process, new information on environmental mutations is continuously acquired; When new environmental mutation information is obtained, adjust the currently running environmental adaptation processing module or start a new environmental adaptation processing module according to the new environmental mutation information. When the environmental recovery monitoring mechanism determines that the environment has returned to normal, it exits the environmental adaptation processing flow and restores the fusion weight of the corresponding data source.

9. A method for locating underground pipeline faults in an underground pipe network according to claim 8, characterized in that, The step of activating the corresponding environmental adaptation processing module based on the type of environmental mutation information includes: From the preset processing module library, select environmental adaptation processing modules that are related to environmental mutation type information; Extract visual perception data and motion perception data from inside the current pipeline; Feature extraction is performed on visual perception data and motion perception data to obtain real-time environmental features; The real-time environmental features are compared with the typical environmental features corresponding to each selected processing module to obtain the degree of matching. Select the environmental adaptability processing module with the highest matching degree; Configure the internal parameters of the selected environmental adaptability processing module according to its requirements.

10. A fault location system for underground pipelines in an underground pipe network, characterized in that, The system includes: The visual data acquisition module is used to acquire visual data inside the pipeline via a mobile device; The environmental adaptability processing module is used to perform environmental adaptability processing on visual data to obtain processed visual data. The target region separation module is used to separate the target regions from the processed visual data to obtain target region information; The anomaly identification and classification module is used to identify and classify anomalies in visual data based on target area information, and obtain the identified defect information. The three-dimensional spatial location association module is used to associate the position of the identified defect information in the two-dimensional image with the three-dimensional motion information of the mobile device to obtain the three-dimensional spatial location information of the defect. The geometric attribute acquisition module is used to acquire the geometric attribute information of defects based on the identified defect information; The distribution representation output module is used to output a distribution representation that includes the three-dimensional spatial location information and geometric attribute information of the defects.

Images