Rain and sewage diversion pipeline detection system based on CCTV and application method

Abstract

The invention provides a rain and sewage diversion pipeline detection system based on a CCTV and an application method, and relates to the technical field of rain and sewage diversion pipeline detection. In the system, an acquisition module acquires a visible light image, an infrared image and a depth image of a pipeline; the processing module performs feature extraction on each image to obtain a texture feature, a thermal anomaly region feature and a spatial feature, and fuses each feature based on an adaptive weight and a feature confidence coefficient to obtain a fused feature; the sensing obstacle avoidance module senses obstacles in the pipeline and generates a path instruction based on sensed target obstacle information; the AI analysis defect detection module performs pipeline defect detection according to the fusion features, performs mutual correction in combination with a pipeline defect detection result and target obstacle information, and adjusts a path instruction according to a correction result; and the robot module moves in the pipeline and climbs over the obstacle based on the adjusted path instruction. According to the invention, the robustness and reliability of rain and sewage diversion pipeline detection can be improved.

Description

Technical Field

[0001] This invention relates to the field of stormwater and sewage separation pipeline inspection technology, and in particular to a stormwater and sewage separation pipeline inspection system and application method based on CCTV. Background Technology

[0002] During the long-term operation of infrastructure such as urban drainage, industrial pipelines, and water supply networks, the internal environment of pipelines is often quite complex. Affected by factors such as geological changes, construction quality, long-term use, corrosion, and sediment accumulation, problems such as cracks, misalignment, collapse, blockage, sediment accumulation, and abrupt slope changes may occur inside the pipeline. To ensure the safety and operational efficiency of pipeline systems, regular pipeline inspection and evaluation are crucial.

[0003] Currently, CCTV pipeline inspection robots are one of the main tools for pipeline inspection, relying on video acquisition, defect identification, and remote monitoring for detection. However, existing defect identification methods still mainly rely on traditional image processing techniques, such as edge detection and template matching. These methods are effective in identifying relatively obvious structural defects (such as cracks, breaks, and foreign object blockages), but their ability to identify subtle defects such as tiny cracks and early signs of corrosion is limited. Moreover, because traditional image processing algorithms are sensitive to changes in lighting, they struggle to extract weak defect features from complex background textures. Their performance drops sharply in complex environments commonly found inside pipelines, such as uneven lighting, strong reflections, water vapor, or shadows, resulting in severely insufficient system robustness. This makes it difficult to conduct reliable health assessments in variable real-world environments and to meet the needs of pipeline health assessments in complex environments. Summary of the Invention

[0004] This invention provides a CCTV-based method, apparatus, and system for detecting rainwater and sewage separation pipelines, addressing the problem that traditional CCTV-based rainwater and sewage separation pipeline detection methods lack robustness and struggle to meet the needs of pipeline health assessment in complex environments.

[0005] In a first aspect, embodiments of the present invention provide a CCTV-based rainwater and sewage separation pipeline inspection system, comprising: a robot module and an AI analysis defect detection module, wherein the robot module is equipped with a data acquisition module, a processing module and a perception and obstacle avoidance module; The acquisition module is used to acquire visible light images, infrared images, and depth images of the pipeline; The processing module is used to extract features from the visible light image, the infrared image, and the depth image respectively to obtain texture features, thermal anomaly region features, and spatial features, and to fuse the texture features, thermal anomaly region features, and spatial features based on adaptive weights and feature confidence to obtain fused features; The obstacle perception and avoidance module is used to perceive obstacles in the pipeline, obtain target obstacle information, and generate path instructions based on the target obstacle information; The AI ​​analysis defect detection module is used to detect pipeline defects based on the fused features, and to perform mutual correction by combining the pipeline defect detection results and the target obstacle information, and to adjust the path instructions based on the correction results. The robot module is used to move within the pipe and traverse obstacles based on adjusted path instructions.

[0006] In one possible implementation, the processing module is specifically used for: Preliminary pipeline defect detection is performed on the texture features, the thermal anomaly region features, and the spatial features to obtain initial pipeline defect detection results; Based on the initial pipeline defect detection results, determine the feature confidence levels corresponding to the texture features, the thermal anomaly region features, and the spatial features, respectively. The adaptive weights corresponding to the texture features, thermal anomaly region features, and spatial features are adjusted based on the feature confidence scores corresponding to the texture features, thermal anomaly region features, and spatial features, respectively. The texture features, thermal anomaly region features, and spatial features are fused according to the corrected adaptive weights corresponding to the texture features, thermal anomaly region features, and spatial features respectively, to obtain fused features.

[0007] In one possible implementation, the initial pipeline defect detection results include the defect type and defect confidence level for each defect; The processing module is specifically used for: The confidence levels of the basic features corresponding to the texture features, the thermal anomaly region features, and the spatial features are determined based on the image quality of the visible light image, the image quality of the infrared image, and the image quality of the depth image. For each defect type, the proportion of defects under that defect type with a defect confidence score greater than a preset threshold is counted and recorded as the target proportion. Based on the correlation degree between each defect type and the texture feature, the thermal anomaly region feature and the spatial feature, and the target proportion corresponding to each defect type, the basic feature confidence of the texture feature, the thermal anomaly region feature and the spatial feature are corrected to obtain the feature confidence of the texture feature, the thermal anomaly region feature and the spatial feature, respectively.

[0008] In one possible implementation, the obstacle avoidance module is specifically used for: Pipeline point cloud data is acquired based on the lidar sensing unit, and a spatial occupancy grid map is constructed based on the pipeline point cloud data. Clustering is performed on the continuously occupied grid cells in the spatial occupancy grid map, and obstacle information is obtained based on the clustering results, which is denoted as the first obstacle information; Information about the second obstacle is obtained based on the ultrasonic obstacle detection unit; Calculate the difference between the estimated distance of the forward obstacle in the second obstacle information and the distance of the nearest obstacle in the first obstacle information; If the difference exceeds a set threshold, the second obstacle information is determined as the target obstacle information; If the difference does not exceed the set threshold, the information of the nearest obstacle in the first obstacle information is determined as the target obstacle information; Path instructions are generated based on the target obstacle information, the risk factor weights corresponding to the types of target obstacles in the target obstacle information, and the current center coordinates of the robot module.

[0009] In one possible implementation, the obstacle avoidance module is specifically used for: If the target obstacle information contains only one target obstacle, then the target offset direction vector is determined based on the coordinates of the target obstacle in the target obstacle information and the center coordinates; the target offset distance is calculated based on the target offset direction vector and the risk factor weight; the target point is determined based on the target offset distance and the center coordinates; and the target point is used as the path instruction. If the target obstacle information includes at least two target obstacles, then a cost function is constructed based on the coordinates of each target obstacle and the corresponding risk factor weight. The local path with the lowest cost is found based on the cost function. The target point is determined based on the local path and the center coordinates, and the target point is used as the path instruction.

[0010] In one possible implementation, the cost function is: ; in, For point The cost function value, For the first Risk factor weights for each target obstacle , The target obstacle information includes the number of target obstacles. For the first The coordinates of the target obstacle For the first The effective radius of influence of a target obstacle.

[0011] In one possible implementation, the AI ​​analysis defect detection module is specifically used for: Determine the two-dimensional image coordinate information corresponding to each defect in the pipeline based on the pipeline defect detection results; The three-dimensional spatial coordinates of each defect in the pipeline are determined by back projection based on the coordinate information of the two-dimensional image. Based on the three-dimensional spatial coordinates of each defect in the pipeline and the target obstacle information, it is identified whether the defect is located on the feasible path of the robot module or near the obstructed area. If the defect is located on the feasible path of the robot module or near the obscured area, the path command is adjusted according to the three-dimensional spatial coordinates of the defect located on the feasible path of the robot module or near the obscured area.

[0012] In one possible implementation, the AI ​​analysis defect detection module is specifically used for: The defect range corresponding to each defect in the pipeline is determined based on the three-dimensional spatial coordinates of each defect in the pipeline. Based on the target obstacle information, determine the range of the target obstacle; Determine whether the defect range corresponding to each defect in the pipeline overlaps with the range of the target obstacle. If the defect range corresponding to a certain defect in the pipeline overlaps with the range of the target obstacle, then the defect is determined to be located on the feasible path of the robot module or near the obstructed area.

[0013] In one possible implementation, the AI ​​analysis defect detection module is further used for: Based on the pipeline defect detection results and historical pipeline defect detection results, the aging trend of the pipeline is predicted, and the aging prediction results of the pipeline are obtained.

[0014] Secondly, embodiments of the present invention provide an application method for a CCTV-based stormwater and sewage separation pipeline inspection system, including: The acquisition module acquires visible light images, infrared images, and depth images of the pipeline; The processing module extracts features from the visible light image, the infrared image, and the depth image to obtain texture features, thermal anomaly region features, and spatial features, and then fuses the texture features, thermal anomaly region features, and spatial features based on adaptive weights and feature confidence to obtain fused features. The obstacle avoidance module senses obstacles within the pipeline, obtains target obstacle information, and generates path instructions based on the target obstacle information; The AI ​​analysis defect detection module performs pipeline defect detection based on the fused features, and performs mutual correction by combining the pipeline defect detection results and the target obstacle information, and adjusts the path instructions according to the correction results; The robot module moves within the pipe and traverses obstacles based on the adjusted path instructions.

[0015] In this embodiment of the invention, visible light images, infrared images, and depth images of the pipeline are acquired by an acquisition module. The acquisition module integrates multi-source sensing components to construct an efficient and comprehensive data acquisition system for complex pipeline environments, ensuring data acquisition quality in low-light, humid, and obstructed environments. Then, a processing module mounted on the robot module extracts features from the visible light, infrared, and depth images to obtain texture features, thermal anomaly region features, and spatial features. These features are then fused based on adaptive weights and feature confidence to obtain fused features. This allows the processing module to extract and fuse features from visible light, infrared, and depth images locally, reducing data transmission pressure, improving the real-time performance of detection feedback, and enhancing the overall system response efficiency. Furthermore, the use of adaptive weights and feature confidence to achieve reliable fusion of visible light, infrared, and depth images helps improve the system's detection reliability. In addition to detecting pipeline defects based on fused features, the AI-analyzed defect detection module also performs mutual correction by combining pipeline defect detection results and target obstacle information. Based on the correction results, the path instructions are adjusted so that the robot module can move inside the pipeline and overcome obstacles based on the adjusted path instructions. This enhances the coordination between the modules of the CCTV-based rainwater and sewage separation pipeline detection system and further improves the robustness and reliability of rainwater and sewage separation pipeline detection from the perspective of multi-module collaboration. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the CCTV-based rainwater and sewage diversion pipeline detection system provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the implementation of the application method of the CCTV-based rainwater and sewage diversion pipeline detection system provided in this embodiment of the invention. Detailed Implementation

[0017] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0018] See Figure 1The diagram shows a schematic of the structure of a CCTV-based rainwater and sewage diversion pipeline inspection system provided in an embodiment of the present invention. The CCTV-based rainwater and sewage diversion pipeline inspection system 10 includes: a robot module 11 and an AI analysis defect detection module 12. The robot module 11 is equipped with a data acquisition module 111, a processing module 112 and a perception and obstacle avoidance module 113.

[0019] The acquisition module 111 is used to acquire visible light images, infrared images, and depth images of the pipeline.

[0020] The processing module 112 is used to extract features from the visible light image, infrared image and depth image respectively to obtain texture features, thermal anomaly region features and spatial features, and to fuse the texture features, thermal anomaly region features and spatial features based on adaptive weights and feature confidence to obtain fused features.

[0021] The obstacle perception and avoidance module 113 is used to perceive obstacles in the pipeline, obtain target obstacle information, and generate path instructions based on the target obstacle information.

[0022] The AI ​​analysis defect detection module 12 is used to detect pipeline defects based on fused features, and to perform mutual correction by combining the pipeline defect detection results and target obstacle information, and to adjust the path instructions based on the correction results.

[0023] Robot module 11 is used to move within the pipe and traverse obstacles based on adjusted path instructions.

[0024] For example, the acquisition module 111 may include a high-definition camera unit, an infrared thermal imaging unit, and a structured light 3D scanning unit, for acquiring visible light images, infrared images, and depth images inside the pipe, respectively, thereby fusing image, thermal imaging, and structured light scanning technologies, and encapsulating them through physical structures to adapt to the deployment limitations in narrow pipe environments.

[0025] In this embodiment, the acquisition module 111 integrates multi-source sensing components to build an efficient and comprehensive data acquisition system for complex pipeline environments. It can still ensure data quality in environments such as low light, humidity, and obstruction, providing a solid data foundation for subsequent algorithm modules such as defect detection, obstacle recognition, and aging analysis.

[0026] The visible light image, infrared image, and depth image acquired by the acquisition module 111 are initially processed at the local edge and then streamed to the processing module 112 for further analysis. The specific component integration method and signal flow logic can be as follows: The high-definition camera unit, infrared thermal imaging unit, and structured light 3D scanning unit are integrated on the front support of the detection robot (i.e., robot module 11) and arranged at equal intervals to maintain a field-of-view overlap angle of not less than 45°. This arrangement ensures that the three sensing results have spatial alignment characteristics, providing conditions for subsequent multimodal feature fusion.

[0027] Building upon this foundation, to effectively support the data processing requirements of high-frequency acquisition and improve the overall system response efficiency, this embodiment further provides a processing module 112, which mainly includes an edge computing unit. This processing module 112, by implementing lightweight processing on the robot body (i.e., robot module 11) or front-end nodes, enables rapid preliminary analysis of visible light images, infrared images, and depth images locally, reducing data transmission pressure, improving the real-time performance of detection feedback, and ensuring reliable output of different types of data under multimodal conditions through feature extraction and fusion.

[0028] For example, the edge computing unit embedded in the processing module 112 can be deployed on the core board of the robot module 11, equipped with at least a quad-core ARM processor and a GPU co-processing module. The edge computing unit can use a lightweight image enhancement algorithm for image processing tasks to extract texture features.

[0029] For example, taking brightness correction as an example, the processing flow can include two steps: grayscale stretching and contrast adjustment. The grayscale stretching function is as follows: ; in, In the original visible light image The pixel grayscale value of the location. , These are the minimum and maximum gray levels in the current frame of the visible light image. These are the enhanced pixel values.

[0030] In addition, adaptive histogram equalization (CLAHE) and Retinex-based brightness normalization methods can be used to enhance the clarity of low-light areas, taking into full account the special environmental conditions of the rainwater and sewage diversion pipeline, and further combined with edge enhancement filtering technology to highlight details such as small cracks.

[0031] For thermal imaging data (i.e., infrared images), the processing module 112 can calculate the variance of the temperature distribution in the region. This is used to identify anomalous hotspot regions (i.e., to extract features of thermal anomaly regions). The value is defined as: ; in, This represents the temperature value of each pixel in the infrared image. The average temperature of the entire infrared image frame. , This refers to the image's width and height in pixels. For example, if... This can then be marked as a "thermal anomaly region" and a mask map can be generated. The mask image can be used for subsequent feature fusion and defect analysis.

[0032] Alternatively, the processing module 112 can use median filtering to remove thermal noise from the infrared image, and use temperature gradient analysis technology based on thermal maps to enhance the thermal response area of ​​leakage or foreign objects by utilizing temperature gradient features, so as to enhance the detection capability of temperature foreign object areas in the pipeline. At the same time, it can also generate pseudo-color hot spot attention maps as needed to improve the model's sensitivity to thermal response areas.

[0033] For depth images generated by structured light, spatial interpolation and gradient calculation can be used to extract abrupt surface changes. By utilizing the interpolation and gradient feature extraction methods of depth images, accurate detection of the contours of collapses or internal obstacles can be ensured.

[0034] After extracting texture features, thermal anomaly region features, and spatial features respectively, attention mechanisms (such as SE modules or CBAM) can be introduced to obtain adaptive weights for each modality feature (i.e., texture features, thermal anomaly region features, and spatial features) to improve the response intensity of key modalities (such as depth images or infrared images) in defect identification, and to evaluate the feature confidence of each modality feature. Then, based on the adaptive weights and feature confidence of each modality feature, weighted fusion is performed to improve the reliability of the fused features.

[0035] In addition, an image spatial registration mechanism can be introduced to ensure that the multimodal images are aligned at the subpixel level before fusion through geometric correction and local optical axis calibration, effectively reducing the positioning error caused by viewpoint shift and distortion. Finally, the fused multimodal features are input into the backbone convolutional neural network of the AI ​​analysis defect detection module 12 for defect detection and classification.

[0036] For example, the perception and obstacle avoidance module 113 may include a lidar perception unit and an ultrasonic obstacle detection unit, which are used to actively perceive and collect information about obstacles in the pipeline. The data from the active perception and information collection can be used to generate path instructions and can also be transmitted to the AI ​​analysis and defect detection module 12 for secondary identification and acquisition of detection information.

[0037] In this embodiment, the perception and obstacle avoidance module 113 is mainly used to identify dynamic or static obstacles inside the pipeline, ensuring the stable operation of detection equipment (such as a robot module) in complex spaces. This module integrates two sensing methods: a lidar sensing unit and an ultrasonic obstacle detection unit, which respectively handle spatial positioning and distance measurement tasks. This design provides reliable obstacle detection capabilities even under harsh conditions such as humidity, high reflectivity, and limited line of sight. After preliminary processing locally, the obstacle detection data can be transmitted to the AI ​​analysis defect detection module in structured data form, providing support information for defect identification path adjustment. This gives the overall system low power consumption, short latency, and strong adaptability.

[0038] In this embodiment, the AI-based defect detection module 12 is entirely composed of a deep learning image processing unit. During pipeline defect detection, the neural network can match the current fused features with preset defect categories based on the weights learned from training samples, outputting the predicted probability of each defect. The system automatically selects the category with the highest predicted value as the current pipeline defect identification result and locates the most significant response region in the feature map, i.e., the spatial location of the defect. Through mapping, this location can be projected back into the original image coordinates as the defect's annotation point. Finally, the identification result, confidence score, defect spatial coordinates, and current time are packaged into a set of structured information to form a standard defect report. This report can be directly called by subsequent aging trend analysis, risk assessment, or path correction modules, forming a system-level closed-loop response chain.

[0039] To improve stability, in practical applications, the system can also introduce a multi-frame fusion mechanism to compare the classification output of the same region in consecutive frames. If a certain category has a significant advantage in multiple frames, the confidence of its final label is enhanced, reducing the interference caused by misjudgment in a single frame.

[0040] For example, after classification, the AI-based defect detection module 12 can generate a structured defect report based on the defect type, location, and morphological characteristics. This report may include the following core fields: Defect type label (crack, corrosion, or blockage).

[0041] Defect location (correspondence between image coordinates and actual space).

[0042] Defect morphology description (length, width, area, or degree of occlusion).

[0043] Classification confidence (indicating the reliability of the recognition result).

[0044] Detect timestamps (used for time series records).

[0045] Defect trend status (static / expanding).

[0046] The AI-powered defect detection module 12 can also combine pipeline defect detection results with target obstacle information for mutual correction. Based on the correction results, it adjusts the path instructions to dynamically feed back and correct the robot's trajectory according to the defect distribution and risk level, thereby dynamically adjusting the robot's movement path. If a defect area obstructs the robot's path or is identified as posing a risk, the module can construct an avoidance path request, proactively notifying the robot module to avoid the defect cluster area or select a more favorable observation angle for image acquisition.

[0047] In addition, the AI ​​analysis defect detection module 12 can also analyze and predict the aging trend of pipelines to make trend predictions on internal pipeline defects.

[0048] For example, a pipeline aging trend model can be further constructed based on the accumulation of historical data. This model can record the number, distribution, type changes, and expansion rate of defects in each pipe section. During multiple inspections, if phenomena such as an increase in defects, lengthening of cracks, widening of temperature differences in corroded areas, and increased density of blockages are found in a certain area, the system will determine that there is an accelerated aging trend in that area. Through the above analysis, the system will output an aging level score for that pipe section, which will be used to support maintenance units in adjusting inspection frequency, arranging maintenance plans, or estimating asset lifespan. Finally, the aging prediction result will be attached to the defect report, forming a future-oriented maintenance risk warning document, as an important component of structural health management in intelligent drainage systems.

[0049] This invention employs an acquisition module to collect visible light, infrared, and depth images of a pipeline. By integrating multi-source sensing components within the acquisition module, a highly efficient and comprehensive data acquisition system for complex pipeline environments is constructed, ensuring data acquisition quality in low-light, humid, and obstructed environments. Then, a processing module mounted on a robot module extracts features from the visible light, infrared, and depth images to obtain texture features, thermal anomaly region features, and spatial features. These features are then fused based on adaptive weights and feature confidence levels to obtain fused features. This allows the processing module to perform feature extraction and fusion of visible light, infrared, and depth images locally, reducing data transmission pressure, improving the real-time performance of detection feedback, and enhancing the overall system response efficiency. Furthermore, the use of adaptive weights and feature confidence levels enables reliable fusion of visible light, infrared, and depth images, thereby contributing to improved system detection reliability. In addition to detecting pipeline defects based on fused features, the AI-analyzed defect detection module also performs mutual correction by combining pipeline defect detection results and target obstacle information. Based on the correction results, the path instructions are adjusted so that the robot module can move inside the pipeline and overcome obstacles based on the adjusted path instructions. This enhances the coordination between the modules of the CCTV-based rainwater and sewage separation pipeline detection system and further improves the robustness and reliability of rainwater and sewage separation pipeline detection from the perspective of multi-module collaboration.

[0050] In one possible implementation, processing module 112 is specifically used for: Preliminary pipeline defect detection is performed based on texture features, thermal anomaly region features, and spatial features to obtain initial pipeline defect detection results.

[0051] Based on the initial pipeline defect detection results, determine the feature confidence levels corresponding to texture features, thermal anomaly region features, and spatial features, respectively.

[0052] The adaptive weights corresponding to texture features, thermal anomaly region features, and spatial features are adjusted based on the feature confidence scores corresponding to texture features, thermal anomaly region features, and spatial features, respectively.

[0053] Based on the modified adaptive weights corresponding to the texture features, thermal anomaly region features, and spatial features respectively, the texture features, thermal anomaly region features, and spatial features are fused to obtain the fused features.

[0054] In this embodiment, to improve the fusion effect of multimodal data, the three image features are not simply concatenated into a three-channel tensor. Instead, they are processed separately through a multi-branch feature extraction structure before being weighted and fused. This processing method can improve the independence of image features and the efficiency of fusion.

[0055] Moreover, in the weighted fusion process, in addition to the adaptive weights, feature confidence levels determined based on the initial pipeline defect detection results are also added, thereby enabling mutual verification between the detection results and the detection process, so as to more reliably achieve the fusion of texture features, thermal anomaly region features and spatial features.

[0056] In one possible implementation, the initial pipeline defect detection results include the defect type and defect confidence level for each defect. Processing module 112 is specifically used for: The confidence levels of basic features corresponding to texture features, thermal anomaly region features, and spatial features are determined based on the image quality of visible light images, infrared images, and depth images.

[0057] For each defect type, the proportion of defects under that defect type whose defect confidence is greater than a preset threshold is counted and recorded as the target proportion.

[0058] Based on the correlation degree between each defect type and texture features, thermal anomaly region features and spatial features, and the target proportion corresponding to each defect type, the basic feature confidence levels corresponding to texture features, thermal anomaly region features and spatial features are corrected to obtain the feature confidence levels corresponding to texture features, thermal anomaly region features and spatial features respectively.

[0059] In this embodiment, the basic feature confidence level of texture features can be determined based on the image quality of the visible light image, the basic feature confidence level of thermal anomaly region features can be determined based on the image quality of the infrared image, and the basic feature confidence level of spatial features can be determined based on the image quality of the depth image.

[0060] Based on this, pipeline defects generally include: cracks, corrosion, and blockages. Cracks include longitudinal cracks, transverse cracks, and irregular crazing, commonly found inside cement structures, and are usually linear or branching. Corrosion often manifests as surface texture erosion, changes in metallic luster, or low-temperature areas on thermal images. Blockages manifest as foreign objects inside the pipeline, sludge accumulation, or obvious physical obstruction. The initial pipeline defect detection results can include the defect type and defect confidence level for each defect. The higher the proportion of defects with a confidence level greater than a preset threshold for each defect type, the more reliable the detected defect is. Texture features extracted from visible light images can characterize texture roughness, crack edge contours, etc., thereby identifying visible defects such as crack edges and blockage surface textures; that is, texture features can correspond to crack defects. Thermal anomaly region features extracted from infrared images can identify color anomaly regions, locations of abrupt changes in thermal gradients, etc., thereby identifying thermal anomalies, judging internal corrosion, cavities, or damp areas; that is, thermal anomaly region features can correspond to corrosion defects. Spatial features extracted from depth images can aid in the analysis of pipe wall deformation, indentation, and deformation amplitude; in other words, spatial features can correspond to blockage-type defects. Therefore, the higher the proportion of defects with a confidence level greater than a preset threshold for each defect type, the higher the confidence level of the features extracted from the corresponding image.

[0061] In addition, considering that each defect type does not correspond one-to-one with texture features, thermal anomaly region features, and spatial features (or visible light images, infrared images, and depth images), the degree of correlation between each defect type and texture features, thermal anomaly region features, and spatial features can be measured. This correlation degree and the aforementioned target ratio can be used to more accurately correct the basic feature confidence of each modality.

[0062] For example, when measuring the correlation between each defect type and texture features, thermal anomaly region features, and spatial features, the current pipeline detection scenario can be determined based on visible light images, infrared images, and depth images. The correlation between each defect type and texture features, thermal anomaly region features, and spatial features can then be determined based on this scenario. For instance, based on historical pipeline detection results, the probability of each defect type being associated with different images under different pipeline detection scenarios can be statistically analyzed, and this probability can be used as the correlation between each defect type and different modal features under that pipeline detection scenario. For example, in a normal detection scenario, if a total of 5 blockage defects are detected, 4 of which are detected based on depth images and 1 based on visible light images, then the correlation between blockage defects and spatial features under normal detection scenarios can be determined as 4 / 5, and the correlation between blockage defects and texture features under normal detection scenarios can be determined as 1 / 5. The method for determining the correlation in other pipeline detection scenarios is similar to that in normal detection scenarios and will not be elaborated further here.

[0063] For example, before determining the current pipeline inspection scenario based on visible light images, infrared images, and depth images, different pipeline inspection scenarios in historical pipeline inspection results can be identified by clustering, so as to determine the current pipeline inspection scenario by calculating the distance between the visible light images, infrared images, and depth images and the cluster center of each pipeline inspection scenario.

[0064] For example, based on the correlation degree between each defect type and texture features, thermal anomaly region features, and spatial features, and the target proportion corresponding to each defect type, the basic feature confidence levels corresponding to texture features, thermal anomaly region features, and spatial features are corrected to obtain the feature confidence levels corresponding to texture features, thermal anomaly region features, and spatial features, respectively. This can include: according to The feature confidence scores corresponding to texture features, thermal anomaly region features, and spatial features are obtained respectively.

[0065] in, For the first Feature confidence of a feature , For the first The target proportion corresponding to each defect type For the first Type of defect and the first The degree of correlation between these features For the first The confidence level of the basic features of a feature.

[0066] For example, It can characterize texture features. It can characterize the features of thermal anomaly regions. It can characterize spatial features, or, It can characterize texture features. It can characterize the features of thermal anomaly regions. It can characterize spatial features, etc. In this embodiment, the order of texture features, thermal anomaly region features and spatial features is not limited.

[0067] In one possible implementation, the obstacle avoidance module 113 is specifically used for: Pipeline point cloud data is acquired using LiDAR sensing units, and a spatial occupancy raster map is constructed based on the pipeline point cloud data.

[0068] Cluster the continuously occupied grid cells in the spatial occupancy grid map, and obtain obstacle information based on the clustering results, which is denoted as the first obstacle information.

[0069] Information about the second obstacle is obtained based on the ultrasonic obstacle detection unit.

[0070] Calculate the difference between the estimated distance of the forward obstacle in the second obstacle information and the distance of the nearest obstacle in the first obstacle information.

[0071] If the difference exceeds the set threshold, the second obstacle information will be identified as the target obstacle information.

[0072] If the difference does not exceed the set threshold, the information of the nearest obstacle in the first obstacle information is determined as the target obstacle information.

[0073] The path instructions are generated based on the target obstacle information, the risk factor weights corresponding to the types of target obstacles in the target obstacle information, and the current center coordinates of the robot module.

[0074] In this embodiment, the lidar sensing unit in the obstacle avoidance module 113 can be a rotating 2D lidar device, with the following limitations: scanning frequency between 10Hz and 20Hz, laser beam scanning angle of 270°, no less than 720 output points per circle, angular resolution of approximately 0.5°, single-point distance measurement range of 0.15 meters to 12 meters, and distance accuracy better than ±3 centimeters. The sampling data structure can be: ; in, For the first The radial distance from each sample point to the center of the sensor (i.e., the lidar sensing unit). For the corresponding scanning angle, Given the number of points per lap, the obstacle point cloud is obtained in Cartesian coordinates after coordinate transformation: ; Each frame of point cloud is rasterized and projected to construct a spatial occupancy raster map. Obstacle recognition can be based on continuous occupancy grid clustering, such as using the DBSCAN algorithm based on Euclidean distance to spatially cluster point clusters and identify obstacle contours.

[0075] When using the DBSCAN clustering algorithm to perform spatial clustering of point clusters, a distance threshold can be set. Perform density clustering with the minimum number of points (MinPts) and output the coordinates of the center point of each obstacle cluster. And its boundary dimensions. After clustering, the system constructs the initial values ​​of the barrier cluster state vector: ; in, This represents the initial velocity, which can be set to 0 or calculated from position changes over multiple frames.

[0076] In one embodiment, to increase adaptability, an adaptive clustering radius strategy can be introduced, whereby the distance from each laser point to the radar center is denoted as . The corresponding cluster radius is: ; in , This is an adjustment coefficient used to balance the clustering consistency of point clouds at near and far distances.

[0077] This embodiment introduces an adaptive clustering radius strategy, which can solve the problem of drastic changes in point density caused by changes in radar viewpoint and improve clustering consistency.

[0078] In addition, a cylindrical model of the pipe wall can be constructed to identify and remove points close to the inner wall, thus avoiding false clustering of the background. For example, when the distance between a point and the pipe wall meets the following conditions... Sometimes it does not participate in clustering.

[0079] Then, after obtaining the obstacle cluster location (i.e., the first obstacle information) based on the lidar sensing unit, the system simultaneously accesses the data from the ultrasonic obstacle detection unit (i.e., the second obstacle information), which includes the estimated distance of the forward obstacle detected by the ultrasonic obstacle detection unit, calculated using the following formula: ; Here This is the estimated distance to the obstacle in front. The speed of ultrasonic wave propagation. This represents the time interval between the transmission and reception of the ultrasonic wave.

[0080] If the estimated distance to the forward obstacle is the closest point estimated by the lidar sensor alone... The difference exceeds the set threshold If the current radar data is flagged as "potentially a false alarm," a redundancy verification mechanism is activated, and the ultrasonic obstacle detection unit takes over the determination of the area. This complementary mechanism ensures the reliability of obstacle detection, especially when reflection distortion is severe or when there is heavy moisture in the pipes.

[0081] In one possible implementation, the obstacle avoidance module 113 is specifically used for: If the target obstacle information contains only one target obstacle, then the target offset direction vector is determined based on the target obstacle's coordinates and center coordinates in the target obstacle information. The target offset distance is calculated based on the target offset direction vector and risk factor weights. The target point is determined based on the target offset distance and center coordinates, and the target point is used as the path instruction.

[0082] If the target obstacle information contains at least two target obstacles, then a cost function is constructed based on the coordinates of each target obstacle and the corresponding risk factor weight. The local path with the lowest cost is found based on the cost function. The target point is determined based on the local path and the center coordinates, and the target point is used as the path instruction.

[0083] For example, the cost function is: ; in, For point The cost function value, For the first Risk factor weights for each target obstacle , The target obstacle information includes the number of target obstacles. For the first The coordinates of the target obstacle For the first The effective radius of influence of a target obstacle.

[0084] In this embodiment, the target obstacle information is merged and packaged into the following structure to generate path instructions and pass them to the AI ​​analysis defect detection module: ; This structured data includes the spatial location of obstacles, their dynamic trends, and information on the shortest forward distance.

[0085] Specifically, to further enhance the robot module's autonomous navigation and dynamic adjustment capabilities in the pipeline environment, in this embodiment, after obtaining the target obstacle information, the real-time position, spatial characteristics, and type information of the obstacle are comprehensively analyzed to output path instructions for dynamic path adjustment to the AI ​​analysis and defect detection module.

[0086] During the analysis process, obstacle type labels are also considered. Establish risk factor weights The settings are as follows: ; This weight will affect the path offset magnitude and detour radius control. Let the current center coordinates of the robot module be... The nearest point of the obstacle in the target obstacle information is The target offset direction vector is: ; The target offset direction vector, after unit normalization, is: ; The target offset distance is calculated as follows: ; in, , representing the initial distance between the obstacle and the robot module. As a regulating factor, it is usually taken as , Finally, the new target point is calculated according to the following formula. Used to guide robots or replan their paths:

[0087] in, This can represent the minimum offset distance required for obstacle avoidance. It is a unit direction vector, representing the direction of obstacle avoidance adjustment, usually perpendicular to the obstacle direction or the path normal direction.

[0088] In some cases, if multiple obstacles exist simultaneously, a spatial cost field can be constructed. This indicates that the robot module is at each location point. The principle behind the construction of "The Cost of Time Travel" is as follows: Base Cost: In unobstructed areas, a low base cost (such as 0 or 1) is set to indicate smooth passage. At the location of obstacles and in their surroundings, a high cost, or even an infinite cost, is assigned through "expansion" or "inflation" to create an obstructed area.

[0089] The effects of multiple obstacles on the cost field can be cumulative, causing the path to deviate from the intersection zone of multiple obstacles. To avoid path oscillation, the cost field can be smoothed using a Gaussian kernel or exponential decay to form a continuous gradient, guiding the path naturally away from high-risk areas for overall path curve generation. For example, the cost function can be: ; Among them, the Effective influence radius of the target obstacle This can be determined by analyzing sensor data and combining it with the coordinates of the target obstacle.

[0090] Gradient descent can be performed on this cost field to find the local path with the lowest cost. ; Then update the target point as follows: ; in, These are the step size control parameters.

[0091] This method enables path planning to have dynamic obstacle avoidance capabilities and is responsive to the distribution of environmental obstacles. The target point is ultimately encapsulated as a sequence of control vectors (i.e., path instructions). , This represents the number of path points in the entire corrected path. After dynamic obstacle avoidance and cost field calculations, a new path is generated, which is... It consists of a series of continuous two-dimensional coordinate points, each It is a path control point, which is the robot module at the _____. The position that the step should reach.

[0092] This control vector sequence can be sent to the AI ​​analysis and defect detection module via the communication bus to dynamically adjust the priority area of ​​the detection task, adjust the camera focus and pitch angle, perform robot pose updates, and avoid potential collision risks.

[0093] In one possible implementation, the AI ​​analysis defect detection module 12 is specifically used for: The coordinate information of the two-dimensional image corresponding to each defect in the pipeline is determined based on the pipeline defect detection results.

[0094] By back-projecting the coordinate information from the two-dimensional image, the three-dimensional spatial coordinates of each defect inside the pipeline can be determined.

[0095] Based on the three-dimensional spatial coordinates of each defect in the pipeline and the target obstacle information, it can be identified whether the defect is located on the feasible path of the robot module or near the obstructed area.

[0096] If the defect is located on the feasible path of the robot module or near the occluded area, the path command is adjusted according to the three-dimensional spatial coordinates of the defect located on the feasible path of the robot module or near the occluded area.

[0097] In this embodiment, the AI ​​analysis defect detection module 12 first performs pipeline defect detection on the fused features to obtain pipeline defect detection results.

[0098] In this embodiment, the AI ​​analysis defect detection module 12 receives target obstacle information with the following structure: ; This structured data includes the spatial location of obstacles, their dynamic trends, and the shortest forward distance information. Based on this, the AI-analyzed defect detection module 12 can comprehensively determine: whether to detour or pause, whether the defect recognition accuracy is affected by obstacle occlusion, and whether the path or camera attitude angle needs to be adjusted.

[0099] Specifically, the pipeline defect detection results obtained based on fusion features can be matched with target obstacle information from the perception and obstacle avoidance module. After processing by the defect recognition model, the fusion features can extract the two-dimensional image coordinate information of potential defects such as cracks, silt, and foreign objects inside the pipeline. Simultaneously, the perception and obstacle avoidance module constructs a spatial distribution model of obstacles using LiDAR and ultrasound, forming a three-dimensional spatial coordinate representation of the obstacles in the world coordinate system. Then, to achieve accurate correlation between the pipeline defect detection results and actual spatial obstacles, the extracted pixel coordinates of the defects in the image plane are back-projected to corresponding spatial points using an intrinsic parameter matrix and depth information. This is then matched with the obstacle spatial information provided by the perception and obstacle avoidance module to identify whether the defect is located on the robot's feasible path or near an occluded area. If interference factors exist, the system will mark the area as "partially visible" and prioritize improving data reliability through path adjustment or multi-angle re-shooting.

[0100] In one possible implementation, the AI ​​analysis defect detection module 12 is specifically used for: Based on the three-dimensional spatial coordinates of each defect within the pipeline, the defect range corresponding to each defect within the pipeline is determined.

[0101] Determine the range of the target obstacle based on the target obstacle information.

[0102] Determine whether the defect range corresponding to each defect in the pipeline overlaps with the target obstacle range.

[0103] If the defect range corresponding to a certain defect in the pipeline overlaps with the target obstacle range, then the defect is determined to be located on the feasible path of the robot module or near the obstructed area.

[0104] In this embodiment, the system first receives obstacle data from the lidar sensing unit and the ultrasonic obstacle detection unit. This data includes the spatial location, distance, size, and whether the obstacle has a movement trend. The system locally determines whether these obstacles are dynamic objects, such as floating objects, rolling objects, or environmental changes caused by robot movement. Such dynamic obstacles are preferentially marked as high-risk areas to guide the attention area during image acquisition or to trigger the robot to adjust its angle and speed.

[0105] Then, the AI-analyzed defect detection module determines the degree of occlusion of the detection area by obstacles. For example, the defect area detected in the image is mapped into a three-dimensional spatial region using point cloud data. .

[0106] The spatial extent of obstacles, given by radar and ultrasound, is represented as follows: .

[0107] Judgment method: If the two regions have spatial overlap: ; There may be obstruction.

[0108] Further assessment of depth value (Z-axis) differences: ; If it is identified as "frontal obstruction", the system will label it accordingly, such as "no obstruction", "partial obstruction", or "complete obstruction".

[0109] The system output includes not only the defect type label (such as crack, corrosion, or blockage) in the current frame, but also the spatial location and confidence level of each type of defect in the image. Based on these factors, the system can classify obstacle interference into the following levels: The defect area and the obstacle do not spatially overlap; or they overlap but the obstacle is behind them, causing no substantial interference to recognition; the obstacle covers part of the defect area, or the depth difference is small, resulting in a decrease in local image quality; the defect area is largely obscured by the obstacle, and the obstacle is in front of the viewer, making effective feature extraction impossible. If an obstacle is found to interfere with defect recognition in a certain area, the system will trigger a dynamic path adjustment mechanism, notifying the robot module to adjust its perspective or direction of travel to avoid the obstruction, optimize the image acquisition angle for the next frame, and improve defect detection quality.

[0110] In one possible implementation, the AI ​​analysis defect detection module 12 is also used for: Based on the pipeline defect detection results and historical pipeline defect detection results, the aging trend of the pipeline is predicted, and the aging prediction results of the pipeline are obtained.

[0111] In this embodiment, after defect identification is completed, the AI-analyzed defect detection module 12 can also perform predictive modeling based on the identification results to estimate pipeline aging trends and potential failure risks. This prediction mainly considers the frequency of defect occurrence, distribution range, morphological complexity, and comparison with historical records. The system can compare previous inspection data with current inspection results to identify newly added, expanded, or recurring defect areas. An aging score is output through an internal scoring mechanism, forming an aging trend curve. The trend results are output in a structured form, along with a rate of change indicator, to support subsequent maintenance decisions for the urban pipeline network management platform. The aging trend prediction results can also trigger automatic adjustments to the inspection frequency or indicate pipe sections requiring higher-precision inspections.

[0112] During the aging prediction process, the deep learning image processing unit constituting the AI ​​analysis defect detection module 12 also possesses temporal dimension analysis capabilities. Specifically, it identifies crack propagation trajectories or corrosion area evolution through continuous frame analysis. If a defect exhibits enlargement or morphological distortion in continuous frames, the system marks it as an "active defect." For corrosion-related defects, combining thermal imaging time-series data allows analysis of temperature distribution trends. If the calorific value continuously decreases or expands in a ring pattern, it indicates potential structural detachment or corrosion layer expansion within the material in that area. Blockage-related defects require verification using robot movement trajectory and obstacle space data. If there is a sudden interruption, image loss, or inability to close the structural point cloud along the detection path, the system infers a physical blockage and can provide feedback to the robot module to trigger detour or secondary detection commands.

[0113] For example, during system operation, each defect identification task can generate an information record with a timestamp and spatial location. The system can compare the current identification result with historical detection records to determine whether it is a newly discovered defect or a change in the state of an existing defect. For instance, if the same location was identified as a small crack three months ago, and this detection shows that it has expanded into a medium-sized crack, the system records the crack expansion rate; if the heat reflectance value of a corroded area decreases significantly, it will also be judged that the corrosion process has intensified.

[0114] By creating time-series trajectories from defect records, the system can manage the lifecycle of each individual defect. This lifecycle data can include the initial discovery time of the defect, the number of historical changes, morphological change trends, historical inspection frequency, and related environmental information (such as humidity, flow rate, and material type). Based on the lifecycle data, the system integrates and analyzes the identified results with historical data to construct an aging trend map and assess the aging of each pipeline segment. This assessment considers not only the number and density of defects but also their development speed, severity, and distribution. The assessment results are categorized into multiple risk levels (e.g., minor, moderate, severe, and requiring immediate intervention) and output to the city's drainage management system's visualization platform for direct dispatch by the operations and maintenance department.

[0115] The AI ​​analysis defect detection module 12 can also update the trend model regularly, automatically refresh historical trend data after each inspection task, and automatically make early warnings or recommend maintenance intervention suggestions based on changes, realizing a closed-loop function from perception to prediction and from identification to decision support.

[0116] Please refer to Figure 2 The present invention also provides an application method for a CCTV-based rainwater and sewage separation pipeline inspection system, comprising: S201, the acquisition module acquires visible light images, infrared images and depth images of the pipeline.

[0117] S202, the processing module extracts features from the visible light image, infrared image and depth image respectively to obtain texture features, thermal anomaly region features and spatial features, and then fuses the texture features, thermal anomaly region features and spatial features based on adaptive weights and feature confidence to obtain fused features.

[0118] S203, the obstacle avoidance module senses obstacles in the pipeline, obtains target obstacle information, and generates path instructions based on the target obstacle information.

[0119] S204, the AI ​​analysis defect detection module detects pipeline defects based on fused features, and performs mutual correction by combining the pipeline defect detection results and target obstacle information, and adjusts the path instructions according to the correction results.

[0120] S205, the robot module moves within the pipe and traverses obstacles based on the adjusted path instructions.

[0121] This invention employs an acquisition module to collect visible light, infrared, and depth images of a pipeline. By integrating multi-source sensing components within the acquisition module, a highly efficient and comprehensive data acquisition system for complex pipeline environments is constructed, ensuring data acquisition quality in low-light, humid, and obstructed environments. Then, a processing module mounted on a robot module extracts features from the visible light, infrared, and depth images to obtain texture features, thermal anomaly region features, and spatial features. These features are then fused based on adaptive weights and feature confidence levels to obtain fused features. This allows the processing module to perform feature extraction and fusion of visible light, infrared, and depth images locally, reducing data transmission pressure, improving the real-time performance of detection feedback, and enhancing the overall system response efficiency. Furthermore, the use of adaptive weights and feature confidence levels enables reliable fusion of visible light, infrared, and depth images, thereby contributing to improved system detection reliability. In addition to detecting pipeline defects based on fused features, the AI-analyzed defect detection module also performs mutual correction by combining pipeline defect detection results and target obstacle information. Based on the correction results, the path instructions are adjusted so that the robot module can move inside the pipeline and overcome obstacles based on the adjusted path instructions. This enhances the coordination between the modules of the CCTV-based rainwater and sewage separation pipeline detection system and further improves the robustness and reliability of rainwater and sewage separation pipeline detection from the perspective of multi-module collaboration.

[0122] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0123] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A CCTV-based rain and sewage diversion pipeline detection system, characterized in that, The application relates to a pipeline inspection robot and an AI analysis defect detection method. The robot module is provided with a collection module, a processing module and a perception obstacle avoidance module. The collection module is used for collecting visible light images, infrared images and depth images of a pipeline. The processing module is used for respectively extracting features of the visible light images, the infrared images and the depth images, obtaining texture features, thermal anomaly region features and space features, and fusing the texture features, the thermal anomaly region features and the space features based on adaptive weights and feature confidence degrees to obtain fused features. The perception obstacle avoidance module is used for perceiving obstacles in the pipeline, obtaining target obstacle information, and generating path instructions based on the target obstacle information. The AI analysis defect detection module is used for detecting pipeline defects according to the fused features, and mutually correcting pipeline defect detection results and the target obstacle information, and adjusting the path instructions according to the correction results. The robot module is used for moving in the pipeline and crossing obstacles based on the adjusted path instructions.

2. The CCTV based detection system for sewer inspection as claimed in claim 1 wherein, The processing module is specifically used for: performing preliminary pipeline defect detection on the texture features, the thermal anomaly region features and the space features to obtain initial pipeline defect detection results; determining feature confidence degrees corresponding to the texture features, the thermal anomaly region features and the space features according to the initial pipeline defect detection results; correcting adaptive weights corresponding to the texture features, the thermal anomaly region features and the space features according to the feature confidence degrees corresponding to the texture features, the thermal anomaly region features and the space features; fusing the texture features, the thermal anomaly region features and the space features according to the corrected adaptive weights corresponding to the texture features, the thermal anomaly region features and the space features to obtain fused features.

3. The CCTV based detection system for storm-sewer pipe of claim 2, wherein, The initial pipeline defect detection results include defect types and defect confidence degrees corresponding to each defect. The processing module is specifically used for: determining basic feature confidence degrees corresponding to the texture features, the thermal anomaly region features and the space features according to image qualities of the visible light images, image qualities of the infrared images and image qualities of the depth images; for each defect type, the proportion of defect confidence degrees of defects under the defect type that are greater than a preset threshold is recorded as a target proportion; according to the correlation degrees of each defect type with the texture features, the thermal anomaly region features and the space features and the target proportions corresponding to each defect type, the basic feature confidence degrees corresponding to the texture features, the thermal anomaly region features and the space features are corrected to obtain feature confidence degrees corresponding to the texture features, the thermal anomaly region features and the space features.

4. The CCTV based detection system for sewer inspection as claimed in claim 1 wherein, The perception obstacle avoidance module is specifically used for: acquiring pipeline point cloud data based on a laser radar perception unit, and constructing a space occupancy grid map according to the pipeline point cloud data. cluster the continuous occupied grids in the space occupied grid map, and obtain obstacle information according to the clustering result, denoted as first obstacle information; obtain second obstacle information based on the ultrasonic obstacle detection unit; calculate a difference value between an estimated distance of a front obstacle in the second obstacle information and a distance of a nearest obstacle in the first obstacle information; if the difference value exceeds a set threshold value, determine the second obstacle information as target obstacle information; if the difference value does not exceed the set threshold value, determine information of the nearest obstacle in the first obstacle information as target obstacle information; generate a path instruction according to the target obstacle information, a risk factor weight corresponding to a type of a target obstacle in the target obstacle information, and a central coordinate where the robot module is currently located.

5. The CCTV based detection system for storm-sewer pipe according to claim 4, wherein, The perception obstacle avoidance module is specifically configured to: if the target obstacle information contains one target obstacle, determine a target offset direction vector according to a coordinate of the target obstacle in the target obstacle information and the central coordinate, calculate a target offset distance according to the target offset direction vector and the risk factor weight, determine a target point according to the target offset distance and the central coordinate, and take the target point as the path instruction; if the target obstacle information contains at least two target obstacles, construct a cost function according to the coordinates of the target obstacles and the corresponding risk factor weights, find a local path with the lowest cost according to the cost function, determine a target point according to the local path and the central coordinate, and take the target point as the path instruction.

6. The CCTV based detection system for storm-sewer pipe of claim 5, wherein, The cost function is: ； in, For point The cost function value, For the first Risk factor weights for each target obstacle , The target obstacle information includes the number of target obstacles. For the first The coordinates of the target obstacle For the first The effective radius of influence of a target obstacle.

7. The CCTV based detection system for sewer inspection as claimed in claim 1 wherein, The AI analysis defect detection module is specifically configured to: determine two-dimensional image coordinate information corresponding to each defect in the pipeline according to a pipeline defect detection result; determine three-dimensional space coordinates corresponding to each defect in the pipeline according to back projection based on the two-dimensional image coordinate information; identify whether a defect is located on a feasible path of the robot module or near an occluded area according to the three-dimensional space coordinates corresponding to each defect in the pipeline and the target obstacle information; if a defect is located on a feasible path of the robot module or near an occluded area, adjust the path instruction according to the three-dimensional space coordinates corresponding to the defect located on the feasible path of the robot module or near the occluded area.

8. The CCTV based detection system for storm-sewer pipe of claim 7, wherein, The AI analysis defect detection module is specifically configured to: determine a defect range corresponding to each defect in the pipeline according to the three-dimensional space coordinates corresponding to each defect in the pipeline; determine a target obstacle range according to the target obstacle information; determine whether the defect range corresponding to each defect in the pipeline overlaps with the target obstacle range; if the defect range corresponding to a certain defect in the pipeline overlaps with the target obstacle range, determine that the defect is located on a feasible path of the robot module or near an occluded area.

9. The CCTV based detection system for sewer inspection as claimed in claim 1 wherein, The AI analysis defect detection module is further configured to: predict an aging trend of the pipeline according to the pipeline defect detection result and a historical pipeline defect detection result, and obtain an aging prediction result of the pipeline.

10. An application method of a CCTV-based rain and sewage diversion pipeline detection system, characterized in that, comprises: a collection module configured to collect a visible light image, an infrared image, and a depth image of the pipeline; The processing module respectively extracts features from the visible light image, the infrared image and the depth image to obtain texture features, thermal anomaly region features and spatial features, and fuses the texture features, the thermal anomaly region features and the spatial features based on adaptive weights and feature confidence to obtain fused features; The perception obstacle avoidance module perceives obstacles in the pipeline to obtain target obstacle information, and generates path instructions based on the target obstacle information; The AI analysis defect detection module detects defects in the pipeline according to the fused features, and mutually corrects the pipeline defect detection result and the target obstacle information, and adjusts the path instructions according to the correction result; The robot module moves in the pipeline and climbs over obstacles based on the adjusted path instructions.

Images