Method and system for locating defects in urban drainage network based on multi-model fusion
By integrating image, water quality, and GIS data through a multi-model fusion method and using CNN, GRU, and GNN models for feature fusion, the problem of insufficient defect location accuracy in urban drainage network detection was solved, achieving high-precision defect identification and location, and constructing a self-updating defect location system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HARBIN INSTITUTE OF TECHNOLOGY (SHENZHEN) (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)
- Filing Date
- 2026-03-16
- Publication Date
- 2026-06-16
AI Technical Summary
Existing urban drainage network inspection technologies are limited in scope and have low utilization rates of multi-source heterogeneous data, resulting in insufficient defect location accuracy and failing to meet the high standards of modern urban intelligent operation and maintenance.
A multi-model fusion approach was adopted to construct an image information extraction model based on CNN, a water quality information extraction model based on GRU, and a geographic information extraction model based on GNN. Image, water quality, and GIS data were integrated, feature fusion was performed through an attention mechanism, and a closed-loop control mechanism was established for model optimization.
It has achieved multi-dimensional feature extraction and comprehensive localization of pipeline defects, significantly improving the accuracy of defect identification and localization, and has constructed a defect localization system with self-updating capabilities, ensuring the precise maintenance of urban drainage pipelines.
Smart Images
Figure CN122221152A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of urban drainage network safety operation and maintenance technology, specifically to a method and system for locating defects in urban drainage networks based on multi-model fusion. Background Technology
[0002] As an indispensable and important component of urban infrastructure construction, urban drainage pipe networks have long undertaken the critical tasks of natural drainage of urban rainwater, transportation of domestic and industrial wastewater, and flood control and drainage in summer. The safe and stable operation of the system is directly related to the guarantee of overall urban public safety and order and the quality of life of the vast majority of residents. Most urban drainage pipe network systems are buried deep underground in complex environments. Due to the combined effects of long-term complex geological subsidence, corrosion and aging of pipe materials, and pressure from changes in surface traffic loads, the pipe structure is prone to various defects such as cracking, deformation, misalignment, and blockage. If these hidden defects are not detected and located accurately in a timely manner, it will not only significantly reduce the drainage and transportation efficiency of the pipe network system, but may also further lead to serious secondary disasters such as road collapse.
[0003] Existing technologies for detecting defects in urban drainage pipe networks mainly rely on traditional manual inspections in manholes or the use of single-type sensors for localized detection. Traditional manual inspections are extremely inefficient and pose serious safety hazards in the harsh underground environment. On the other hand, single-sensor detection technologies can only obtain single-dimensional information about a local part of the pipe network, making it difficult to comprehensively and objectively reflect the overall operating status of the pipe network. In complex pipe network environments, the defect identification rate is low and the positioning accuracy is seriously insufficient, failing to meet the high standards of modern urban intelligent operation and maintenance. Summary of the Invention
[0004] The purpose of this invention is to provide a method and system for locating defects in urban drainage pipe networks based on multi-model fusion, so as to solve the problems of single detection methods, low utilization rate of multi-source heterogeneous data, and insufficient defect location accuracy in complex pipe network environments in the existing technology.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] The urban drainage network defect localization method based on multi-model fusion includes the following steps: S1: Construct a multi-source heterogeneous database and collect internal image data, real-time water quality parameter data and spatial topology data of the network. S2: Construct image information extraction models based on CNN, water quality information extraction models based on GRU, and geographic information extraction models based on GNN to extract visual features, temporal features, and topological features respectively; S3: Perform multimodal fusion of the extracted visual features, temporal features and topological features to generate a global feature description, and output the defect localization result based on the global feature description; S4: Establish a feedback optimization mechanism and collect validation data to iteratively update the model.
[0007] Further, S1 includes: S11: Use acquisition equipment to photograph the inside of the pipeline and collect image data including at least one of the following conditions: pipeline cracks, corrosion, and foreign object intrusion. S12: Real-time collection of water quality parameters including at least one of pH value, dissolved oxygen, oxidation-reduction potential, conductivity, turbidity and suspended solids concentration by online water quality sensors deployed at key nodes of the pipeline network; S13: Extract spatial topology data of the pipeline network from the geographic information system, including at least one of the following: pipe segment length, pipe diameter, burial depth, and pipe material.
[0008] Furthermore, the data acquisition device is a CCTV inspection robot or drone equipped with a high-definition camera.
[0009] Further, S2 includes: S21: Construct an image information extraction model based on CNN to identify features of pipe wall cracks, corrosion, and foreign object intrusion; S22: Construct a water quality information extraction model based on GRU, learn the temporal variation law of water quality parameters, and extract the characteristics of abnormal fluctuations in water quality; S23: Construct a geographic information extraction model based on GNN to transform the pipeline network topology into graph data and extract spatial dependencies and structural features.
[0010] Further, S3 includes: S31: Unify the feature vectors extracted by CNN, GRU, and GNN to the same feature space dimension through linear mapping; S32: The attention mechanism is used to calculate the weight coefficients of each feature vector, and the weighted fusion is performed to generate a global feature description; S33: Input the global feature description into the fully connected layer and output the defect location coordinates, defect type, and confidence level.
[0011] Furthermore, the formula for calculating the weight coefficients of the attention mechanism is as follows:
[0012] in, Indicates the first The weight coefficients of each feature vector. , Indicates the first Attention score of each feature vector. This represents a learnable query vector. Indicates the first 1 eigenvector; This represents the sum of the exponents of the attention scores for all feature vectors.
[0013] Further, S4 includes: S41: Push the defect location results output by the model to the operation and maintenance terminal for on-site verification or secondary testing; S42: Collect data on false alarms, missed alarms, and positioning deviations; analyze the causes of errors and sample characteristics. S43: Archive the validation data into the training database to expand the sample library; S44: Fine-tune and train CNN, GRU, and GNN models using expanded samples, and update model parameters; S45: Dynamically adjust the weight parameters of the fusion stage based on the fine-tuned model performance to optimize the fusion strategy.
[0014] Furthermore, fine-tuning training employs the gradient descent algorithm, with the parameter update formula as follows:
[0015] in, This represents the updated set of model parameters. This represents the set of model parameters before the update. Represents the gradient. Indicates based on parameters The loss function.
[0016] Furthermore, the global feature description vector is input into the Softmax classifier for forward computation, and the output includes the type label of the pipeline defect, the specific location coordinates, and the confidence value of the prediction result. The high-confidence defect location result is selected through a threshold judgment mechanism, and non-maximum suppression is applied to the output result to remove duplicate detection boxes.
[0017] A multi-model fusion-based urban drainage network defect location system includes: The data acquisition unit includes an image acquisition module, a water quality monitoring module, and a pipeline GIS topology acquisition module, which are used to acquire images of the pipeline network, water quality parameters, and geographic information data, respectively. The multi-model building unit includes a CNN-based image information extraction model, a GRU-based water quality information extraction model, and a GNN-based geographic information extraction model. The fusion unit is used to perform multimodal fusion of visual features, temporal features and topological features to generate a global feature description; The closed-loop control unit includes a positioning generation module, which outputs defect positioning results based on global feature description; The feedback optimization module is used to collect validation data, expand the sample library, fine-tune the model, and dynamically adjust the fusion weights.
[0018] The beneficial effects of this invention are: (1) This invention integrates multi-source data from images, water quality, and GIS, and utilizes the technical characteristics of CNN in visual recognition, GRU in temporal prediction, and GNN in spatial topology analysis to integrate feature information from different dimensions. This solves the problem of incomplete detection information from a single data source and completes the multi-dimensional feature extraction and comprehensive localization of pipeline defects. CNN is used to extract apparent defect features such as pipe wall cracks and corrosion, GRU is used to capture the temporal fluctuation patterns of water quality parameters, and GNN is used to mine the spatial topological dependencies between pipeline nodes, thus achieving a comprehensive perception of the pipeline network's operating status from three dimensions: visual, temporal, and spatial.
[0019] (2) An attention mechanism is used to adaptively weight and fuse visual features, temporal features and topological features. This can dynamically adjust the contribution weight of each modal feature according to different pipeline network scenarios, generate a more discriminative global feature description, and significantly improve the model’s defect identification accuracy and positioning accuracy in complex environments.
[0020] (3) Through a closed-loop control mechanism, the system uses operation and maintenance verification data to fine-tune the model and update its parameters, enabling the model to adapt to changes in the pipe network environment, continuously correct positioning deviations, and optimize fusion strategies. A defect location system with self-updating capabilities has been constructed to ensure the accuracy of the multi-model fusion output results, providing data support and technical assurance for the precise maintenance of urban drainage pipe networks. Attached Figure Description
[0021] Figure 1 This is a flowchart illustrating the urban drainage network defect location method based on multi-model fusion of the present invention. Figure 2 This is a schematic diagram of the module composition of an urban drainage network defect location system based on multi-model fusion. Detailed Implementation The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.
[0022] Embodiments of the present invention: like Figure 1 and 2As shown, a method for locating defects in urban drainage pipe networks based on multi-model fusion includes the following steps: S1: Data Acquisition A multi-source heterogeneous database is constructed, integrating internal pipeline images, real-time water quality parameters, and geospatial information to ensure the comprehensiveness and timeliness of the data. This provides basic data support for subsequent model training and inference. The accuracy of pipeline defect detection is improved through multi-source data fusion. At the same time, a unified data storage and management mechanism is established. The raw data is cleaned and preprocessed to remove noise interference, and data from different sources are synchronized in time and aligned spatially to form a standardized dataset for model use, enabling efficient data retrieval and fast loading.
[0023] Further, step S1 includes: S11: Using CCTV inspection robots or drones equipped with high-definition cameras, enter the drainage pipe network, collect high-definition static image data of the inner surface of the pipe wall according to the preset path, and transmit the image data to the storage server in real time. At the same time, the device location information and timestamp at the time of collection are recorded to ensure the accurate correlation between image data and geographical location information.
[0024] Image data indicators include pipe wall crack texture features, corrosion spot color distribution, foreign object intrusion shape and outline, pipe deformation geometry, interface misalignment offset distance, sediment accumulation height, pipe inner wall illumination intensity, camera motion parameters, image acquisition timestamp information, equipment unique identifier code, image resolution parameters, and image file storage size. Data acquisition is performed using a CCTV inspection robot equipped with a 20-megapixel high-definition industrial camera, acquiring images of the inner surface of the pipe wall at a frequency of 25 frames per second. The image data is then transmitted to a cloud server via a wireless transmission module. The server uses the JPEG algorithm to compress and store the images and establish an index, while also recording the geographical coordinates and ambient lighting parameters at the time of acquisition to ensure the integrity and traceability of the image data.
[0025] S12: By deploying online water quality sensors at key nodes of the pipeline network, water samples are collected at set time intervals to obtain real-time data on six water quality parameters, including pH, dissolved oxygen, oxidation-reduction potential, conductivity, turbidity, and suspended solids concentration. The collected data is then transmitted to the monitoring center database via a wireless network to enable real-time monitoring and early warning of water quality changes.
[0026] The water quality data indicators include six parameters: pH value, dissolved oxygen concentration, oxidation-reduction potential reading, water conductivity parameter, liquid turbidity NTU value, and suspended solids concentration. Data acquisition is carried out by installing multi-parameter water quality analyzers at key nodes of the pipeline network, which automatically collect water quality parameters every 5 minutes. Electrochemical and optical sensors are used for detection, and the analog signals are converted into digital signals and uploaded to the data center. The data center marks abnormal data and stores it in the time-series database. At the same time, the sensors are calibrated regularly to ensure data accuracy and calibration logs are recorded.
[0027] S13: Extract spatial topology data of the pipeline network from the urban geographic information system, obtain four basic information items for each pipe segment: length, diameter, burial depth, and material, construct a spatial attribute database of the pipeline network, and perform unified coordinate transformation on the extracted coordinate data to match the coordinate system of the positioning data of the image acquisition equipment.
[0028] The GIS data indicators include five basic information items reflecting the physical attributes of the pipeline network: the coordinates of the starting and ending nodes of the pipeline segment, the horizontal length of the pipeline, the nominal diameter of the pipeline, the burial depth of the top of the pipeline, and the material of the pipe. Data acquisition is carried out by calling the standard interface of the urban geographic information system, reading the pipeline network topology layer, parsing the geometric attributes and attribute table information in the vector data file, calculating the spatial distance relationship between nodes and storing it in the database, generating the pipeline network spatial topology matrix, and converting the data format to the adjacency list format required by GNN and performing normalization processing.
[0029] S2: Model building, including: S21: CNN Model Construction A CNN-based image information extraction model is constructed. The model takes collected pipeline network image data as input, extracts the apparent defect features of the pipeline wall through multiple convolutional and pooling layers, and outputs image feature vectors. The model is trained using the cross-entropy loss function, and the Adam optimizer is used to update the network weights. The Dropout layer is used to prevent the model from overfitting. Finally, a high-dimensional image feature representation is output, which provides a visual basis for subsequent feature fusion. The model performs transfer learning based on ImageNet pre-trained weights.
[0030] Feature extraction formula:
[0031] in, Indicates the first The output feature map of each convolutional layer contains abstract feature information such as texture, edge, and shape extracted from the image after the convolution operation; This represents a non-linear activation function. In this embodiment, the ReLU function is selected. Its function is to perform a non-linear transformation on the convolution output to solve the problem that linear models cannot express complex mappings. Indicates the first The weight matrix of the convolutional kernel is used to learn effective feature representations of the image by iteratively updating the parameters in the matrix during training using the gradient descent algorithm. This represents the convolution operator, which implements the sliding window operation of the convolution kernel on the input feature map, extracting local feature information of the image through the local receptive field; Indicates the first The output feature map of the layer is the input data of the current layer. This data retains the feature information of the image after being processed by the previous network layers. Indicates the first The layer's bias term vector is added to the convolution result to adjust the bias of neuron activation, thereby improving the model's expressive power.
[0032] S22: GRU Model Construction A time-series prediction model based on GRU, or water quality information extraction model, is constructed. Inputting time-series data of water quality parameters, the model controls information flow through reset and update gates to capture the changing patterns of water quality parameters over time, extracting abnormal fluctuations in water quality, and outputting a time-series feature vector. The model parameters are optimized using the mean squared error loss function to capture long-term and short-term dependencies. Multi-layer stacking enhances the model's ability to express complex time-series patterns. Input data is normalized to accelerate model convergence, and an early stopping strategy is used to prevent overfitting.
[0033] Feature extraction formula:
[0034] in, Indicates time The hidden state output vector encodes all input sequence information from the initial time to the current time step; Indicates time The update gate vector controls the proportion of hidden state information from the previous time step that is passed to the current time step; The element-wise multiplication operator represents two vectors of the same dimension, which perform multiplication on the corresponding elements. Indicates time The hidden state vector, i.e. the memory information of the previous moment, is used to assist in calculating the current state; This represents the hyperbolic tangent activation function, which maps input values to the range of -1 to 1, and is used to introduce a nonlinear transformation. The weight matrix represents the input data, which linearly transforms the current input data to the same feature space as the hidden state. Indicates time The input data vector is the time series data of water quality parameters collected at the current time step; The weight matrix represents the circular connection, which linearly transforms the hidden state information from the previous time step and uses it to calculate the current state. Indicates time The reset gate vector controls the degree to which the hidden state information from the previous time step is ignored when calculating the candidate hidden state.
[0035] S23: GNN Model Construction A geographic information extraction model based on GNN is constructed, which abstracts pipeline nodes and segments into nodes and edges of a graph structure. The model takes node attribute features and edge attribute features as input, aggregates neighborhood node information through graph convolution operation, extracts the spatial dependency relationship and topological structure features of the pipeline network, and outputs a feature vector containing global topological information. The model is trained using the Adam optimizer, uses a graph attention mechanism to dynamically allocate the weights of neighboring nodes, introduces residual connections to solve the gradient vanishing problem in deep graph networks, and performs batch normalization on node features to stabilize the training process.
[0036] Feature extraction formula:
[0037] in, Indicates the first Layer nodes The feature vector is an updated representation that aggregates information from neighboring nodes; This represents a non-linear activation function. In this embodiment, the ReLU function is used to increase the expressive power of the GNN. This represents the summation operator, used to sum nodes. The feature information of all neighboring nodes is accumulated and aggregated. Represents the index of an element in the set of neighboring nodes, representing a node. A specific adjacent node object; Represents a node The set of neighboring nodes, containing nodes... All directly connected upstream and downstream nodes; Represents a node The degree of the node, i.e. The number of connected edges is used to normalize the features; Represents a node The degree of the node, i.e., its degree with neighboring nodes. The number of connected edges is used to stabilize the gradient distribution; Indicates the first The layer's weight matrix is used to linearly transform the features of neighboring nodes to extract high-dimensional features; Indicates the first Layer neighbor nodes The feature vector contains nodes In the Layer attribute information.
[0038] S3: Model Fusion Features extracted from different models are deeply fused, and a multimodal information complementarity strategy is used to generate a global feature description through feature mapping and weighted calculation. The global features are then input into a classifier for calculation, and the specific location coordinates, defect type, and confidence level of pipeline defects are output. The backpropagation algorithm is used to jointly optimize the parameters of all sub-networks to achieve end-to-end defect localization. The defect classification and localization tasks are optimized simultaneously through a multi-task learning mechanism. Different loss weights are set to balance the contribution of each task to model training. The validation set is used to monitor model performance and save the optimal model.
[0039] Further, step S3 includes: S31: Input the visual feature vector extracted by CNN, the temporal feature vector extracted by GRU, and the topological feature vector extracted by GNN into the fully connected layer. Through linear transformation, the above feature vectors are mapped to the feature space of the same dimension to ensure the consistency of different modal features on the numerical scale. Then, L2 normalization is performed on the mapped features to eliminate modulus differences.
[0040] S32: The attention mechanism is used to calculate the weight coefficients of each feature vector. The mapped feature vectors are then weighted and summed according to the weight coefficients to generate a global feature description vector containing multi-source information. This enables the model to adaptively focus on feature modes that are more important for defect discrimination. The weight coefficients are normalized using the Softmax function to ensure numerical stability.
[0041] S3: Input the global feature description vector into the Softmax classifier for forward computation, and output the type label of the pipeline defect, the specific location coordinates, and the confidence value of the prediction result. Use a threshold judgment mechanism to filter out the defect location results with high confidence to reduce the false alarm rate, and perform non-maximum suppression on the output results to remove duplicate detection bounding boxes.
[0042] Formula for model fusion attention mechanism: The attention mechanism first calculates the similarity score between the query vector and each feature vector, and then normalizes it using the Softmax function to obtain the weight coefficients. The formula for calculating the weight coefficients is as follows:
[0043] Indicates the first The attention score of each feature vector measures the similarity between the query vector and the feature vector. This represents a learnable query vector, which is automatically learned through model training and used to select the most relevant feature for the current task from multiple feature vectors. This represents the matrix transpose operator, which converts the query vector into a row vector so that it can be multiplied by the eigenvector; Indicates the first The aligned feature vectors have been mapped to a unified feature space dimension through a fully connected layer; the weight coefficients are obtained by normalizing the attention score using the Softmax function.
[0044] in, Indicates the first The weight coefficients of each feature vector are dynamically calculated through an attention mechanism, reflecting the importance of the feature to the defect localization task in the current scenario. This represents an exponential function with the natural constant e as its base, used to convert attention scores into positive values for normalization calculations; Indicates the first Attention score of each feature vector; This represents the sum of the exponents of the attention scores of all feature vectors, used to normalize the weight coefficients so that their sum is 1; The value represents the total number of feature vectors involved in the fusion. In this embodiment, this value is 3, corresponding to the image feature vector extracted by CNN, the water quality feature vector extracted by GRU, and the geographic information feature vector extracted by GNN, respectively. "j" represents the index of the j-th feature vector involved in the fusion, used to identify feature vectors from different sources. In the attention mechanism, the role of j is to traverse all feature vectors involved in the fusion.
[0045] The final fused feature vector is obtained by weighted summation:
[0046] in, This represents the fused global feature vector, which integrates all features from the image, water quality, and geographic information, and contains multi-dimensional descriptive information about pipeline defects. It is used as input to the classifier for final decision-making. The summation operator is used to accumulate and merge feature vectors from different modalities after weighting, generating a unified feature representation that includes information from multiple sources.
[0047] S4: Closed-loop control Establish a feedback mechanism to continuously optimize the model using actual operation and maintenance results, collect and verify the defect location results output by the model, analyze the causes of errors and expand the sample library, fine-tune the model parameters using new samples, dynamically adjust the weights of the fusion strategy, achieve iterative improvement of model performance and self-evolution of the system, regularly evaluate the model's performance on the latest data and trigger the retraining process to ensure that the model can adapt to changes in the pipeline environment and new defect patterns, forming a closed-loop optimization system of data collection, model training and result feedback to ensure the long-term stable operation of the system.
[0048] Further, step S4 includes: S41: Push the defect location results output by the model to the operation and maintenance terminal. Professional personnel will carry the detection equipment to the designated location for secondary verification to confirm the authenticity of the defect and the accuracy of the location. The verification results will be recorded in the system as feedback data, and on-site photos will be taken as auxiliary verification basis.
[0049] S42: Collect false alarm data, missed alarm data, and positioning deviation data during the verification process; statistically analyze the error distribution; analyze the causes of errors and the corresponding sample characteristic distribution patterns; identify the weak links of the model in specific scenarios or specific types of defects; and generate a detailed error analysis report to guide the subsequent model optimization direction.
[0050] S43: Organize the new validation data and its corresponding real labels, standardize them according to the preset data format, archive them into the training database, expand the diversity of the sample library, focus on adding difficult samples that the model is prone to errors to improve the model's discrimination ability, and conduct strict quality checks on the new data to ensure the accuracy of the labels.
[0051] S44: The original CNN image model, GRU water quality model and GNN geographic information model are fine-tuned using the expanded sample library. The model parameters are updated with a small learning rate, some of the bottom network parameters are frozen to retain the learned general features, only the top network is trained to adapt to the new data, and an early stopping strategy is adopted to prevent the model from overfitting during the fine-tuning process.
[0052] S45: Based on the performance of the fine-tuned model on the validation set, dynamically adjust the weight parameters in the model fusion stage, optimize the feature fusion strategy, realize the self-evolution of the system, find the optimal weight combination configuration through grid search or Bayesian optimization algorithm, and deploy the updated strategy parameters to the production environment to improve positioning accuracy.
[0053] The formula used in step S4 is:
[0054] in, This represents the updated set of model parameters, which includes the weight matrices and bias vector parameters of all layers in CNN, GRU, and GNN. This represents the set of model parameters before the update, i.e., the initial parameter values of the model at the start of fine-tuning training. These parameters come from the weight file saved after the previous round of training. This represents the learning rate parameter, which controls the size of the update step of the model parameters in the gradient direction, and directly determines the convergence speed and the final stability of the model. Indicates the parameter The gradient operator calculates the vector of partial derivatives of the loss function with respect to the model parameters, precisely indicating the fastest descent direction of parameter updates; Indicates based on parameters The loss function value quantifies the degree of difference between the model's prediction and the true label, and is used to guide the optimization and adjustment of model parameters in the direction of reducing error; This represents the subtraction operator, used to subtract the gradient term from the current parameter value, thereby enabling iterative updates of the model parameters toward the minimum of the loss function.
[0055] The urban drainage network defect location system based on multi-model fusion, corresponding to the above method embodiments, includes: The data acquisition unit includes an image acquisition module, a water quality monitoring module, and a pipeline GIS topology acquisition module, which are used to acquire images of the pipeline network, water quality parameters, and geographic information data, respectively. The multi-model building unit includes a CNN-based image information extraction model, a GRU-based water quality information extraction model, and a GNN-based geographic information extraction model. The fusion unit is used to perform multimodal fusion of visual features, temporal features and topological features to generate a global feature description; The closed-loop control unit includes a positioning generation module, which outputs defect positioning results based on global feature description; and feedback optimization modules, such as a field command issuance module, a sample archiving expansion module, and a model fine-tuning module, which are used to collect validation data, expand the sample library, fine-tune the model, and dynamically adjust the fusion weights.
[0056] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other.
[0057] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0058] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0059] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for locating defects in urban drainage pipe networks based on multi-model fusion, characterized in that, Includes the following steps: S1: Construct a multi-source heterogeneous database to collect internal image data of the pipeline network, real-time water quality parameter data, and spatial topology data of the pipeline network. S2: Construct image information extraction models based on CNN, water quality information extraction models based on GRU, and geographic information extraction models based on GNN to extract visual features, temporal features, and topological features respectively; S3: Perform multimodal fusion of the extracted visual features, temporal features and topological features to generate a global feature description, and output the defect localization result based on the global feature description; S4: Establish a feedback optimization mechanism and collect validation data to iteratively update the model.
2. The urban drainage network defect location method based on multi-model fusion according to claim 1, characterized in that, S1 includes: S11: Use acquisition equipment to photograph the inside of the pipeline and collect image data including at least one of the following conditions: pipeline cracks, corrosion, and foreign object intrusion. S12: Real-time collection of water quality parameters including at least one of pH value, dissolved oxygen, oxidation-reduction potential, conductivity, turbidity and suspended solids concentration by online water quality sensors deployed at key nodes of the pipeline network; S13: Extract spatial topology data of the pipeline network from the geographic information system, including at least one of the following: pipe segment length, pipe diameter, burial depth, and pipe material.
3. The urban drainage network defect location method based on multi-model fusion according to claim 2, characterized in that: The data acquisition equipment is a CCTV inspection robot or drone equipped with a high-definition camera.
4. The urban drainage network defect location method based on multi-model fusion according to claim 2, characterized in that: S2 includes: S21: Construct an image information extraction model based on CNN to identify features of pipe wall cracks, corrosion, and foreign object intrusion; S22: Construct a water quality information extraction model based on GRU, learn the temporal variation law of water quality parameters, and extract the characteristics of abnormal fluctuations in water quality; S23: Construct a geographic information extraction model based on GNN to transform the pipeline network topology into graph data and extract spatial dependencies and structural features.
5. The urban drainage network defect location method based on multi-model fusion according to claim 4, characterized in that: S3 includes: S31: Unify the feature vectors extracted by CNN, GRU, and GNN to the same feature space dimension through linear mapping; S32: The attention mechanism is used to calculate the weight coefficients of each feature vector, and the weighted fusion is performed to generate a global feature description; S33: Input the global feature description into the fully connected layer and output the defect location coordinates, defect type, and confidence level.
6. The urban drainage network defect location method based on multi-model fusion according to claim 5, characterized in that: The formula for calculating the weight coefficients of the attention mechanism is as follows: in, Indicates the first The weight coefficients of each feature vector. , Indicates the first Attention score of each feature vector. This represents a learnable query vector. Indicates the first 1 eigenvector; This represents the sum of the exponents of the attention scores for all feature vectors.
7. The urban drainage network defect location method based on multi-model fusion according to claim 1, characterized in that: S4 includes: S41: Push the defect location results output by the model to the operation and maintenance terminal for on-site verification or secondary testing; S42: Collect data on false alarms, missed alarms, and positioning deviations; analyze the causes of errors and sample characteristics. S43: Archive the validation data into the training database to expand the sample library; S44: Fine-tune the training of CNN, GRU, and GNN models using expanded samples and update the model parameters; S45: Dynamically adjust the weight parameters of the fusion stage based on the fine-tuned model performance to optimize the fusion strategy.
8. The urban drainage network defect location method based on multi-model fusion according to claim 7, characterized in that: Fine-tuning training uses the gradient descent algorithm, with the parameter update formula as follows: in, This represents the updated set of model parameters. This represents the set of model parameters before the update. Represents the gradient. Indicates based on parameters The loss function.
9. The urban drainage network defect location method based on multi-model fusion according to claim 5, characterized in that: The global feature description vector is input into the Softmax classifier for forward computation, and the output includes the type label of the pipeline defect, the specific location coordinates, and the confidence value of the prediction result. A threshold judgment mechanism is used to filter out the defect location results with high confidence, and non-maximum suppression is applied to the output results to remove duplicate detection boxes.
10. A multi-model fusion-based urban drainage network defect location system, characterized in that, include: The data acquisition unit includes an image acquisition module, a water quality monitoring module, and a pipeline GIS topology acquisition module, which are used to acquire images of the pipeline network, water quality parameters, and geographic information data, respectively. The multi-model building unit includes an image information extraction model based on CNN, a water quality information extraction model based on GRU, and a geographic information extraction model based on GNN. The fusion unit is used to perform multimodal fusion of visual features, temporal features and topological features to generate a global feature description; The closed-loop control unit includes a positioning generation module, which outputs defect positioning results based on global feature description; The feedback optimization module is used to collect validation data, expand the sample library, fine-tune the model, and dynamically adjust the fusion weights.