A method and system for assessing the condition of urban drainage pipelines

Abstract

This invention proposes a method and system for assessing the condition of urban drainage pipelines, relating to the fields of computer vision and image processing technology. The scheme includes: acquiring original images of the interior of urban drainage pipelines and constructing a dataset of original pipeline images; inputting the dataset into an MFSC-GANomaly anomaly detection model to extract pipeline residual maps for each original pipeline image; calculating the defect area of ​​each residual map based on the Canny algorithm and morphological operations; inputting the defect area corresponding to the residual map into a defect area calculation model to obtain the actual defect area corresponding to the defect area in the residual map; determining the defect score corresponding to the actual defect area based on the area loss function, the actual defect area, and drainage pipeline defect rating standards; and calculating the pipeline defect score value based on the pipeline defect scoring function and the defect score to determine the structural defect level and defect type of the pipe segment corresponding to the actual defect area. This invention helps improve the accuracy and stability of pipeline condition assessment results.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and in particular to a method and system for assessing the condition of urban drainage pipelines. Background Technology

[0002] Pipeline condition assessment is a comprehensive health evaluation of a pipeline system, aiming to improve operational efficiency, extend equipment lifespan, and reduce potential safety risks and environmental impacts. In CCTV inspection methods, pipeline condition assessment is conducted concurrently with pipeline inspection. Inspectors identify anomalies through pipeline images, qualitatively determine the type and nature of defects, assign scores to defects according to national standards, and then statistically analyze these scores to obtain the condition assessment level for the entire inspected pipe section.

[0003] Chinese patent CN115841466A discloses a data-driven autonomous decision-making method for forging processes, comprising the following steps: classifying drainage pipe defects into five categories; classifying drainage pipe defects into four levels based on their severity; determining the defect level based on the magnitude of the physical quantity α, and then scoring these four levels; determining the area, length, and angle of drainage pipe defects by counting pixels to obtain defect data; using the defect data to measure the size of drainage pipe defects according to the characteristics of these five categories of drainage pipe defects; determining the type, severity, and density of drainage pipe defects based on the level classification and importance scoring; and then establishing a pipe section repair model based on the severity and defect type to evaluate the pipe section repair level. However, the evaluation criteria adopted in the above applications lack quantitative comprehensive evaluation indicators that simultaneously consider multiple defects, making it difficult to compare the health conditions of different pipes and leading to inaccurate evaluation results. Therefore, it is necessary to provide a method and system for assessing the condition of urban drainage pipes to improve the accuracy and stability of pipe condition assessment results. Summary of the Invention

[0004] In view of this, the present invention proposes a method and system for assessing the condition of urban drainage pipelines. By acquiring images of the pipeline interior and constructing a dataset, combined with the MFSC-GANomaly anomaly detection model, the method achieves automated extraction and analysis of pipeline defects, thereby improving the accuracy and stability of pipeline condition assessment results.

[0005] This invention provides a method for assessing the condition of urban drainage pipelines, the method comprising:

[0006] Collect raw images of the interior of urban drainage pipes and construct a raw image dataset of pipes;

[0007] The original pipeline image dataset is input into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each original pipeline image in the original pipeline image dataset, and the defect area of ​​each pipeline residual map is calculated according to the Canny algorithm and morphological operations.

[0008] Input the defect area corresponding to the pipeline residual map into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map.

[0009] Based on the area loss function, the actual defect area, and the drainage pipeline defect rating standard, the defect score corresponding to the actual defect area is determined. Then, based on the pipeline defect scoring function and the defect score, the pipeline defect score value is calculated to determine the structural defect level and defect type of the pipe segment corresponding to the actual defect area.

[0010] Based on the above technical solutions, preferably, the MFSC-GANomaly anomaly detection model includes a generator, an MFSC module, and an anomaly detection module, wherein,

[0011] The generator is used to extract features from the original pipeline images in the input original pipeline image dataset and receive the fusion features output by the MFSC module to obtain the reconstructed pipeline image.

[0012] The MFSC module is used to extract fine-grained low-level features, medium-grained mid-level features, and coarse-grained high-level semantic features respectively, and introduces an attention mechanism to fuse the low-level features, the mid-level features, and the high-level semantic features to obtain fused features.

[0013] The anomaly detection module is used to subtract the reconstructed pipeline image from the original pipeline image to obtain a pipeline residual map.

[0014] Based on the above technical solutions, preferably, the drainage pipeline defect rating standard specifically includes the pipe segment structural defect level, the pipe segment structural defect type, and the drainage pipeline defect level. The pipe segment structural defect level is determined based on the pipe segment damage status function and the pipe segment defect parameters. The pipe segment structural defect type is determined based on the pipe segment defect density function. The drainage pipeline defect level is determined based on the pipeline defect scoring function.

[0015] More preferably, the expression for the pipe section damage status function is:

[0016]

[0017] n = n1 + n2

[0018] S max=max{P i}

[0019] Where S represents the pipe section damage status function, n represents the total number of defects in the pipe residual diagram, n1 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1.5m, n2 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, and P i1 P represents the number of defects with a longitudinal net distance greater than 1.5m in the pipeline residual diagram. i2 The defect fraction in the pipeline residual diagram represents the defect fraction with a longitudinal net distance greater than 1m and not greater than 1.5m, where α represents the structural defect influence coefficient, and P... i S represents the defect score of the i-th defect in the pipeline residual diagram. max This represents the maximum defect score in the pipeline residual diagram.

[0020] More preferably, the expressions for the pipe segment defect density function and the pipe defect scoring function are as follows:

[0021]

[0022] P i (x) = 1.4e -8 x 5 +3.04e -6 x 4 +1.88e -4 x 3 +2.53e -3 x 2 +0.11x+3.65e -2

[0023] Where SM represents the pipe segment defect density function, SL represents the total length of the pipe segment, and L represents the length of the pipe segment under inspection. i1 L represents the length of defects with a longitudinal net distance greater than 1.5m in the pipeline residual diagram. i2 P represents the length of defects with a longitudinal net distance greater than 1m and not greater than 1.5m in the pipeline residual diagram. i (x) represents the pipeline scoring function, i represents the i-th pipeline residual map, and x represents the defect score corresponding to the i-th pipeline residual map.

[0024] More preferably, obtaining the actual defect area corresponding to the defect area in the pipeline residual diagram specifically includes:

[0025] Calculate the actual area represented by each pixel based on the camera parameters and the pipe diameter;

[0026] Based on the defect area and area conversion function of the pipeline residual map, the number of defect pixels corresponding to the defect area in the pipeline residual map is converted into the actual defect area.

[0027] More preferably, the expression of the area transformation function is:

[0028]

[0029] P A =P H ×P V

[0030] A defect =P A ×N

[0031] Among them, P H P represents the horizontal pixel size. V Indicates vertical pixel size, D represents the distance from the camera to the object, and FOV. horizontal FOV represents the horizontal field of view. vertical P represents the vertical field of view. A Indicates the area of ​​a single pixel, N represents the number of pixels corresponding to defects in the pipeline residual map, and A defect This represents the actual area corresponding to the defect in the pipeline residual diagram.

[0032] A second aspect of this application provides a condition assessment system for urban drainage pipelines, the system comprising an image acquisition module, an anomaly detection module, and a pipeline scoring module, wherein...

[0033] The image acquisition module is used to acquire original images of the pipes inside the urban drainage pipes and to construct a dataset of original pipe images.

[0034] The anomaly detection module is used to input the original pipeline image dataset into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each original pipeline image in the original pipeline image dataset, and calculate the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operations. The defect area corresponding to the pipeline residual map is input into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map.

[0035] The pipeline scoring module is used to determine the defect score corresponding to the actual defect area based on the area loss function, the actual defect area, and the drainage pipeline defect rating standard, and to calculate the pipeline defect score value based on the pipeline defect scoring function and the defect score, so as to determine the structural defect level and defect type of the pipe segment corresponding to the actual defect area.

[0036] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory.

[0037] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of a method for assessing the condition of urban drainage pipelines.

[0038] The urban drainage pipeline condition assessment method and system provided by this invention have the following advantages over existing technologies:

[0039] (1) By collecting images inside the pipeline and constructing a dataset, combined with the MFSC-GANomaly anomaly detection model, the automatic extraction and analysis of pipeline defects were realized, improving the accuracy and stability of pipeline condition assessment results. At the same time, the Canny algorithm and morphological operations were used to accurately extract the area features of defects from the pipeline residual map and match the extracted defect area with the actual defect area, further improving the accuracy of defect features. The continuous scoring method based on the area loss function and defect scoring function was adopted, which can reflect the damage degree of pipeline defects more meticulously compared with the traditional discrete scoring system.

[0040] (2) The MFSC module extracts fine-grained low-level features, medium-grained mid-level features and coarse-grained high-level semantic features of the pipeline image respectively, so that the MFSC-GANomaly anomaly detection model can comprehensively capture various texture, structure and semantic information of the pipeline image. The MFSC module introduces an attention mechanism to adaptively weight and fuse features at different scales, which can highlight key features, suppress redundant information and improve feature expression ability. The generator can receive the fused features output by the MFSC module. Compared with the features extracted by the generator encoder directly, the rich features provided by the MFSC module help to improve the reconstruction effect of the generator. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A schematic diagram illustrating the steps of a method for assessing the condition of urban drainage pipelines provided by this invention;

[0043] Figure 2This is a comparative diagram of the two scoring criteria provided by the present invention;

[0044] Figure 3 A schematic diagram illustrating a typical defect in a drainage pipe provided by the present invention;

[0045] Figure 4 A schematic diagram of the framework of the urban drainage pipeline condition assessment system provided by the present invention;

[0046] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0047] Explanation of reference numerals in the attached diagram: 1. Urban drainage pipeline status assessment system; 11. Image acquisition module; 12. Anomaly detection module; 13. Pipeline scoring module; 2. Electronic equipment; 21. Processor; 22. Communication bus; 23. User interface; 24. Network interface; 25. Memory. Detailed Implementation

[0048] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0049] This invention discloses a method for assessing the condition of urban drainage pipelines, with reference to Figure 1 The steps of this method include S1 to S4.

[0050] Step S1: Collect original images of the pipes inside the town's drainage pipes and construct a dataset of original pipe images.

[0051] In this step, high-definition digital cameras or video cameras can be used to capture images of the inside of urban drainage pipelines at actual construction or maintenance sites. Using computer graphics technology, a digital twin model of the urban drainage pipeline is constructed to simulate the internal environment of the pipeline, and pipeline images are rendered to control the type and extent of pipeline defects, generating a fully annotated pipeline image dataset.

[0052] Step S2: Input the original pipeline image dataset into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each original pipeline image in the original pipeline image dataset, and calculate the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operations.

[0053] In this embodiment, the MFSC-GANomaly anomaly detection model includes a generator, an MFSC module, and an anomaly detection module. The generator is used to extract features from the original pipe images in the input original pipe image dataset and receive the fused features output by the MFSC module to obtain the reconstructed pipe image. The MFSC module is used to extract fine-grained low-level features, medium-grained mid-level features, and coarse-grained high-level semantic features respectively, and introduces an attention mechanism to fuse the low-level features, mid-level features, and high-level semantic features to obtain the fused features. The anomaly detection module is used to subtract the original pipe image from the reconstructed pipe image to obtain the pipe residual map.

[0054] Furthermore, to obtain the defect area, the defect shape needs to be obtained. The conventional approach is to extract the defect shape from the pipe image. However, pipe images often contain complex backgrounds and cluttered content, making direct extraction difficult and inaccurate. Therefore, the MFSC-GANomaly model is used for anomaly detection. This model reconstructs the pipe image, and the difference between the reconstructed and original images yields a residual map. This residual map contains pipe defect information, making it easier to extract the defect shape and calculate the defect area. The GANomaly generator's encoder consists of seven convolutional layers, extracting pipe features from the pipe image. In shallow networks, the receptive field is small, and the extracted features are fine-grained information such as color, texture, edges, and corners. These are high-resolution details of the pipe, valuable for the generator to reconstruct the pipe image. As the number of network layers increases, the receptive field also increases, leading to greater overlap and compression of image information. Therefore, deeper networks extract coarse-grained information, containing semantic information about the overall image.

[0055] The MFSC module extracts feature information from the generator encoder at three scales, introduces an attention mechanism to fuse these features, resulting in richer and more detailed features. Finally, it performs cross-connections at a deeper level, using an attention mechanism to fuse the connected features to obtain more expressive features and improve the generator's reconstruction performance. After obtaining the latent vector, the model upsamples the vector to obtain the reconstructed image. The reconstruction performance of the GANomaly model generator on pipe images significantly impacts the pipe anomaly detection results. However, due to the complex background and multi-scale texture features of pipe images, reconstructing pipe images solely using deep features containing more high-level semantics is insufficient to guarantee reconstruction quality, affecting the accuracy and anomaly detection capability of the final anomaly detection. To provide the generator's decoder with richer features, the MFSC module fuses multi-scale features encoded by the encoder module, fully integrating fine-grained and global semantic information from the pipe image, and concatenates and fuses them with the corresponding feature map of the decoder, providing the decoder with more feature information to improve its reconstruction performance.

[0056] Shape extraction from an image is achieved through edge detection. Edges are areas of significant brightness change in an image, manifested as dramatic changes in pixel values ​​in digital images. Pixels with the most drastic value changes are identified as edges. Most edge detection algorithms detect edges by finding the extrema of the first derivative or the zeros of the second derivative. The Canny algorithm is a classic multi-stage detection algorithm with high accuracy, low false alarm rate, and adaptability to different usage environments through parameter settings; therefore, the Canny algorithm was chosen for edge detection. While the Canny algorithm can generally extract defect edges from pipe images completely and clearly, some images still exhibit non-closed edges. To obtain continuously closed defect edges, further morphological operations are performed on the detected edges.

[0057] Defect feature extraction extracts abnormal shapes from images. The quality of the detection result mainly depends on the accuracy of the defect boundaries and the magnitude of the error in the defect area. In defect feature extraction algorithms, both the extracted results and the true values ​​of the defects are represented as sets of pixels.

[0058]

[0059] Among them, C true It is a collection of defective real pixels, C prePrecision (P) and recall (R) represent the set of defective pixels extracted by the model. Precision (P) and recall (R) reflect the overall accuracy of defect extraction, but they do not reflect the degree of fit between defect edges. Boundary precision is an indicator that reflects the degree of matching between the model's predicted boundary and the true boundary. There are several ways to calculate boundary precision. One method is to calculate the proportion of pixels on the detected edge that match the true edge, as shown in the following formula.

[0060]

[0061] Among them, B true B is the set of pixels representing the true edge of the defect. pre It is the set of defect edge pixels extracted by the model.

[0062] Step S3: Input the defect area corresponding to the pipeline residual map into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map.

[0063] This step also includes steps S31 to S32.

[0064] Step S31: Calculate the actual area represented by each pixel based on the camera parameters and the pipe diameter;

[0065] Step S32: Based on the defect area and area conversion function of the pipeline residual map, convert the number of defect pixels corresponding to the defect area in the pipeline residual map into the actual defect area.

[0066] In this embodiment, the area transformation function is expressed as:

[0067]

[0068] P A =P H ×P V

[0069] A defect =P A ×N

[0070] Among them, P H P represents the horizontal pixel size. V Indicates vertical pixel size, D represents the distance from the camera to the object, and FOV. horizontal FOV represents the horizontal field of view. vertical P represents the vertical field of view. A Indicates the area of ​​a single pixel, N represents the number of pixels corresponding to defects in the pipeline residual map, and A defect This represents the actual area corresponding to the defect in the pipeline residual diagram.

[0071] Defect area is chosen as the key feature reflecting the level of defect damage. The area loss ratio S is defined below for classifying pipeline defects. The formula for calculating S is as follows:

[0072]

[0073] Among them, A defect A represents the defect area. pipe This indicates the cross-sectional area of ​​the pipe.

[0074] Drainage Pipeline Defect Rating Standard Table

[0075]

[0076]

[0077] Based on the defect rating criteria in CJJ181-2012, the defect rating standards for drainage pipelines shown in the table above were formulated. The discrete scoring criterion is expressed as a piecewise function. Polynomial fitting of this piecewise function yields the formula for calculating the defect score Pi of the continuous scoring criterion, as shown in the pipeline defect scoring function. The continuous scoring system replaces the discrete scoring system in CJJ181-2012. A comparison of the two scoring criteria is shown in the figure below. Figure 2 As shown.

[0078] Step S4: Based on the area loss function, the actual defect area, and the drainage pipeline defect rating standard, determine the defect score corresponding to the actual defect area, and calculate the pipeline defect score value based on the pipeline defect scoring function and the defect score, so as to determine the structural defect level and defect type of the pipe section corresponding to the actual defect area.

[0079] In this embodiment, the drainage pipeline defect rating standard specifically includes the pipe segment structural defect level, the pipe segment structural defect type, and the drainage pipeline defect level. The pipe segment structural defect level is determined based on the pipe segment damage status function and the pipe segment defect parameters, the pipe segment structural defect type is determined based on the pipe segment defect density function, and the drainage pipeline defect level is determined based on the pipeline defect scoring function.

[0080] Furthermore, the pipeline segment assessment is conducted from both structural and functional perspectives. Structural and functional defects are categorized and statistically analyzed, and assessment parameters are calculated separately to obtain the assessment results. The following section will describe the pipeline assessment process from the perspective of structural defects. After the segment inspection is completed, the defect information of the inspected pipeline segment is statistically analyzed to obtain pipeline damage status parameters and defect density, and the overall condition of the pipeline segment is judged. When two or more defects appear simultaneously within a 1m longitudinal range of the pipeline, the defect scores are superimposed, with a score exceeding 10 points counted as 10 points. Longitudinal clearance refers to the shortest distance between two or more defects along the longitudinal direction of the pipeline. When the longitudinal clearance between two defects in the pipeline is too close, a superposition effect may occur, increasing the risk of pipeline damage. Therefore, when calculating the pipeline segment assessment parameters, weighting is applied based on the longitudinal clearance of the defects.

[0081] The expression for the pipe section damage status function is:

[0082]

[0083] n = n1 + n2

[0084] S max =max{P i}

[0085] Where S represents the pipe section damage status function, n represents the total number of defects in the pipe residual diagram, n1 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1.5m, n2 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, and P i1 P represents the number of defects with a longitudinal clearance greater than 1.5m in the pipeline residual diagram. i2 P represents the number of defects with a longitudinal clearance greater than 1m and not greater than 1.5m in the pipeline residual diagram. α represents the structural defect influence coefficient, which is related to the defect spacing. When the longitudinal clearance is greater than 1m and not greater than 1.5m, α = 1.1. i S represents the defect score of the i-th defect in the pipeline residual diagram. max This represents the maximum defect score in the pipeline residual plot.

[0086] The pipeline damage condition parameter S is a weighted average based on the longitudinal net distance of defects. The maximum pipeline damage condition parameter S max This represents the worst defect level of the pipe segment. The pipe segment defect parameter F is defined as the larger of the two values, i.e., F = max{S, S...} max The damage condition description corresponding to the defect parameter F can be determined by referring to Table 1, the comparison table for the assessment of structural defects in pipe sections.

[0087] Table 1

[0088]

[0089] The expressions for the pipe segment defect density function and the pipe defect scoring function are as follows:

[0090]

[0091] P i (x) = 1.4e -8 x 5 +3.04e -6 x 4 +1.88e -4 x 3 +2.53e -3 x 2 +0.11x+3.65e -2

[0092] Where SM represents the pipe segment defect density function, SL represents the total length of the pipe segment, and L represents the length of the pipe segment under inspection. i1 L represents the length of defects with a longitudinal clearance greater than 1.5m in the pipeline residual diagram. i2 P represents the length of defects in the pipeline residual diagram with a longitudinal net distance greater than 1m and not greater than 1.5m. i (x) represents the pipeline scoring function, i represents the i-th pipeline residual map, and x represents the defect score corresponding to the i-th pipeline residual map.

[0093] Table 2

[0094]

[0095] After obtaining the pipe segment defect parameter F and the pipe segment defect density SM, the structural damage status of the pipeline can be determined by referring to Table 1, the pipe segment structural defect level assessment table, and Table 2, the pipe segment structural defect type assessment table.

[0096] In one example, corrosion, cracking, leakage, disconnection, deposition, and obstructions are typical defects in pipelines, occurring frequently and having a serious impact during inspection. The rating criteria for several defects in the CJJ181-2012 standard are shown in Table 3.

[0097] Table 3

[0098]

[0099] The rating criteria for several typical defects are all related to the defect's geometric dimensions. The rating criteria for corrosion, cracking, and seepage are directly related to the defect area; the size of the defect area directly reflects the damage level. Given the pipe diameter, the dislocation distance can be directly calculated from the dislocation area; therefore, the dislocation area can also indirectly reflect the degree of dislocation. For the two functional defects, sediment and obstruction, the rating criteria are sediment thickness and water flow cross-sectional area loss. These two criteria are not obvious in the radial image of the pipe, but they can be approximated by calculating the defect area. The horizontal width of the pipe increases from bottom to top; the larger the defect area, the larger the horizontal width, the greater the sediment thickness, and the greater the water flow cross-sectional area loss. In general, the damage level of several typical defects can be reflected by the defect area in the circumferential direction of the pipe. Figure 2 Images of several typical defects are shown, demonstrating that the defect area can effectively reflect the level of damage caused by the defect.

[0100] Defect information is extracted from the residual map obtained by subtracting the reconstructed pipe image generated by the MFSC-GANomaly model from the original pipe image. Therefore, the quality of the reconstructed pipe image significantly affects the accuracy of the evaluation results. The model using the MFSC-mid module can generate images with higher structural similarity, but it introduces some defects into the reconstructed image, which affects the accuracy of defect area extraction. Therefore, in the defect feature extraction experiment, the MFSC-GANomaly model using MFSC-small was used for preliminary work. The residual map is further obtained from the reconstructed image of the anomaly image from the MFSC-GANomaly model, and the defect area is extracted using the Canny algorithm and closing operation. The lower and upper threshold bounds of the Canny algorithm are set to 50 and 150, respectively, and the structuring element of the closing operation is designed as a 2*2 rectangle. Experiments were conducted on the anomaly datasets of three pipe datasets, and the results are shown in Table 4.

[0101] Table 4

[0102]

[0103] On all three datasets, the precision was above 0.8, the recall was close to 1, and the boundary precision was also above 0.8. These data demonstrate that the defect detection algorithm can effectively capture abnormal pixels in images, ensuring a high recall rate to prevent false negatives. However, some false positives exist among the extracted defects. This is a sacrifice made to maintain high recall, resulting in a slightly larger calculated defect area and more severe condition assessment results. In this invention, a continuous scoring system is used instead of a discrete scoring system. This prevents the error caused by the overestimation of the defect area from causing a drastic increase in the defect score, and the pipeline assessment results still accurately reflect the pipeline's health status.

[0104] The pipeline defect parameters F and defect density SM are calculated, and the pipeline condition is then assessed against the standards and regulations in Tables 2 and 3. A key task in pipeline condition assessment is to extract the key features of pipeline defects and develop a defect damage level calculation method based on these key features. Therefore, the pipeline assessment process can simplify the classification of pipeline defects and place greater emphasis on assessing the degree of damage caused by these defects. Structural and functional defects can be scored using the same evaluation criteria, and pipeline condition parameters can be calculated uniformly to assess pipeline condition. Furthermore, to improve the accuracy and reliability of the assessment, a continuous scoring system will be considered to more comprehensively describe the nature and impact of pipeline defects.

[0105] In the CE-PIPE pipeline dataset experiment, pipeline condition assessment was conducted using both manual methods and pipeline assessment models. Table 5 shows the defect assessment results for a single pipeline image from the CE-PIPE dataset, illustrating the assessment results for each defective image.

[0106] Table 5

[0107]

[0108] As shown in Table 5, the proposed continuous defect rating system is largely consistent with the results of the manual method in defect level identification, with only a misclassification in frame 17, where a level 2 defect was incorrectly classified as a level 1 defect. This is because the defect damage parameters at that location are close to the threshold between level 1 and level 2 defects. Compared to the traditional discrete rating method, the continuous rating method provides more information on pipeline defects and can more intuitively reflect the damage level of defects.

[0109] After obtaining the pipeline defect score, statistics are performed to quantify the overall condition of the pipe segment. In the CJJ181-2012 standard, 1m is used as the smallest unit for calculating pipe segment defects. Multiple defect scores within a 1m longitudinal section are superimposed for calculation. When the longitudinal dimension of a defect is no greater than 1m, the defect length is counted as 1m. The pipe inner diameter is 800mm. The camera unit uses a horizontal field of view of 80 degrees and a vertical field of view of 50 degrees, resulting in a pipe image size of 671mm*373mm. Since the input to the anomaly detection model maintains the same size in both length and width, the maximum size of the diagnostic image is 373mm. To achieve 360-degree circumferential acquisition of the pipe's inner wall, the center of each image is offset by 40 degrees during acquisition, using 9 images to capture the complete inner wall of the pipe. Pipeline information within 1m is represented by 27 consecutive images. Next, defects in the pipeline are statistically analyzed in 1-meter units.

[0110] Table 6 shows the defect score statistics in units of 1 meter.

[0111] Table 6

[0112]

[0113] The results in Table 6 show that both the 0-1 meter and 2-3 meter pipe sections have a discrete score of 3. However, the continuous scoring indicates that the 2-3 meter pipe section has more severe damage. By statistically analyzing the defects over 1 meter intervals, the defect parameters F and defect density SM of the entire pipe section can be calculated.

[0114] Table 7 shows the calculation results of pipeline damage parameters.

[0115] Table 7

[0116]

[0117] According to the CJJ181-2012 standard, the pipeline assessment method proposed in this scheme is consistent with the results of manual assessment. The defect level of the inspected pipe section is level two, and the pipeline defect type belongs to overall defect. This case verifies the effectiveness and reliability of the proposed pipeline assessment model, which can obtain results consistent with manual assessment. At the same time, the continuous scoring strategy adopted for calculating pipeline damage parameters can provide detailed information on pipeline damage.

[0118] By acquiring images of the pipeline interior and constructing a dataset, combined with the MFSC-GANomaly anomaly detection model, automated extraction and analysis of pipeline defects were achieved, improving the accuracy and stability of the evaluation results. Simultaneously, the Canny algorithm and morphological operations were used to accurately extract the area features of defects from the pipeline residual image and correlate the extracted defect area with the actual defect area, further improving the accuracy of defect features. A continuous scoring method based on the area loss function and defect scoring function was adopted, which, compared to traditional discrete scoring systems, can more meticulously reflect the degree of damage to pipeline defects.

[0119] Based on the above method, this application discloses an urban drainage pipeline condition assessment system, with reference to... Figure 3 The urban drainage pipeline status assessment system 1 includes an image acquisition module 11, an anomaly detection module 12, and a pipeline scoring module 13, wherein...

[0120] The image acquisition module 11 is used to acquire original images of the pipes inside the urban drainage pipes and to construct a dataset of original pipe images;

[0121] The anomaly detection module 12 is used to input the original pipeline image dataset into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each original pipeline image in the original pipeline image dataset, and calculate the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operations. The defect area corresponding to the pipeline residual map is input into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map.

[0122] The pipeline scoring module 13 is used to determine the defect score corresponding to the actual defect area based on the area loss function, the actual defect area and the drainage pipeline defect rating standard, and to calculate the pipeline defect score value based on the pipeline defect scoring function and the defect score, so as to determine the structural defect level and defect type of the pipe section corresponding to the actual defect area.

[0123] In one example, the MFSC-GANomaly anomaly detection model includes a generator, an MFSC module, and an anomaly detection module, wherein,

[0124] The generator is used to extract features from the original pipe images in the input original pipe image dataset and receives the fused features output by the MFSC module to obtain the reconstructed pipe image.

[0125] The MFSC module is used to extract fine-grained low-level features, medium-grained mid-level features, and coarse-grained high-level semantic features respectively, and introduces an attention mechanism to fuse the low-level features, mid-level features, and high-level semantic features to obtain fused features.

[0126] The anomaly detection module is used to subtract the original pipeline image from the reconstructed pipeline image to obtain the pipeline residual map.

[0127] In one example, the drainage pipeline defect rating standard specifically includes the pipe segment structural defect level, the pipe segment structural defect type, and the drainage pipeline defect level. The pipe segment structural defect level is determined based on the pipe segment damage status function and the pipe segment defect parameters, the pipe segment structural defect type is determined based on the pipe segment defect density function, and the drainage pipeline defect level is determined based on the pipeline defect scoring function.

[0128] In one example, the expression for the pipe segment damage status function is:

[0129]

[0130] n = n1 + n2

[0131] S max =max{P i}

[0132] Where S represents the pipe section damage status function, n represents the total number of defects in the pipe residual diagram, n1 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1.5m, n2 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, and P i1 P represents the number of defects with a longitudinal clearance greater than 1.5m in the pipeline residual diagram. i2 P represents the number of defects with a longitudinal net distance greater than 1m and not greater than 1.5m in the pipeline residual diagram, α represents the structural defect influence coefficient, and P represents the defect fraction. i S represents the defect score of the i-th defect in the pipeline residual diagram. max This represents the maximum defect score in the pipeline residual plot.

[0133] In one example, the expressions for the pipe segment defect density function and the pipe defect scoring function are as follows:

[0134]

[0135] P i (x) = 1.4e -8 x 5 +3.04e -6 x 4 +1.88e -4 x 3 +2.53e -3 x 2 +0.11x+3.65e -2

[0136] Where SM represents the pipe segment defect density function, SL represents the total length of the pipe segment, and L represents the length of the pipe segment under inspection. i1 L represents the length of defects with a longitudinal clearance greater than 1.5m in the pipeline residual diagram. i2 P represents the length of defects in the pipeline residual diagram with a longitudinal net distance greater than 1m and not greater than 1.5m. i (x) represents the pipeline scoring function, i represents the i-th pipeline residual map, and x represents the defect score corresponding to the i-th pipeline residual map.

[0137] In one example, the anomaly detection module 12 is used to calculate the actual area represented by each pixel based on the camera parameters and the pipe diameter, and convert the number of defect pixels corresponding to the defect area in the pipe residual map into the actual defect area based on the defect area and area conversion function of the pipe residual map.

[0138] In one example, the area transformation function is expressed as:

[0139]

[0140] P A =P H×P V

[0141] A defect =P A ×N

[0142] Among them, P H P represents the horizontal pixel size. V Indicates vertical pixel size, D represents the distance from the camera to the object, and FOV. horizontal FOV represents the horizontal field of view. vertical P represents the vertical field of view. A Indicates the area of ​​a single pixel, N represents the number of pixels corresponding to defects in the pipeline residual map, and A defect This represents the actual area corresponding to the defect in the pipeline residual diagram.

[0143] Please see Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 2 may include: at least one processor 21, at least one network interface 24, user interface 23, memory 25, and at least one communication bus 22.

[0144] The communication bus 22 is used to enable communication between these components.

[0145] The user interface 23 may include a display screen and a camera. Optionally, the user interface 23 may also include a standard wired interface and a wireless interface.

[0146] The network interface 24 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0147] The processor 21 may include one or more processing cores. The processor 21 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 25, and by calling data stored in the memory 25. Optionally, the processor 21 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 21.

[0148] The memory 25 may include random access memory (RAM) or read-only memory. Optionally, the memory 25 may include non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 25 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 25 may also be at least one storage device located remotely from the aforementioned processor 21. Figure 4 As shown, the memory 25, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for assessing the condition of urban drainage pipelines.

[0149] exist Figure 5In the electronic device 2 shown, the user interface 23 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 21 can be used to call an application program stored in the memory 25 for a method of assessing the status of urban drainage pipelines. When executed by one or more processors, the electronic device performs one or more methods as described in the above embodiments.

[0150] A computer-readable storage medium storing instructions that, when executed by one or more processors, cause a computer to perform one or more methods as described in the embodiments above.

[0151] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0156] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0157] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for assessing the condition of urban drainage pipelines, characterized in that, The method includes: Collect raw images of the interior of urban drainage pipes and construct a raw image dataset of pipes; The original pipeline image dataset is input into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each original pipeline image in the original pipeline image dataset, and the defect area of ​​each pipeline residual map is calculated according to the Canny algorithm and morphological operations. Input the defect area corresponding to the pipeline residual map into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map. Obtaining the actual defect area corresponding to the defect area in the pipeline residual diagram specifically includes: Calculate the actual area represented by each pixel based on the camera parameters and the pipe diameter; Based on the defect area and area conversion function of the pipeline residual map, the number of defect pixels corresponding to the defect area in the pipeline residual map is converted into the actual defect area; The expression for the area transformation function is: ； ； ； ； Among them, P H P represents the horizontal pixel size. V Indicates vertical pixel size, D represents the distance from the camera to the object, and FOV. horizontal FOV represents the horizontal field of view. vertical P represents the vertical field of view. A Indicates the area of ​​a single pixel, N represents the number of pixels corresponding to defects in the pipeline residual map, and A defect This represents the actual area corresponding to the defect in the pipeline residual diagram; Based on the area loss function, the actual defect area, and the drainage pipe defect rating standard, the defect score corresponding to the actual defect area is determined, and the pipe defect score value is calculated based on the pipe defect scoring function and the defect score to determine the structural defect level and defect type of the pipe segment corresponding to the actual defect area. The drainage pipeline defect rating standard specifically includes the pipe segment structural defect level, the pipe segment structural defect type, and the drainage pipeline defect level. The pipe segment structural defect level is determined based on the pipe segment damage status function and the pipe segment defect parameters. The pipe segment structural defect type is determined based on the pipe segment defect density function. The drainage pipeline defect level is determined based on the pipeline defect scoring function. The expression for the pipe section damage status function is: ； ； ； Where S represents the pipe section damage status function, n represents the total number of defects in the pipe residual diagram, n1 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1.5m, n2 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, and P i1 P represents the number of defects with a longitudinal net distance greater than 1.5m in the pipeline residual diagram. i2 The defect fraction in the pipeline residual diagram represents the defect fraction with a longitudinal net distance greater than 1m and not greater than 1.5m, where α represents the structural defect influence coefficient, and P... i This represents the defect score of the i-th defect in the pipeline residual diagram. This represents the maximum defect score in the pipeline residual plot; The expressions for the pipe segment defect density function and the pipe defect scoring function are as follows: ； ； Where SM represents the pipe segment defect density function, SL represents the total length of the pipe segment, and L represents the length of the pipe segment under inspection. i1 L represents the length of defects with a longitudinal net distance greater than 1.5m in the pipeline residual diagram. i2 This indicates the length of defects in the pipeline residual diagram whose longitudinal net distance is greater than 1m and not greater than 1.5m. P i ( x ) represents the pipeline scoring function. i Indicates the first i Zhang's pipeline residual diagram, x Indicates the first i The defect score corresponding to the residual diagram of the pipeline.

2. The method as described in claim 1, characterized in that, The MFSC-GANomaly anomaly detection model includes a generator, an MFSC module, and an anomaly detection module, wherein... The generator is used to extract features from the original pipeline images in the input original pipeline image dataset and receive the fusion features output by the MFSC module to obtain the reconstructed pipeline image. The MFSC module is used to extract fine-grained low-level features, medium-grained mid-level features, and coarse-grained high-level semantic features respectively, and introduces an attention mechanism to fuse the low-level features, the mid-level features, and the high-level semantic features to obtain fused features. The anomaly detection module is used to subtract the reconstructed pipeline image from the original pipeline image to obtain a pipeline residual map.

3. A system for assessing the condition of urban drainage pipelines, characterized in that, The urban drainage pipeline status assessment system (1) includes an image acquisition module (11), an anomaly detection module (12), and a pipeline scoring module (13), wherein, The image acquisition module (11) is used to acquire original images of the pipes inside the urban drainage pipes and to construct a dataset of original pipe images. The anomaly detection module (12) is used to input the original pipeline image dataset into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each original pipeline image in the original pipeline image dataset, and calculate the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operations. The defect area corresponding to the pipeline residual map is input into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map. Obtaining the actual defect area corresponding to the defect area in the pipeline residual diagram specifically includes: Calculate the actual area represented by each pixel based on the camera parameters and the pipe diameter; Based on the defect area and area conversion function of the pipeline residual map, the number of defect pixels corresponding to the defect area in the pipeline residual map is converted into the actual defect area; The expression for the area transformation function is: ； ； ； ； Among them, P H P represents the horizontal pixel size. V Indicates vertical pixel size, D represents the distance from the camera to the object, and FOV. horizontal FOV represents the horizontal field of view. vertical P represents the vertical field of view. A Indicates the area of ​​a single pixel, N represents the number of pixels corresponding to defects in the pipeline residual map, and A defect This represents the actual area corresponding to the defect in the pipeline residual diagram; The pipeline scoring module (13) is used to determine the defect score corresponding to the actual defect area based on the area loss function, the actual defect area and the drainage pipeline defect rating standard, and to calculate the pipeline defect score value based on the pipeline defect scoring function and the defect score, so as to determine the structural defect level and defect type of the pipe section corresponding to the actual defect area. The drainage pipeline defect rating standard specifically includes the pipe segment structural defect level, the pipe segment structural defect type, and the drainage pipeline defect level. The pipe segment structural defect level is determined based on the pipe segment damage status function and the pipe segment defect parameters. The pipe segment structural defect type is determined based on the pipe segment defect density function. The drainage pipeline defect level is determined based on the pipeline defect scoring function. The expression for the pipe section damage status function is: ； ； ； Where S represents the pipe section damage status function, n represents the total number of defects in the pipe residual diagram, n1 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1.5m, n2 represents the number of defects in the pipe residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, and P i1 P represents the number of defects with a longitudinal net distance greater than 1.5m in the pipeline residual diagram. i2 The defect fraction in the pipeline residual diagram represents the defect fraction with a longitudinal net distance greater than 1m and not greater than 1.5m, where α represents the structural defect influence coefficient, and P... i This represents the defect score of the i-th defect in the pipeline residual diagram. This represents the maximum defect score in the pipeline residual plot; The expressions for the pipe segment defect density function and the pipe defect scoring function are as follows: ； ； Where SM represents the pipe segment defect density function, SL represents the total length of the pipe segment, and L represents the length of the pipe segment under inspection. i1 L represents the length of defects with a longitudinal net distance greater than 1.5m in the pipeline residual diagram. i2 This indicates the length of defects in the pipeline residual diagram whose longitudinal net distance is greater than 1m and not greater than 1.5m. P i ( x ) represents the pipeline scoring function. i Indicates the first i Zhang's pipeline residual diagram, x Indicates the first i The defect score corresponding to the residual diagram of the pipeline.

4. An electronic device, characterized in that, The device includes a processor (21), a memory (25), a user interface (23), and a network interface (24). The memory (25) is used to store instructions. The user interface (23) and the network interface (24) are used to communicate with other devices. The processor (21) is used to execute the instructions stored in the memory (25) to cause the electronic device (2) to perform the method as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-2.

Images