Urban drainage pipeline state evaluation method and system

Abstract

The invention provides an urban drainage pipeline state evaluation method and system, and relates to the technical field of computer vision and image processing, and the method comprises the steps: collecting an original image of a pipeline in an urban drainage pipeline, and constructing a pipeline original image data set; inputting the pipeline original image data set into an MFSC-GANopen anomaly detection model to extract a pipeline residual image of each pipeline original image in the pipeline original image data set, and calculating a defect area of each pipeline residual image according to a Canny algorithm and morphological operation; inputting a defect area corresponding to the pipeline residual image into a defect area calculation model to obtain an actual defect area corresponding to the defect area in the pipeline residual image; and determining a defect score corresponding to the actual defect area according to the area loss function, the actual defect area and a drainage pipeline defect rating standard, and calculating a pipeline defect rating value based on the pipeline defect rating function and the defect score so as to determine a structural defect grade and a defect type of the pipe section corresponding to the actual defect area. According to the invention, the accuracy and stability of a pipeline state evaluation result can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a method and system for evaluating the state of urban drainage pipelines. Background Art

[0002] Pipeline condition assessment is a comprehensive health assessment of the pipeline system, which aims to improve the operating efficiency of the pipeline system, extend the service life of equipment, and reduce potential safety risks and environmental impacts. In the CCTV inspection method, pipeline condition assessment is carried out simultaneously with pipeline inspection. Inspectors find abnormalities through pipeline images, make qualitative judgments on the type and nature of defects, assign defect scores according to the grade standards in the national standard, and then conduct statistical analysis on the scores to obtain the condition assessment grade of the entire inspected pipe section.

[0003] The Chinese patent with publication number CN115841466A discloses a data-driven autonomous decision-making method for a forging process, which includes the following steps: classifying drainage pipe defect types into five categories; classifying drainage pipe defects into four levels according to their severity; determining the defect level according to the size of the physical quantity α value, and then scoring according to these four levels; determining the area, length and angle of drainage pipe defects by counting pixels, obtaining defect data, and using the defect data to measure the size of drainage pipe defects according to the characteristics of these five types of drainage pipe defects; determining the type, severity, and defect density of drainage pipe defects according to the level classification and importance of drainage pipe defects, and then establishing a pipe section repair model based on the severity and defect type to evaluate the pipe section repair level. However, the evaluation criteria adopted in the above applications lack quantitative comprehensive evaluation indicators that consider multiple defects at the same time, resulting in a lack of comparability between the health conditions of different pipes, resulting in inaccurate evaluation results. Therefore, it is very necessary to provide a method and system for evaluating the status of urban drainage pipes to improve the accuracy and stability of pipeline status evaluation results. Summary of the invention

[0004] In view of this, the present invention proposes a method and system for evaluating the status of urban drainage pipelines. By collecting internal images of pipelines and constructing a data set, combined with the MFSC-GANomaly anomaly detection model, the automatic extraction and analysis of pipeline defects are realized, and the accuracy and stability of pipeline status evaluation results are improved.

[0005] The present invention provides a method for evaluating the state of urban drainage pipelines, the method comprising:

[0006] Collect original images of urban drainage pipes and construct a dataset of original pipe images;

[0007] Inputting the pipeline original image data set into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each pipeline original image in the pipeline original image data set, and calculating the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operation;

[0008] Inputting the defect area corresponding to the pipeline residual map into a defect area calculation model to obtain an actual defect area corresponding to the defect area in the pipeline residual map;

[0009] According to the area loss function, the actual defect area and the drainage pipe defect rating standard, the defect score corresponding to the actual defect area is determined, and based on the pipeline defect scoring function and the defect score, the pipeline defect scoring value is calculated to determine the structural defect grade and defect type of the pipe section corresponding to the actual defect area.

[0010] On the basis of the above technical solution, preferably, the MFSC-GANomaly anomaly detection model includes a generator, an MFSC module and an anomaly detection module, wherein:

[0011] The generator is used to perform a feature extraction operation on the pipeline original image in the input pipeline original image data set, and receive the fusion features output by the MFSC module to obtain a pipeline reconstructed image;

[0012] The MFSC module is used to extract fine-grained low-level features, medium-grained middle-level features, and coarse-grained high-level semantic features, and introduce an attention mechanism to fuse the low-level features, the middle-level features, and the high-level semantic features to obtain fused features;

[0013] The anomaly detection module is used to perform a subtraction between the pipeline reconstructed image and the pipeline original image to obtain a pipeline residual map.

[0014] On the basis of the above technical scheme, preferably, the drainage pipe defect rating standard specifically includes the pipe segment structural defect grade, the pipe segment structural defect type and the drainage pipe defect grade, wherein the pipe segment structural defect grade is determined according to the pipe segment damage condition function and the pipe segment defect parameters, the pipe segment structural defect type is determined according to the pipe segment defect density function, and the drainage pipe defect grade is determined according to the pipe defect scoring function.

[0015] More preferably, the expression of the pipe section damage condition function is:

[0016]

[0017] n=n1+n2

[0018] S max=max{P i}

[0019] Where S represents the pipe section damage function, n represents the total number of defects in the pipeline residual map, n1 represents the number of defects with a longitudinal clearance greater than 1.5 m in the pipeline residual map, n2 represents the number of defects with a longitudinal clearance greater than 1 m and not greater than 1.5 m in the pipeline residual map, P i1 It represents the defect fraction with longitudinal clearance greater than 1.5m in the pipeline residual diagram, P i2 represents the defect fraction with a longitudinal clearance greater than 1m and not greater than 1.5m in the pipeline residual diagram, α represents the structural defect influence coefficient, P i represents the defect score of the i-th defect in the pipeline residual graph, S max Represents the maximum defect score in the pipeline residual graph.

[0020] More preferably, the expressions of the pipe section defect density function and the pipeline defect score function are respectively:

[0021]

[0022] P i (x) = 1.4e -8 x 5 +3.04e -6 x 4 +1.88e -4 x 3 +2.53e -3 x 2 +0.11x+3.65e -2

[0023] Among them, SM represents the defect density function of the pipe section, SL represents the total length of the pipe section, L represents the length of the inspected pipe section, and L i1 Indicates the defect length of the pipeline residual diagram with a longitudinal clearance greater than 1.5m, L i2 It indicates the defect length of the pipeline residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, P i (x) represents the pipeline scoring function, i represents the i-th pipeline residual graph, and x represents the defect score corresponding to the i-th pipeline residual graph.

[0024] More preferably, the obtaining of the actual defect area corresponding to the defect area in the pipeline residual map specifically includes:

[0025] According to the camera parameters and the pipe diameter, the actual area represented by each pixel is calculated;

[0026] Based on the defect area of ​​the pipeline residual image and the area conversion function, the number of defect pixels corresponding to the defect area in the pipeline residual image is converted into the actual defect area.

[0027] More preferably, the area conversion function is expressed as:

[0028]

[0029] P A =P H ×P V

[0030] A defect =P A ×N

[0031] Among them, P H Indicates the horizontal pixel size, P V represents the vertical pixel size, D represents the distance from the camera to the object, FOV horizontal Indicates the horizontal field of view, FOV vertical Represents the vertical field of view, P A represents the area of ​​a single pixel, N represents the number of pixels corresponding to the defect in the pipeline residual map, and A defect Indicates the actual area corresponding to the defect in the pipeline residual diagram.

[0032] In a second aspect of the present application, a system for evaluating the status of urban drainage pipelines is provided. The system comprises an image acquisition module, an anomaly detection module and a pipeline scoring module, wherein:

[0033] The image acquisition module is used to acquire the original pipeline image inside the urban drainage pipeline and construct the pipeline original image data set;

[0034] The anomaly detection module is used to input the pipeline original image data set into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each pipeline original image in the pipeline original image data set, and calculate the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operation, and input the defect area corresponding to the pipeline residual map into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map;

[0035] The pipeline scoring module is used to determine the defect score corresponding to the actual defect area according to the area loss function, the actual defect area and the drainage pipeline defect rating standard, and calculate the pipeline defect scoring value based on the pipeline defect scoring function and the defect score to determine the structural defect level and defect type of the pipe section corresponding to the actual defect area.

[0036] In a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory.

[0037] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of a method for evaluating the status of urban drainage pipelines.

[0038] The urban drainage pipeline status assessment method and system provided by the present invention have the following beneficial effects compared with the prior art:

[0039] (1) By collecting internal images of the pipeline and constructing a data set, combined with the MFSC-GANomaly anomaly detection model, the automatic extraction and analysis of pipeline defects is realized, and the accuracy and stability of pipeline status assessment results are improved. At the same time, the Canny algorithm and morphological operations are used to accurately extract the area characteristics of the defects from the pipeline residual image and correspond the extracted defect area with the actual defect area, further improving the accuracy of the defect characteristics. The continuous scoring method based on the area loss function and the defect scoring function is adopted, which can more carefully reflect the damage degree of pipeline defects compared with the traditional discrete scoring system.

[0040] (2) The MFSC module is used to extract fine-grained low-level features, medium-grained mid-level features, and coarse-grained high-level semantic features of the pipeline image, so that the MFSC-GANomaly anomaly detection model can comprehensively capture the various texture, structure, and semantic information of the pipeline image. The MFSC module introduces an attention mechanism to perform adaptive weighted fusion of features of different scales, which can highlight key features, suppress redundant information, and improve feature expression capabilities. The generator can receive the fused features output by the MFSC module. Compared with directly using the features extracted by the generator encoder, the rich features provided by the MFSC module help improve the reconstruction effect of the generator. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A schematic diagram of the steps of a method for evaluating the state of urban drainage pipes provided by the present invention;

[0043] Figure 2A schematic diagram comparing two scoring criteria provided by the present invention;

[0044] Figure 3 A schematic diagram of typical defects of the drainage pipe provided by the present invention;

[0045] Figure 4 A schematic diagram of the framework of the urban drainage pipeline status assessment system provided by the present invention;

[0046] Figure 5 This is a schematic structural diagram of an electronic device provided by the present invention.

[0047] Explanation of the accompanying drawings: 1. Urban drainage pipeline status assessment system; 11. Image acquisition module; 12. Anomaly detection module; 13. Pipeline scoring module; 2. Electronic device; 21. Processor; 22. Communication bus; 23. User interface; 24. Network interface; 25. Memory. DETAILED DESCRIPTION

[0048] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The present invention discloses a method for evaluating the state of urban drainage pipelines, referring to Figure 1 The method includes steps S1 to S4.

[0050] Step S1, collecting original images of the interior of urban drainage pipes and constructing a data set of original pipe images.

[0051] In this step, a high-definition digital camera or video camera can be used to capture the interior of the urban drainage pipeline at the actual construction or maintenance site. Computer graphics technology is used to build a digital twin urban drainage pipeline model, simulate the internal environment of the pipeline, and render and generate pipeline images to control the type and degree of pipeline defects and generate a well-labeled pipeline image dataset.

[0052] Step S2: Input the pipeline original image dataset into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each pipeline original image in the pipeline original image dataset, and calculate the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operation.

[0053] In this embodiment, the MFSC-GANomaly anomaly detection model includes a generator, an MFSC module and an anomaly detection module, wherein the generator is used to perform a feature extraction operation on a pipeline original image in an input pipeline original image data set, and receive fused features output by the MFSC module to obtain a pipeline reconstructed image; the MFSC module is used to extract fine-grained low-level features, medium-grained middle-level features and coarse-grained high-level semantic features, respectively, and introduce an attention mechanism to fuse the low-level features, middle-level features and high-level semantic features to obtain fused features; the anomaly detection module is used to subtract the pipeline reconstructed image from the pipeline original image to obtain a pipeline residual map.

[0054] Furthermore, if the defect area is to be obtained, the defect shape needs to be obtained. The conventional idea is to extract the defect shape from the pipeline image, but the background and content of the pipeline image are complex, and it is difficult to extract directly from the pipeline image and the accuracy is difficult to guarantee. Therefore, the MFSC-GANomaly model is used to complete the abnormality detection work. The model reconstructs the pipeline image. The residual map can be obtained by subtracting the reconstructed image from the original image. The residual map contains the pipeline defect information. The difficulty of extracting the defect shape and calculating the defect area from the residual map will be reduced. The encoder of the GANomaly generator is composed of 7 convolutional layers to extract pipeline features from the pipeline image. In the shallow network, the receptive field is small, and the extracted features are some fine-grained information, such as color, texture, edge, corners, etc. These are all high-resolution pipeline details information, which are very valuable for the generator to reconstruct the pipeline image. With the increase of the number of network layers, the receptive field continues to increase, and the overlapping area between the receptive fields increases, compressing the image information. Therefore, the features extracted by the deep network are coarse-grained information, which is the semantic information containing the integrity of the image.

[0055] The MFSC module extracts feature information of three scales from the generator encoder, introduces the attention mechanism to fuse the features of the three scales to form features with richer features and more complete details, and finally performs cross-connection in the deep layer, and uses the attention mechanism to fuse the connected features to obtain more expressive features to improve the reconstruction effect of the generator. After obtaining the latent vector, the model obtains the reconstructed image by upsampling the vector. The reconstruction effect of the pipeline image by the GANomaly model generator has a significant impact on the pipeline anomaly detection results. However, due to the complex background of the pipeline image and the presence of multi-scale texture features, it is difficult to ensure the reconstruction effect by reconstructing the pipeline image only through deep features containing more advanced semantics, which affects the accuracy and anomaly capture ability of the final anomaly detection. In order to provide the decoder of the generator with more information-rich features, the MFSC module can fuse the multi-scale features encoded by the encoder module, fully fuse the fine-grained information and global semantic information in the pipeline image, and splice and fuse with the corresponding feature map of the decoder, providing more feature information for the decoder to improve the reconstruction effect of the decoder.

[0056] Extracting shapes from images is achieved through edge detection. Edges are parts of an image where the brightness changes significantly. In digital images, this is reflected as a dramatic change in pixel values. The pixels in the image where the pixel values ​​change most dramatically are identified as edges. Most edge detection algorithms detect edges by finding the extreme points of the first-order derivative of the image or the zero points of the second-order derivative. The Canny algorithm is a classic multi-stage detection algorithm with high accuracy and low false alarm rate. It can adapt the algorithm to different usage environments through parameter settings. Therefore, the Canny algorithm is selected to complete edge detection. The Canny algorithm can basically extract the defective edges in the pipeline image completely and clearly, but there is still a problem of unclosed edges in some images. In order to obtain continuously closed defective edges, further morphological operations will be performed on the detected edges.

[0057] Defect feature extraction extracts abnormal shapes from images. The quality of the detection results is mainly reflected in whether the defect boundary is accurate and the error size of the defect area. In the defect feature extraction algorithm, the extraction results and the true value of the defect are expressed in the form of pixel sets.

[0058]

[0059] Among them, C true is the set of defective real pixels, C preIt is the set of defective pixels extracted by the model. The precision P and recall R reflect the overall accuracy of defect extraction, but do not reflect the degree of match of the defect edge. Boundary accuracy is an indicator that reflects the degree of match between the model's predicted boundary and the true boundary. There are also many ways to calculate boundary accuracy. The boundary accuracy is calculated by detecting the proportion of pixels on the edge that match the true edge. The calculation is shown in the following formula.

[0060]

[0061] Among them, B true is the pixel set of the real edge of the defect, B pre It is the set of defect edge pixels extracted by the model.

[0062] Step S3, inputting the defect area corresponding to the pipeline residual map into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map.

[0063] This step also includes steps S31 to S32.

[0064] Step S31, calculating the actual area represented by each pixel according to the camera parameters and the pipe diameter;

[0065] Step S32: based on the defect area of ​​the pipeline residual image and the area conversion function, the number of defect pixels corresponding to the defect area in the pipeline residual image is converted into the actual defect area.

[0066] In this embodiment, the area conversion function is expressed as:

[0067]

[0068] P A =P H ×P V

[0069] A defect =P A ×N

[0070] Among them, P H Indicates the horizontal pixel size, P V represents the vertical pixel size, D represents the distance from the camera to the object, FOV horizontal Indicates the horizontal field of view, FOV vertical Represents the vertical field of view, P A represents the area of ​​a single pixel, N represents the number of pixels corresponding to the defect in the pipeline residual map, and A defect Indicates the actual area corresponding to the defect in the pipeline residual diagram.

[0071] The defect area is selected as the key feature reflecting the defect damage level. The area loss ratio S is defined below to classify the pipeline defects. The calculation formula of S is expressed as:

[0072]

[0073] Among them, A defect Denotes the defect area, A pipe Represents the cross-sectional area of ​​the pipe.

[0074] Drainage Pipe Defect Rating Standard Table

[0075]

[0076]

[0077] Combined with the defect rating criteria in the CJJ181-2012 regulations, the drainage pipe defect rating standards shown in the table above are formulated. The discrete rating criteria are expressed in the form of a piecewise function. The defect score Pi calculation formula of the continuous rating criteria is obtained by polynomial fitting of the piecewise function, as shown in the pipeline defect rating function. The continuous rating system is used to replace the discrete rating system in the CJJ181-2012 regulations. The comparison chart of the two rating criteria is shown in the figure below. Figure 2 shown.

[0078] Step S4, determining the defect score corresponding to the actual defect area according to the area loss function, the actual defect area and the drainage pipe defect rating standard, and calculating the pipeline defect score value based on the pipeline defect scoring function and the defect score to determine the structural defect grade and defect type of the pipe section corresponding to the actual defect area.

[0079] In this embodiment, the drainage pipe defect rating standard specifically includes the pipe segment structural defect grade, the pipe segment structural defect type and the drainage pipe defect grade, wherein the pipe segment structural defect grade is determined according to the pipe segment damage condition function and the pipe segment defect parameters, the pipe segment structural defect type is determined according to the pipe segment defect density function, and the drainage pipe defect grade is determined according to the pipe defect scoring function.

[0080] Furthermore, the pipe section assessment work is carried out from two perspectives: structural and functional. The structural defects and functional defects are classified and counted, and the pipe section assessment parameters are calculated separately to obtain the assessment results. The following will introduce the pipeline assessment process from the perspective of structural defects. After the section inspection is completed, the defect information of the inspected pipe section will be counted to obtain the pipe section damage condition parameters and defect density, and the overall state of the pipe section will be judged. When two or more defects appear simultaneously within the longitudinal range of 1m of the pipeline, the defect scores are superimposed, and 10 points are counted when it exceeds 10 points. The longitudinal clearance refers to the shortest distance between two or more defects along the longitudinal direction of the pipeline. When the longitudinal clearance of two defects in the pipeline is too close, a superposition effect may occur, increasing the risk of pipeline damage. Therefore, when calculating the pipe section assessment parameters, the longitudinal clearance of the defects will be weighted.

[0081] The expression of the pipe section damage condition function is:

[0082]

[0083] n=n1+n2

[0084] S max =max{P i}

[0085] Among them, S represents the damage condition function of the pipe section, n represents the total number of defects in the pipeline residual diagram, n1 represents the number of defects in the pipeline residual diagram with a longitudinal clearance greater than 1.5m, n2 represents the number of defects in the pipeline residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, P i1 It represents the defect fraction with longitudinal clearance greater than 1.5m in the pipeline residual diagram, P i2 It indicates the defect fraction with longitudinal clearance greater than 1m and less than 1.5m in the pipeline residual diagram. α indicates the structural defect influence coefficient. The structural defect influence coefficient is related to the defect spacing. When the longitudinal clearance is greater than 1m and less than 1.5m, α=1.1, P i represents the defect score of the i-th defect in the pipeline residual graph, S max Represents the maximum defect fraction in the pipe residual plot.

[0086] The pipeline damage condition parameter S is a weighted average of the longitudinal clearance groups of defects. The maximum pipeline damage condition parameter S max represents the worst defect level of the pipe section. The pipe section defect parameter F is defined as the larger value of the two, that is, F = max{S,S max The damage condition description corresponding to the defect parameter F can be determined according to the comparison table of pipe section structural defect grade assessment in Table 1.

[0087] Table 1

[0088]

[0089] The expressions of the pipe section defect density function and pipeline defect score function are:

[0090]

[0091] P i (x) = 1.4e -8 x 5 +3.04e -6 x 4 +1.88e -4 x 3 +2.53e -3 x 2 +0.11x+3.65e -2

[0092] Among them, SM represents the defect density function of the pipe section, SL represents the total length of the pipe section, L represents the length of the inspected pipe section, and L i1 Indicates the defect length of the pipeline residual diagram with a longitudinal clearance greater than 1.5m, L i2 Indicates the defect length of the pipeline residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, P i (x) represents the pipeline scoring function, i represents the i-th pipeline residual graph, and x represents the defect score corresponding to the i-th pipeline residual graph.

[0093] Table 2

[0094]

[0095] After obtaining the pipe segment defect parameter F and the pipe segment defect density SM, the structural damage state of the pipeline can be determined by comparing the pipe segment structural defect grade assessment comparison table in Table 1 and the pipe segment structural defect type assessment table in Table 2.

[0096] In one example, corrosion, rupture, leakage, disconnection, deposition and obstruction are typical defects of pipelines, which occur frequently and have serious impacts during the inspection process. The rating standards for several defects in the CJJ181-2012 regulations are shown in Table 3.

[0097] Table 3

[0098]

[0099] The rating standards for several typical defects are all related to the geometric size of the defects. The rating standards for corrosion, rupture and penetration are all directly related to the defect area, and the size of the defect area directly reflects the damage level of the defect. When the pipe diameter is known, the disconnection distance can be directly calculated based on the disconnection area. Therefore, the disconnection area can also indirectly reflect the degree of disconnection. For the two functional defects of sedimentation and obstacles, the rating standards are sediment thickness and water-passing section loss. These two standards are not obvious in the radial image of the pipeline, but can also be approximately obtained by calculating the defect area. The horizontal width of the pipeline increases from bottom to top. The larger the defect area, the larger the horizontal width, the larger the sediment thickness, and the greater the loss of the water-passing section. In general, the degree of damage of several typical defects can be reflected by the defect area in the circumference of the pipeline. Figure 2 Images of several typical defects are displayed, and it can be seen that the defect area can better reflect the damage level of the defect.

[0100] Defect information is extracted from the residual image obtained by subtracting the pipeline reconstruction image generated by the MFSC-GANomaly model from the original pipeline image. Therefore, the quality of the pipeline reconstruction image also seriously affects the accuracy of the evaluation results. The model using the MFSC-mid module can generate images with higher structural similarity, but some defects are introduced in the reconstructed image, which will affect the accuracy of defect area extraction. Therefore, when performing defect feature extraction experiments, the MFSC-GANomaly model using MFSC-small is used to complete the preparatory work. The reconstructed image of the abnormal image of the MFSC-GANomaly model is further calculated to obtain the residual map, and the defect area is extracted according to the Canny algorithm and the closed operation. The lower and upper thresholds of the Canny algorithm are set to 50 and 150 respectively, and the structural element of the closed operation is designed to be a 2*2 rectangle. Experiments were conducted on the abnormal data sets of the three pipeline data sets, and the results are shown in Table 4.

[0101] Table 4

[0102]

[0103] On the three data sets, the precision is above 0.8, the recall is close to 1, and the boundary accuracy is also above 0.8. The above data show that the defect detection algorithm can better capture abnormal pixels in the image and ensure a high recall rate to prevent missed detection. However, there are some false detections in the extracted defects. This is a sacrifice made to ensure a high recall rate, which will cause the calculated defect area to be slightly larger, resulting in more severe status assessment results. In the present invention, a continuous scoring system is used instead of a discrete scoring system. This error caused by the large calculation of the defect area will not cause a sharp increase in the defect score, and the results of the pipeline assessment can still accurately reflect the health status of the pipeline.

[0104] Calculate the pipe segment defect parameter F and the pipe segment defect density SM, and then identify the pipeline status by comparing the regulations and standards in Table 2 and Table 3. The key task of pipeline status assessment is to extract the key features of pipeline defects and formulate defect damage level calculation based on key features. Therefore, the classification of pipeline defects can be simplified in the pipeline assessment process, and more attention can be paid to the identification of the damage degree of pipeline defects. Structural defects and functional defects are scored with the same evaluation criteria, and the pipe segment status parameters are uniformly calculated to evaluate the pipeline status. At the same time, in order to improve the accuracy and reliability of the assessment, a continuous scoring system will be considered to more comprehensively describe the nature and impact of pipeline defects.

[0105] In the pipeline dataset CE-PIPE experiment, two methods, manual and pipeline assessment model, are used to assess the pipeline status. Table 5 shows the evaluation results of a single defect image in CU-PIPE.

[0106] Table 5

[0107]

[0108] From the data in Table 5, it can be seen that the proposed continuous defect rating system is basically consistent with the results of the manual method in terms of defect level identification. Only in the 17th frame, a misjudgment occurs, and the secondary defect is misclassified as a primary defect. This is because the defect damage parameter of the defect is close to the boundary threshold between the primary defect and the secondary defect. Compared with the traditional discrete scoring method, the continuous scoring method can provide more pipeline defect information and can more intuitively reflect the damage level of the defect.

[0109] After obtaining the pipeline defect score, statistics are performed to quantify the overall status of the pipe section. In the CJJ181-2012 specification, 1m is used as the minimum pipe section defect calculation unit. Multiple defect scores within 1m in the longitudinal direction of the pipeline are superimposed and calculated. When the longitudinal dimension of the defect is not greater than 1m, the defect length is calculated as 1m. The inner diameter of the pipeline is 800mm, and the horizontal and vertical field angles of the camera unit are 80 degrees and 50 degrees respectively. The actual size of the collected pipeline image is 671mm*373mm. Since the input of the anomaly detection model maintains the same size in both length and width, the maximum size of the diagnostic image is 373mm. In order to achieve the acquisition of the 360-degree inner wall of the pipeline circumference, the center of each image is offset by 40 degrees during acquisition, and 9 images are used to capture the complete inner wall of the pipeline. The pipeline information within 1m is represented by 27 consecutive images. Next, the defects of the pipeline are counted in units of 1m.

[0110] Table 6 shows the defect score statistics in units of 1 meter.

[0111] Table 6

[0112]

[0113] From the results in Table 6, we can see that the discrete scores of the two pipe sections of 0-1m and 2-3m are both 3 points, but the continuous scoring shows that the damage of the 2-3m pipe section is more serious. After counting the defects according to 1m, we can calculate the entire pipe section to obtain the pipeline section defect parameter F and pipe section defect density SM.

[0114] Table 7 shows the calculation results of pipeline damage parameters.

[0115] Table 7

[0116]

[0117] Comparing with the CJJ181-2012 standard, it can be seen that the pipeline assessment method proposed in this scheme is consistent with the manual assessment results. The defect level of the inspected pipe section is level 2, and the pipeline defect type is an overall defect. This case verifies the effectiveness and reliability of the proposed pipeline assessment model, which can obtain results consistent with manual assessment. At the same time, the continuous scoring strategy adopted for the calculation results of pipeline damage parameters can provide detailed information on pipeline damage.

[0118] By collecting internal images of the pipeline and constructing a data set, combined with the MFSC-GANomaly anomaly detection model, the automatic extraction and analysis of pipeline defects is realized, and the accuracy and stability of the evaluation results are improved. At the same time, the Canny algorithm and morphological operations are used to accurately extract the area characteristics of the defects from the pipeline residual image and correspond the extracted defect area with the actual defect area, further improving the accuracy of the defect characteristics. The continuous scoring method based on the area loss function and the defect scoring function is adopted, which can reflect the damage degree of pipeline defects more carefully than the traditional discrete scoring system.

[0119] Based on the above method, the embodiment of the present application discloses a system for evaluating the status of urban drainage pipes. Figure 3 The urban drainage pipeline status assessment system 1 includes an image acquisition module 11, an anomaly detection module 12 and a pipeline scoring module 13, wherein:

[0120] The image acquisition module 11 is used to acquire the original image of the inner part of the urban drainage pipe and construct the original image data set of the pipe;

[0121] The anomaly detection module 12 is used to input the pipeline original image data set into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each pipeline original image in the pipeline original image data set, and calculate the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operation, and input the defect area corresponding to the pipeline residual map into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual map;

[0122] The pipeline scoring module 13 is used to determine the defect score corresponding to the actual defect area according to the area loss function, the actual defect area and the drainage pipeline defect rating standard, and calculate the pipeline defect score value based on the pipeline defect scoring function and the defect score to determine the structural defect level and defect type of the pipe section corresponding to the actual defect area.

[0123] In one example, the MFSC-GANomaly anomaly detection model includes a generator, an MFSC module, and an anomaly detection module, wherein:

[0124] The generator is used to extract features from the pipeline original image in the input pipeline original image dataset, and receive the fused features output by the MFSC module to obtain the pipeline reconstructed image;

[0125] The MFSC module is used to extract fine-grained low-level features, medium-grained middle-level features, and coarse-grained high-level semantic features, and introduces an attention mechanism to fuse the low-level features, middle-level features, and high-level semantic features to obtain fused features;

[0126] The anomaly detection module is used to subtract the pipeline reconstructed image from the pipeline original image to obtain the pipeline residual map.

[0127] In one example, the drainage pipe defect rating standard specifically includes the pipe segment structural defect grade, the pipe segment structural defect type and the drainage pipe defect grade, wherein the pipe segment structural defect grade is determined based on the pipe segment damage condition function and the pipe segment defect parameters, the pipe segment structural defect type is determined based on the pipe segment defect density function, and the drainage pipe defect grade is determined based on the pipe defect scoring function.

[0128] In an example, the expression of the pipe segment damage condition function is:

[0129]

[0130] n=n1+n2

[0131] S max =max{P i}

[0132] Among them, S represents the damage condition function of the pipe section, n represents the total number of defects in the pipeline residual diagram, n1 represents the number of defects in the pipeline residual diagram with a longitudinal clearance greater than 1.5m, n2 represents the number of defects in the pipeline residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, P i1 It represents the defect fraction with longitudinal clearance greater than 1.5m in the pipeline residual diagram, P i2 It represents the defect fraction with longitudinal clearance greater than 1m and less than 1.5m in the pipeline residual diagram, α represents the structural defect influence coefficient, P i represents the defect score of the i-th defect in the pipeline residual graph, S max Represents the maximum defect fraction in the pipe residual plot.

[0133] In an example, the expressions of the pipe segment defect density function and the pipeline defect score function are:

[0134]

[0135] P i (x) = 1.4e -8 x 5 +3.04e -6 x 4 +1.88e -4 x 3 +2.53e -3 x 2 +0.11x+3.65e -2

[0136] Among them, SM represents the defect density function of the pipe section, SL represents the total length of the pipe section, L represents the length of the inspected pipe section, and L i1 Indicates the defect length of the pipeline residual diagram with a longitudinal clearance greater than 1.5m, L i2 Indicates the defect length of the pipeline residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, P i (x) represents the pipeline scoring function, i represents the i-th pipeline residual graph, and x represents the defect score corresponding to the i-th pipeline residual graph.

[0137] In one example, the anomaly detection module 12 is used to calculate the actual area represented by each pixel based on the camera parameters and the pipe diameter, and based on the defect area and area conversion function of the pipe residual map, convert the number of defective pixels corresponding to the defect area in the pipe residual map into the actual defect area.

[0138] In one example, the area conversion function is expressed as:

[0139]

[0140] P A =P H×P V

[0141] A defect =P A ×N

[0142] Among them, P H Indicates the horizontal pixel size, P V represents the vertical pixel size, D represents the distance from the camera to the object, FOV horizontal Indicates the horizontal field of view, FOV vertical Represents the vertical field of view, P A represents the area of ​​a single pixel, N represents the number of pixels corresponding to the defect in the pipeline residual map, and A defect Indicates the actual area corresponding to the defect in the pipeline residual diagram.

[0143] See also Figure 4 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 2 may include: at least one processor 21 , at least one network interface 24 , a user interface 23 , a memory 25 , and at least one communication bus 22 .

[0144] The communication bus 22 is used to realize the connection and communication between these components.

[0145] The user interface 23 may include a display screen (Display) and a camera (Camera), and the optional user interface 23 may also include a standard wired interface and a wireless interface.

[0146] The network interface 24 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0147] Among them, the processor 21 may include one or more processing cores. The processor 21 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 25, and calling data stored in the memory 25. Optionally, the processor 21 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 21 can integrate one or more combinations of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 21, and it can be implemented separately through a chip.

[0148] Among them, the memory 25 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 25 includes a non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 25 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 25 may also be optionally at least one storage device located away from the aforementioned processor 21. As Figure 4 As shown, the memory 25 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of a method for evaluating the status of a town drainage pipeline.

[0149] exist Figure 5In the electronic device 2 shown, the user interface 23 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 21 can be used to call an application program for a method for evaluating the status of urban drainage pipes stored in the memory 25. When executed by one or more processors, the electronic device executes one or more methods in the above-mentioned embodiments.

[0150] A computer-readable storage medium stores instructions. When executed by one or more processors, the computer executes one or more methods in the above embodiments.

[0151] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0152] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0157] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, it will be easy for those skilled in the art to think of other embodiments of the present disclosure. This application is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the technical field that are not recorded in the present disclosure.

[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the status of urban drainage pipelines, characterized in that: The method comprises: Collect original images of urban drainage pipes and construct a dataset of original pipe images; Inputting the pipeline original image data set into the MFSC-GANomaly anomaly detection model to extract the pipeline residual map of each pipeline original image in the pipeline original image data set, and calculating the defect area of ​​each pipeline residual map according to the Canny algorithm and morphological operation; Inputting the defect area corresponding to the pipeline residual map into a defect area calculation model to obtain an actual defect area corresponding to the defect area in the pipeline residual map; According to the area loss function, the actual defect area and the drainage pipe defect rating standard, the defect score corresponding to the actual defect area is determined, and based on the pipeline defect scoring function and the defect score, the pipeline defect scoring value is calculated to determine the structural defect grade and defect type of the pipe section corresponding to the actual defect area.

2. The method according to claim 1, characterized in that The MFSC-GANomaly anomaly detection model includes a generator, an MFSC module and an anomaly detection module, wherein: The generator is used to perform a feature extraction operation on the pipeline original image in the input pipeline original image data set, and receive the fusion features output by the MFSC module to obtain a pipeline reconstructed image; The MFSC module is used to extract fine-grained low-level features, medium-grained middle-level features, and coarse-grained high-level semantic features, and introduce an attention mechanism to fuse the low-level features, the middle-level features, and the high-level semantic features to obtain fused features; The anomaly detection module is used to perform a subtraction between the pipeline reconstructed image and the pipeline original image to obtain a pipeline residual map.

3. The method according to claim 1, characterized in that The drainage pipe defect rating standard specifically includes the pipe segment structural defect grade, the pipe segment structural defect type and the drainage pipe defect grade, wherein the pipe segment structural defect grade is determined according to the pipe segment damage condition function and the pipe segment defect parameter, the pipe segment structural defect type is determined according to the pipe segment defect density function, and the drainage pipe defect grade is determined according to the pipe defect scoring function.

4. The method according to claim 3, characterized in that The expression of the pipe section damage condition function is: n=n1+n2 S max =max{P i } Where S represents the pipe section damage function, n represents the total number of defects in the pipeline residual map, n1 represents the number of defects with a longitudinal clearance greater than 1.5 m in the pipeline residual map, n2 represents the number of defects with a longitudinal clearance greater than 1 m and not greater than 1.5 m in the pipeline residual map, P i1 It represents the defect fraction with longitudinal clearance greater than 1.5m in the pipeline residual diagram, P i2 represents the defect fraction with a longitudinal clearance greater than 1m and not greater than 1.5m in the pipeline residual diagram, α represents the structural defect influence coefficient, P i represents the defect score of the i-th defect in the pipeline residual graph, S max Represents the maximum defect score in the pipeline residual graph.

5. The method according to claim 3, characterized in that The expressions of the pipe section defect density function and the pipeline defect scoring function are respectively: P i (x)=1.4e -8 x 5 +3.04e -6 x 4 +1.88e -4 x 3 +2.53e -3 x 2 +0.11x+3.65e -2 Among them, SM represents the defect density function of the pipe section, SL represents the total length of the pipe section, L represents the length of the inspected pipe section, and L i1 Indicates the defect length of the pipeline residual diagram with a longitudinal clearance greater than 1.5m, L i2 It indicates the defect length of the pipeline residual diagram with a longitudinal clearance greater than 1m and not greater than 1.5m, P i (x) represents the pipeline scoring function, i represents the i-th pipeline residual graph, and x represents the defect score corresponding to the i-th pipeline residual graph.

6. The method according to claim 1, characterized in that The obtaining of the actual defect area corresponding to the defect area in the pipeline residual map specifically includes: According to the camera parameters and the pipe diameter, the actual area represented by each pixel is calculated; Based on the defect area of ​​the pipeline residual image and the area conversion function, the number of defect pixels corresponding to the defect area in the pipeline residual image is converted into the actual defect area.

7. The method according to claim 6, characterized in that The expression of the area conversion function is: P A =P H ×P V A defect =P A ×N Among them, P H Indicates the horizontal pixel size, P V represents the vertical pixel size, D represents the distance from the camera to the object, FOV horizontal Indicates the horizontal field of view, FOV vertical Represents the vertical field of view, P A represents the area of ​​a single pixel, N represents the number of pixels corresponding to the defect in the pipeline residual map, and A defect Indicates the actual area corresponding to the defect in the pipeline residual diagram.

8. A system for evaluating the status of urban drainage pipes, characterized in that: The urban drainage pipeline status assessment system (1) comprises an image acquisition module (11), an anomaly detection module (12) and a pipeline scoring module (13), wherein: The image acquisition module (11) is used to acquire original images of the interior of the urban drainage pipeline and to construct a data set of original pipeline images; The anomaly detection module (12) is used to input the pipeline original image data set into the MFSC-GANomaly anomaly detection model to extract the pipeline residual image of each pipeline original image in the pipeline original image data set, and calculate the defect area of ​​each pipeline residual image according to the Canny algorithm and morphological operation, and input the defect area corresponding to the pipeline residual image into the defect area calculation model to obtain the actual defect area corresponding to the defect area in the pipeline residual image; The pipeline scoring module (13) is used to determine the defect score corresponding to the actual defect area according to the area loss function, the actual defect area and the drainage pipeline defect rating standard, and calculate the pipeline defect scoring value based on the pipeline defect scoring function and the defect score to determine the structural defect level and defect type of the pipe section corresponding to the actual defect area.

9. An electronic device, characterized in that: The electronic device (2) comprises a processor (21), a memory (25), a user interface (23) and a network interface (24), wherein the memory (25) is used to store instructions, the user interface (23) and the network interface (24) are used to communicate with other devices, and the processor (21) is used to execute the instructions stored in the memory (25) so that the electronic device (2) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images