A method for identifying structural defects of a drainage pipe based on image semantic segmentation

Abstract

The present application belongs to the technical field of sewer defect image processing, and specifically discloses a sewer structural defect identification method based on image semantic segmentation. Through the synergistic effect of adaptive enhancement, multi-scale segmentation and physical constraint optimization, the robustness and adaptability of defect identification under complex working conditions are significantly improved. By introducing adaptive enhancement processing based on pipe geometric features and establishing a closed-loop feedback mechanism from the back-end optimization result to the front-end enhancement parameter, the system can actively learn and adapt to the variable illumination, reflection and attachment interference in the specific pipe, thereby maintaining stable and efficient identification performance in various harsh detection environments. Through deep fusion of multi-scale features and physical rule constraints, the accuracy and reliability of defect identification are improved. The multi-scale feature fusion segmentation model ensures good detection capability for various defects with large size differences, significantly reducing the false positive rate.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of sewer defect image processing, and relates to a sewer structural defect identification method based on image semantic segmentation. BACKGROUND

[0002] Sewer structural defect detection and identification is an important part of city pipe network maintenance, especially when dealing with problems such as pipe aging, corrosion and rupture. The basic goal is to efficiently evaluate the internal state of the pipe through image analysis technology, thereby providing a reliable basis for subsequent repair and maintenance. However, due to the complex internal environment of the sewer, uneven lighting conditions, and various interference factors such as dirt and water stains, image acquisition quality is often significantly affected, posing a great challenge to defect identification.

[0003] The existing technology usually adopts a method based on traditional image processing when dealing with sewer defect identification. These methods rely on manually designed feature extraction algorithms, which locate the defect area by performing grayscale transformation, edge detection or morphological operations on the image. At the same time, some technologies introduce simple machine learning models, which are trained using pre-labeled sample data to achieve defect classification. In addition, some improved schemes attempt to combine a pipe defect physical rule library to optimize the preliminary segmentation results by setting empirical parameters.

[0004] However, the above technical means still have certain limitations in practical application. For example, traditional image processing methods have weak adaptability to lighting changes and complex backgrounds, which can easily lead to false positives or false negatives. Machine learning models can improve recognition accuracy, but their performance is highly dependent on the quality and quantity of training data, and it is difficult to effectively capture multi-scale features. In addition, the application of the pipe defect physical rule library is mostly limited to static constraints, lacking dynamic adjustment capability, which may not perform well in the face of new defects or special scenarios. These problems make the existing technology still have room for improvement in terms of accuracy, robustness and efficiency of defect identification. SUMMARY

[0005] In view of this, in order to solve the problems raised in the background art, a sewer structural defect identification method based on image semantic segmentation is proposed.

[0006] The purpose of the application can be achieved by the following technical solutions: The application provides a sewer structural defect identification method based on image semantic segmentation, which includes: image acquisition enhancement, using a pipe detection device to obtain original images of the inside of the sewer, based on pipe inner wall contour detection, dividing the inside of the pipe into a top highlight area, a bottom shadow area and a side wall area, and performing region adaptive enhancement processing accordingly to generate an optimized image.

[0007] The semantic segmentation processing inputs the optimized image into a multi-scale feature fusion segmentation model for semantic segmentation processing, adopts a spatial attention mechanism to focus on a pipe wall region, and strengthens defect features through channel attention, and outputs a preliminary defect segmentation graph.

[0008] The morphological constraint optimization performs connected domain analysis and rule verification on the preliminary defect segmentation graph according to a preset pipe defect physical rule library, corrects or removes defect regions that violate continuity constraints and structural integrity constraints, and generates an optimized defect segmentation graph.

[0009] The defect quantification analysis performs defect parameter quantification processing based on the optimized defect segmentation graph, associates with pipe odometer data generation, and outputs a structural defect identification result with a spatial position.

[0010] The defect optimization feedback dynamically updates local enhancement parameters based on position information of the corrected or removed defect regions and a pipe sub-region type to which the defect regions belong.

[0011] Compared with the prior art, the present application has the following advantages: 1. The present application significantly improves the robustness and adaptability of defect identification under complex working conditions through the synergistic effect of adaptive enhancement, multi-scale segmentation and physical constraint optimization. By introducing adaptive enhancement processing based on pipe geometric features in the image processing pre-stage and establishing a closed-loop feedback mechanism from the back-end optimization result to the front-end enhancement parameter, the system can actively learn and adapt to the variable illumination, reflection and attachment interference in the specific pipe interior, so as to maintain stable and efficient identification performance in various harsh detection environments.

[0012] 2. The present application greatly improves the accuracy and reliability of defect identification by deeply fusing multi-scale features and physical rule constraints. The multi-scale feature fusion segmentation model ensures good detection ability for various defects with large size differences, and the subsequent introduction of physical rule optimization steps based on pipe structure mechanics and morphological knowledge can effectively remove false defects that do not conform to physical principles generated by pure data-driven models, significantly reducing the false alarm rate and ensuring the authenticity and reliability of the identification result.

[0013] 3. The present application realizes the objectivity of detection work and improves the engineering application value by standardizing and quantifying the identification result. This method not only identifies defects, but also automatically calculates key geometric parameters such as the length and area of the defect, and evaluates the severity level according to industry standards, finally generating a structured quantitative report. This replaces the subjectivity and inefficiency of traditional manual interpretation and estimation, provides a direct and reliable decision basis for pipe maintenance and repair, and makes pipe asset management more scientific and forward-looking. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments, obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0015] Figure 1 The schematic diagram of the step flow of the method of the present application is shown.

[0016] Figure 2 The schematic diagram of the sub-region division of the image acquisition enhancement module is shown.

[0017] Figure 3 The structural schematic diagram of the multi-scale feature fusion segmentation model is shown.

[0018] Figure 4 The rule checking flowchart of the morphological constraint optimization processing is shown.

[0019] Figure 5 The output example diagram of the defect quantification analysis module is shown.

[0020] Reference signs:

[0021] 1-top highlight area, 2-bottom shadow area, 3-side wall area, 4-encoder network, 5-feature pyramid structure, 6-space attention mechanism, 7-channel attention mechanism, 8-continuity constraint rule, 9-structural integrity constraint rule, 10-quantification report. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0023] Please refer to Figure 1 As shown in the drawings, the present application provides a drainage pipe structural defect identification method based on image semantic segmentation, and the specific steps are as follows: image acquisition enhancement, obtaining the original image inside the drainage pipe by using a pipe detection device, dividing the inside of the pipe into a top highlight area, a bottom shadow area and a side wall area based on pipe inner wall contour detection, and performing region adaptive enhancement processing accordingly to generate an optimized image.

[0024] It should be noted that the adaptive enhancement processing based on pipe region detection is performed on the obtained original image inside the drainage pipe, aiming to eliminate the interference caused by complex uneven illumination, reflection and silt and other factors inside the pipe on subsequent defect identification.

[0025] In a preferred embodiment of the present application, the specific analysis method for targeted regional adaptive enhancement processing is as follows: the inner wall contour of the pipeline in the original image is recognized by a fast pipeline region detection algorithm, the center coordinates and radius corresponding to the pipeline inner wall contour are obtained, and the pipeline internal region is divided into multiple sub-regions based on the center coordinates and radius, the sub-regions including a top highlight area, a bottom shadow area and a side wall area.

[0026] For example, the sub-region division method is as follows: an polar coordinate system is established with the pipeline center coordinates as the origin and the radius R as the radius. The division is made according to the angle: And The top highlight area is merged, corresponding to the top direct light area of the pipeline; The bottom shadow area is the weak light area at the bottom of the pipeline; And The side wall area is the medium light area on both sides. Each sub-region is defined by the polar coordinate angle range, and the boundary is determined by the center coordinates and radius, realizing accurate partitioning based on the geometric characteristics of the pipeline and providing a basis for subsequent regional adaptive enhancement.

[0027] The pixel gray scale statistical characteristics of each sub-region, including the average brightness and variance, are analyzed, and the corresponding local brightness compensation coefficient and contrast stretching parameter are calculated based on the pixel gray scale statistical characteristics.

[0028] The regional adaptive enhancement processing is performed based on the local brightness compensation coefficient and contrast stretching parameter.

[0029] For example, the average brightness and the variance of each sub-region are calculated, and a specific calculation method of a local brightness compensation coefficient and a contrast stretching parameter is as follows: 1. The local brightness compensation coefficient is calculated with the average brightness of the sub-region as the core basis, and a target brightness threshold is set, and the calculation formula is: brightness compensation coefficient = .

[0030] If the sub-region is the top highlight area which is too high, such as , the coefficient value range is 0.6-0.8, and the brightness is reduced to suppress overexposure.

[0031] If the sub-region is the bottom shadow area which is too low, such as , the coefficient value range is 1.2-1.5, and the brightness is increased to enhance the details.

[0032] If the sub-region is the side wall area, which is moderate, the coefficient is close to 1.0, and only fine tuning is done.

[0033] 2. Contrast stretching parameter calculation based on sub-region luminance variance and gray histogram distribution, dynamically adjust the stretching interval:

[0034] Calculate the minimum value of the gray scale of the current sub-region and maximum value .

[0035] Set the target stretching interval .

[0036] Stretching parameters are derived through a linear transformation formula: contrast stretching coefficient = .

[0037] For the top highlight area, high gray scale is concentrated, such as , compress the high gray interval, such as mapping 200-255 to 150-255, while stretching the low gray interval to retain details.

[0038] For the bottom shadow area, low gray scale is concentrated, such as , stretch the low gray interval, such as mapping 0-50 to 0-100, compress the high gray interval to avoid overexposure;

[0039] For the side wall area, the gray scale distribution is uniform, adjust according to the variance : if , the stretching coefficient is set to 1.2-1.5; if , the coefficient is set to 0.8-1.0.

[0040] It should be noted that the calculated local luminance compensation coefficient and contrast stretching parameter are applied to the corresponding sub-region, and the region adaptive enhancement processing is performed, such as luminance enhancement for dark areas, luminance suppression for overexposed areas, and normalization of the contrast of each region, to generate an enhanced image with more balanced illumination and contrast.

[0041] Perform texture filtering on the enhanced image to generate an optimized image.

[0042] It should be noted that in order to avoid the interference of non-defect textures formed by water flow, fine silt, etc. during the enhancement process, a texture filtering process is performed on the enhanced image, which can use edge-preserving filters such as guided filtering. Such filters can effectively protect the edge information of real defects while smoothing the image and suppressing small textures, ultimately obtaining an optimized image that is clear, less disturbed, and highlights defect features, providing high-quality input for subsequent accurate segmentation.

[0043] It should be noted that the geometric structure prior knowledge of the image is closely combined with the local image statistical characteristics, and through the way of first positioning the pipeline and then optimizing the partition, the intelligent guidance of the image enhancement process is realized, instead of indiscriminate processing of the whole image. This cooperative processing mode of detecting first and enhancing later can solve the difficult problems in practical engineering, such as the huge difference between top and bottom light in the pipeline and the strong local reflection. The beneficial effect is that the robustness and effectiveness of image preprocessing are greatly improved, the environmental interference is actively suppressed and the key defect characteristics are retained, which provides input data with better quality and higher signal-to-noise ratio for the subsequent multi-scale feature fusion segmentation model, thereby significantly reducing the defect missed detection and misjudgment caused by image quality problems in the subsequent steps from the source, and improving the accuracy and reliability of the whole defect recognition process.

[0044] It should be noted that the original image is an internal image of a drainage pipeline obtained by an endoscopic camera device. In order to overcome the image quality problem under complex lighting conditions, an adaptive enhancement processing technology based on pipeline region detection is adopted in this step. As shown in Figure 2 The original image is divided into multiple sub-regions, including a top highlight area 1, a bottom shadow area 2 and a side wall area 3. This division is based on a fast pipeline region detection algorithm, which completes the region division by identifying the pipeline inner wall contour and extracting the center coordinates and radius. In each sub-region, the pixel gray statistical characteristics, including the average brightness and variance, are calculated to determine the local brightness compensation coefficient and the contrast stretching parameter. These parameters are used to perform region adaptive enhancement processing to improve the image quality of each sub-region. For example, the top highlight area 1 is usually overexposed due to the direct impact of the light source, so the brightness needs to be reduced and the contrast needs to be increased; while the bottom shadow area 2 needs to increase the brightness to enhance the detail visibility. After the above processing, the enhanced image is subjected to texture filtering processing to remove noise, and finally an optimized image is generated.

[0045] The semantic segmentation processing inputs the optimized image into a multi-scale feature fusion segmentation model for semantic segmentation processing, adopts a spatial attention mechanism to focus on the pipe wall area, and through a channel attention to strengthen the defect features, and outputs a preliminary defect segmentation map.

[0046] In a preferred embodiment of the present application, the specific construction method for performing semantic segmentation processing is as follows: an encoder network containing multiple convolution and down-sampling layers is used to extract feature maps of the optimized image at different resolutions.

[0047] It should be explained that the optimized image is sent into an encoder network containing multiple convolution and down-sampling layers, with the increase of network layers, the spatial resolution of the image is gradually reduced, while the abstract level of the features is correspondingly improved, thereby generating a series of feature maps containing from bottom texture to high-level semantic information at different depths of the network.

[0048] The multi-scale fusion features are generated by fusing feature maps of the optimized images at different resolutions through a feature pyramid structure.

[0049] It should be further explained that the feature pyramid structure adopts a top-down path to up-sample the high-layer, low-resolution but rich-semantic feature maps and horizontally connect and fuse them with the high-resolution but rich-detail feature maps from the shallower layers, ensuring that the final feature representation takes into account both global context and local details.

[0050] A spatial attention mechanism is applied to the multi-scale fusion features to focus on the key areas of the pipe wall.

[0051] It should be noted that this mechanism generates a spatial weight map by learning, giving each position in the image an importance score, so that the model can concentrate computing resources on the pipe wall areas where defects are more likely to occur, while ignoring the background.

[0052] It should be further explained that in the pipe image, defects are mostly concentrated in the pipe wall area, while there may be background interference such as device reflection and non-pipe area in the image. This mechanism automatically summarizes the rules of where defects are more likely to occur by learning a large number of pipe image data, generating a spatial weight map of the same size as the input image. In the weight map, the value of each pixel represents the importance score of that position: the score of the pipe wall and other areas prone to defects is high, and the score of the background area is low. During subsequent processing, the model will allocate computing resources according to this map - focusing on feature extraction and detailed analysis in high-score areas, and reducing attention in low-score areas, thereby suppressing background interference while improving the recognition accuracy of pipe wall defects.

[0053] A channel attention mechanism is used to strengthen defect-related feature channels, and an attention-weighted feature is output for pixel-level defect classification.

[0054] It should be noted that this mechanism analyzes the correlation between different feature channels and learns to assign a weight to each channel, thereby enhancing the expression ability of those feature channels that contribute more to identifying specific defects and suppressing irrelevant or noise channels. The feature weighted by spatial and channel attention, i.e., the attention-weighted feature, is fed into the final classification layer for processing, outputting a pixel-level defect classification result, which is a segmentation map of the same size as the input image, where each pixel is assigned a clear class label, such as normal pipe wall, crack, breakage, or collapse.

[0055] It should be noted that the above scheme can dynamically focus attention on key areas and key features. Through the feature pyramid structure, the model obtains the ability to process large-scale defects and small-scale defects at the same time. The synergistic effect of spatial and channel double attention mechanism gives the model the ability to intelligently filter information. The deep integration of this multi-scale analysis and double attention mechanism produces a synergistic effect, enabling the model to more accurately locate and identify defects of all scales when faced with complex backgrounds and diverse defect morphologies, effectively distinguishing real defects from visual disturbances, significantly improving the accuracy and completeness of defect segmentation, and greatly reducing the probability of false positives and false negatives, thereby laying a solid foundation for the reliability of the entire identification method.

[0056] It should be noted that the optimized image is input into a multi-scale feature fusion segmentation model for semantic segmentation processing, as shown in the process of Figure 3 The model consists of an encoder network 4, a feature pyramid structure 5, a spatial attention mechanism 6, and a channel attention mechanism 7. The encoder network 4 extracts feature maps of the optimized image at different resolutions through multiple convolution and down-sampling layers, which are then fed into the feature pyramid structure 5 for multi-scale fusion. The role of the feature pyramid structure 5 is to integrate feature information at different resolutions to ensure that the model can capture multi-level details from global to local. On this basis, the spatial attention mechanism 6 further focuses on the key areas of the pipe wall, strengthening the spatial information related to defects; at the same time, the channel attention mechanism 7 highlights the saliency of the defect region by weighting and strengthening the defect-related feature channels. Finally, based on the attention-weighted features, the pixel-level defect classification results are output, generating a preliminary defect segmentation map.

[0057] The morphological constraint optimization performs connected component analysis and rule verification on the preliminary defect segmentation map based on a pre-set pipe defect physical rule library, corrects or removes defect regions that violate continuity constraints and structural integrity constraints, and generates an optimized defect segmentation map.

[0058] It should be noted that the construction of the pipe defect physical rule library is based on the knowledge of pipe structure mechanics and morphology. First, the physical properties and engineering measurement data of typical defects such as collapse, rupture, and interface misalignment are sorted out, and two types of constraint rules, continuity and structural integrity, are extracted. The continuity constraint rule specifies quantitative thresholds such as defect region length-to-width ratio, number of continuous pixels, extension direction, and angle between pipe axis; the structural integrity constraint rule sets verification standards such as collapse and adjacent deformation area ratio, rupture region to pipe wall boundary minimum distance, and interface misalignment region symmetry index for specific defects, forming a rule set that can be directly used for defect verification.

[0059] In a preferred embodiment of the present application, the specific analysis method of the connected domain analysis and rule verification is as follows: connected domain analysis is performed on the preliminary defect segmentation map, all pixels in the map that are spatially connected and have the same pixel value are aggregated into a defect region, and then a defect region set is obtained, which is composed of various defect regions.

[0060] The defect region set is matched and verified with the continuity constraint rules in the pipeline defect physical rule library, and the defect region set is also matched and verified with the structural integrity constraint rules in the pipeline defect physical rule library.

[0061] According to the verification result, defect regions that violate the constraint rules are corrected or removed.

[0062] Boundary smoothing processing is performed on the defect regions that meet the constraint rules.

[0063] For example, the system first matches and verifies each defect region with the continuity constraint rules in the library, for example, checks whether a region marked as a crack meets the minimum length and whether it is linearly extended rather than an isolated mass. Then, the system matches and verifies the same defect region set with the structural integrity constraint rules in the library, for example, checks whether a region marked as a collapse is necessarily accompanied by a broken or deformed region to meet the mechanical logic of structural failure. According to the verification results of the two steps, the system will correct or remove defect regions that violate the above constraint rules, for example, correct an isolated point-like crack region that does not meet the continuity rules to the background, or directly remove a collapse region that cannot exist independently in physics. For regions that pass all physical rule verification and are confirmed as valid defects, the system finally performs boundary smoothing processing on them, for example, using morphological opening or closing operation to eliminate burrs and irregularities of the segmentation boundary, so that the outline is closer to the natural form of the real defect, and finally generates an optimized defect segmentation map that is verified and optimized by physical knowledge.

[0064] In a preferred embodiment of the present application, the specific content of the continuity constraint rules is as follows: the ratio of the long side to the short side of the minimum bounding rectangle of the defect region is calculated, and it is judged whether it is greater than or equal to a preset long-short ratio threshold.

[0065] The total number of pixels of the defect region is calculated, and it is judged whether it is greater than or equal to a preset minimum continuous pixel number threshold.

[0066] The main extension direction of the defect region is calculated by principal component analysis or other methods, and compared with the direction of the pipeline center axis to obtain the angle between the extension direction of the defect region and the axial direction of the pipeline, and compared with a preset angle range threshold of the extension direction of the defect region and the axial direction of the pipeline to judge whether it is within the range threshold.

[0067] It should be noted that the setting of the above threshold value or threshold range is based on the physical characteristics of the pipeline defects and the engineering measured data. The length-to-short ratio threshold value refers to the real defect, such as the elongated morphology of the crack, which is determined by excluding the lump-like interference by statistically determining the geometric characteristics of the typical defect. The minimum number of continuous pixels threshold value is combined with the minimum physical size of the defect, which is converted into the number of image pixels to filter out the tiny noise or isolated points. The angle range threshold value is based on the stress law of the pipeline, and the angle distribution of the real defect, such as the longitudinal or circumferential crack and the axial direction, is statistically determined to set a range that conforms to the logic of structural mechanics to ensure that the angle with the pipeline axis conforms to the common defect extension mode.

[0068] Specifically, for each crack-like defect region, the system will select to perform the following three-dimensional quantitative verification. First, calculate the length-to-short ratio threshold value of the region, that is, by calculating the ratio of the long side to the short side of the minimum circumscribed rectangle surrounding the region, to determine whether its morphology is long enough; the length of a real crack should be much greater than its width. Second, count the minimum number of continuous pixels threshold value of the region, that is, calculate the total number of pixel points constituting the region to ensure that it has practical significance in physical size and filter out isolated points that are too small and may be image noise. Third, determine the extension direction of the region and the angle range threshold value with the pipeline axis, that is, calculate the main extension direction of the crack region by principal component analysis and the like, and compare it with the direction of the previously obtained pipeline center axis to determine whether its trend conforms to the common mode of pipeline stress failure, such as longitudinal cracks or circumferential cracks. Only when a crack-like defect region meets the three preset quantitative threshold value standards at the same time, it will be identified as a real crack defect that conforms to the physical characteristics, otherwise it will be considered as a misjudgment and removed from the defect set.

[0069] In a preferred embodiment of the present application, the specific content of the structural integrity constraint rule is as follows: identify the defect region marked as collapse, identify the defect region marked as deformation around it, obtain the area of the collapse defect region and the area of the deformation defect region, calculate the area ratio of the collapse defect region to the deformation defect region, and compare it with the preset area ratio threshold value of the collapse-like defect region to the adjacent deformation region.

[0070] It should be explained that the physical collapse must be developed from the excessive deformation of the pipe wall, and there is a certain correlation in space and severity. The preset area ratio threshold value of the collapse-like defect region to the adjacent deformation region is a quantitative standard set based on the pipeline structural mechanics law and engineering measured data. The threshold value is determined by statistically determining the area ratio distribution of the collapse region to the adjacent deformation region in a large number of real collapse cases, and is used to verify the physical reasonableness of the collapse defect: if the actual calculated area ratio exceeds the threshold value, it may be a misjudgment and needs to be corrected or removed to ensure that the identification result conforms to the objective law of structural failure.

[0071] The defect region marked as a crack is identified, the shortest distance from each pixel point of the defect region to the pre-detected pipe wall boundary is calculated, and the shortest distance is compared with a preset minimum distance threshold of the crack-type defect region and the pipe wall boundary.

[0072] It needs to be explained that this is intended to ensure that the crack occurs inside the pipe wall structure, rather than being an artifact or interference at the image edge. The minimum distance threshold of the crack-type defect region and the pipe wall boundary is a quantitative standard set based on the physical characteristics of the crack defect. As a structural damage of the pipe wall, the crack must occur on the pipe wall body, and its region should be adjacent to or located within the pipe wall boundary range, rather than in the empty area inside the pipe wall. The threshold is determined by statistical analysis of the actual distance distribution between the defect region and the pipe wall boundary in real crack cases, and is usually set to a small value. During verification, if the shortest distance from all pixels of a region marked as a crack to the pipe wall boundary is greater than the threshold, it means that the region deviates from the pipe wall and may be a false positive, which needs to be corrected or removed to ensure that the identified crack defect conforms to the objective law of occurring in the pipe wall.

[0073] The defect region marked as an interface misalignment is identified, the geometric features of the pipe profiles on both sides of the defect region are analyzed, the symmetry index is calculated, and the symmetry index is compared with a preset symmetry index threshold of the interface misalignment-type defect.

[0074] It needs to be explained that a real interface misalignment usually exhibits parallel misalignment of two pipe ports, and the profiles should maintain a certain symmetry and regularity. The symmetry index threshold of the interface misalignment-type defect is a quantitative standard set based on the structural characteristics of the pipe interface. The symmetry index is obtained by calculating the geometric matching degree of the profiles on both sides of the misalignment region, such as the symmetric distribution of profile line spacing and angle deviation, and the range is 0-1, where 1 represents complete symmetry. The threshold is determined according to the symmetry index distribution of a large number of real misalignment cases, and is used to verify the authenticity of the defect: if the actual index is lower than the threshold, it may be a false positive, which needs to be corrected or removed to ensure that the identified misalignment conforms to the structural characteristics of the relative displacement of the connection part.

[0075] For example, a symmetry index calculation method is as follows: taking the interface misalignment region as the center, the profile curves of the pipe edges on both sides are extracted. Taking the theoretical center axis of the pipe interface as the symmetry reference, 50 sampling points are uniformly selected on one side of the profile, and the coordinates of the symmetric points of each point about the reference line are calculated. The symmetric points are matched with the nearest points on the other side of the profile, the Euclidean distance deviations of all point pairs are counted, and the average deviation value is taken. The symmetry index calculation formula is: , wherein represents the average deviation value corresponding to the Euclidean distance deviation of all point pairs, the average spacing of the profiles on both sides. The closer the index is to 1, the better the symmetry of the profiles on both sides.

[0076] It should be noted that the preliminary defect segmentation map then enters a morphological constraint optimization step, as shown in Figure 4 The core of this step is to perform connected component analysis on the preliminary defect segmentation map and check it in combination with a pre-set physical rule library of pipeline defects. The connected component analysis aggregates all spatially connected and pixel value identical pixel points in the map into a defect region, forming a defect region set. These defect region sets are in turn matched and checked with the continuity constraint rule 8 and the structural integrity constraint rule 9. The continuity constraint rule 8 mainly includes three aspects: calculating the length-to-short side ratio of the minimum bounding rectangle of the defect region, judging whether it is greater than or equal to the pre-set length-to-short side ratio threshold; calculating the total number of pixel points of the defect region, judging whether it is greater than or equal to the pre-set minimum continuous pixel number threshold; calculating the main extension direction of the defect region by principal component analysis or other methods, and comparing it with the pre-set defect region extension direction and the pipeline axial direction angle range threshold. The structural integrity constraint rule 9 checks specific types of defects, such as identifying collapsed defect regions and their surrounding deformed defect regions, calculating their area ratio and comparing it with the pre-set threshold; identifying broken defect regions and calculating the shortest distance from each pixel point to the pipe wall boundary, judging whether it satisfies the minimum distance threshold of broken defect regions and the pipe wall boundary; identifying interface misalignment defect regions and analyzing the geometric features of the two sides of the pipe profile, calculating the symmetry index and comparing it with the pre-set threshold. According to the checking results, the defect regions that violate the constraint rules are modified or removed, and the defect regions that meet the rules are subjected to boundary smoothing processing to generate an optimized defect segmentation map.

[0077] Defect quantification analysis, based on the optimized defect segmentation map, performs defect parameter quantification processing, associates pipeline odometer data to generate a defect database, and outputs a structural defect recognition result with spatial position.

[0078] In a preferred embodiment of the present application, the specific analysis method for performing defect parameter quantification processing is as follows: calculating the geometric feature parameters of each type of defect in the optimized defect segmentation map.

[0079] Specifically, for the pixel region identified as a crack, the system calculates its total length and average width through a skeleton extraction algorithm; for the region identified as a collapse or a break, the system calculates its absolute area covered by pixel counting, and converts it into a percentage of the pipeline cross section according to the pipeline structural feature parameters.

[0080] According to the pre-set defect evaluation standard, match the defect level.

[0081] Integrate the odometer data recorded by the pipeline detection equipment to obtain the accurate longitudinal position coordinates of each image to be analyzed in the pipeline.

[0082] Create a defect database associated with geographic information. The table structure of the defect database includes pipe segment number, longitudinal location of defect, defect category, defect level, defect geometric feature parameters, and the corresponding original image frame file path.

[0083] Each defective region is sorted according to its longitudinal position in the pipeline and stored in the defect database.

[0084] Generate a visualized quantitative report on the distribution of pipeline defects. The report displays the entire inspected pipeline section in the form of a two-dimensional unfolded diagram, and marks the type, level and key parameters of each defect at the corresponding position on the diagram, providing an intuitive basis for pipeline repair and maintenance decisions.

[0085] In a preferred embodiment of the present invention, the specific analysis method for matching defect levels according to preset defect assessment standards is as follows: a defect assessment standard rule base is established, wherein multiple severity levels are preset for each defect category, and a corresponding geometric morphological parameter threshold range is defined for each level.

[0086] The assessed defect level is bound to the corresponding defect entity information to form the final structured defect identification result.

[0087] It should be noted that after optimizing the defect segmentation map and entering the defect quantification analysis step, defect evaluation will be performed based on geometric feature parameters. For example... Figure 5 As shown, this step first calculates the geometric characteristic parameters of various defects, including the area, perimeter, major axis length, and minor axis length of the defect region. These parameters are then matched with preset defect assessment criteria to determine the severity level of the defect. For example, for collapse-type defects, a defect is classified as severe when its area exceeds a certain threshold; while for rupture-type defects, the classification is based on its distance from the pipe wall boundary. Furthermore, defect location information is correlated with the timestamp of the pipe endoscopy video for subsequent tracing and localization. Finally, the step generates a quantitative report10 containing the defect category, location, size, and severity level, providing a basis for subsequent maintenance decisions.

[0088] Defect optimization feedback dynamically updates local enhancement parameters based on the location information of the corrected or eliminated defect areas and the type of the pipe sub-region to which they belong.

[0089] In a preferred embodiment of the present invention, a closed-loop feedback step is further included, the specific content of which is as follows: recording the location information of the defect region that has been corrected in the morphological constraint optimization process.

[0090] An adaptive enhancement parameter adjustment instruction is generated based on the location information of the corrected defect area.

[0091] The adaptive enhancement parameter adjustment instruction is fed back to the adaptive enhancement processing step.

[0092] The adaptive enhancement parameter adjustment instruction is fed back to the adaptive enhancement processing step.

[0093] It should be noted that in the optimization process, the system will record the position information of the defect area identified by the pipeline defect physical rule library and corrected or removed in detail. These information is essentially the error log of the misjudgment of the system in the previous step. The system will generate adaptive enhancement parameter adjustment instructions based on the recorded position information. The specific method is to analyze the image features such as illumination and contrast of the misjudged area in the original image, find out the image root cause of the misjudgment, and generate a parameter adjustment strategy for correcting such misjudgment. Then, the system feeds back the newly generated adjustment instruction to the adaptive enhancement processing step at the front end of the process. When the system processes subsequent video frames or new pipeline images, the adaptive enhancement processing step will dynamically update its internal local enhancement parameters based on the received adjustment instruction.

[0094] In a preferred embodiment of the present application, the adaptive enhancement parameter adjustment instruction includes: positioning the position information of the corrected defect area, calculating the average brightness, brightness variance, and pixel grayscale histogram of the defect area.

[0095] A mapping relationship model of the region illumination feature and the enhancement parameter is established.

[0096] A preferred embodiment provides a specific construction method of a mapping relationship model of the region illumination feature and the enhancement parameter, which is as follows: collecting sample data of different pipeline regions, extracting region illumination features: average brightness , brightness variance , and grayscale histogram peak position . The enhancement parameters of the corresponding region optimized by artificial optimization are recorded synchronously, including brightness compensation coefficient and contrast stretching coefficient . The , , are input features, and the , are output labels, and a random forest regression model is trained. In the model training, the parameters are optimized by grid search to minimize the error between the predicted , and the artificial optimization value. For example, the model learns that when , , that is, in dark areas and low contrast, the output , ; when , , that is, in overexposure and high noise, the output 、 , form a mapping relationship model.

[0097] For example, if the system finds that a large number of corrected regions have very high brightness variance, the model will establish a rule that when encountering a region with such characteristics, the contrast stretching strength applied to it should be reduced.

[0098] According to the characteristics of the current frame correction region, the enhanced parameters of the subsequent frame are predicted.

[0099] It should be noted that since pipeline detection is usually continuous video frames, the subsequent frame is adjacent to the current frame in space, and the environmental characteristics are coherent. Therefore, the system can predict the enhancement needs of similar regions in the subsequent frame by the characteristics of the current frame correction region, combine the inter-frame correlation rules, and adjust the brightness compensation coefficient, contrast stretching parameter, etc. in advance, so that the enhancement processing of the subsequent frame is more accurate, the repeated misjudgment is reduced, and the efficiency and stability of continuous detection are improved.

[0100] It should be noted that in the entire process, the closed-loop feedback control mechanism plays a key role. Specifically, the position information of the defect region corrected in the morphological constraint optimization step is recorded and used to generate adaptive enhancement parameter adjustment instructions. These instructions are fed back to the image acquisition enhancement step to dynamically update the local enhancement parameters. This closed-loop feedback mechanism ensures continuous optimization of image acquisition quality and reduces the need for manual intervention and misjudgment rate.

[0101] It should be noted that by establishing a standardized automated process, the abstract pixel patterns recognized by the visual algorithm are systematically translated into engineering language that is commonly used in the engineering field, quantifiable, and has clear physical meaning. It completes the last leap from qualitative recognition to quantitative analysis, and is the key link for the entire intelligent recognition method to realize the engineering application value. The beneficial effect is that it greatly improves the objectivity and efficiency of pipeline detection work, and completely replaces the traditional cumbersome, error-prone and subjective work method that relies on manual visual inspection and manual estimation. The output of the quantitative report provides accurate and standardized digital archives for the health status of the pipeline, making the work of maintenance priority ranking, maintenance scheme development, and long-term asset health tracking have solid and reliable data support, thereby significantly improving the scientific nature and economic benefits of pipeline operation and management.

[0102] It should be noted that the present application realizes efficient identification of structural defects of a drainage pipeline through the above specific embodiments. Among them, the image acquisition enhancement step solves the image quality problem under complex lighting conditions, the multi-scale feature fusion segmentation model improves the accuracy of defect area positioning, the morphological constraint optimization step realizes fine matching combined with the pipeline defect physical rule library, and the defect quantitative analysis step provides a comprehensive defect evaluation report. The steps are closely connected through data flow and jointly constitute a complete defect identification system.

[0103] It should be noted that the technical principle of the present application lies in constructing a phased and step-by-step refined image processing and analysis workflow. The method first performs targeted preprocessing on the original image through adaptive enhancement technology based on the geometric characteristics of the pipeline space, actively suppresses the interference of uneven illumination and reflection specific to the pipeline interior, and thereby provides a high-quality visual basis for subsequent analysis. On this basis, a deep learning segmentation model capable of fusing multi-scale feature information is used to perform comprehensive pixel-level defect identification on the optimized image, ensuring that defect features of different sizes can be effectively captured. The core innovation lies in not directly adopting the preliminary segmentation result of the model, but introducing a constraint optimization link based on the physical rules of pipeline engineering, simulating the logical judgment of experts in the field to perform secondary verification and correction on the preliminary result, and eliminating false defects that do not conform to physical reality. Finally, the accurate segmentation map verified by the physical rules is converted into structured engineering quantitative data, completing the conversion from image to quantifiable report. The whole principle is a combination of data-driven intelligent identification and knowledge-driven logical verification, forming a closed-loop processing logic that complements and verifies each other.

[0104] The above content is only an example and description of the concept of the present application, and those skilled in the art can make various modifications or supplements or use similar ways to replace the described specific embodiments, as long as they do not deviate from the concept of the present application or exceed the scope defined by the present application, which shall fall within the protection scope of the present application.

Claims

1. A method for identifying structural defects of a sewer pipe based on image semantic segmentation, characterized in that: The method comprises the following steps: Image acquisition enhancement: using a pipeline detection device to obtain an original image inside a drainage pipeline, dividing the inside of the pipeline into a top highlight area, a bottom shadow area and a side wall area based on a pipeline inner wall contour detection, and performing region adaptive enhancement processing on the areas to generate an optimized image; The specific analysis method of the region adaptive enhancement processing is as follows: Identify the pipeline inner wall contour in the original image through a fast pipeline region detection algorithm, obtain the center coordinates and radius corresponding to the pipeline inner wall contour, divide the inside of the pipeline into a plurality of sub-regions based on the center coordinates and radius, and the sub-regions include a top highlight area, a bottom shadow area and a side wall area; Analyze the pixel gray statistical characteristics of each sub-region, including average brightness and variance, calculate the corresponding local brightness compensation coefficient and contrast stretching parameter based on the pixel gray statistical characteristics; Perform region adaptive enhancement processing based on the local brightness compensation coefficient and contrast stretching parameter; Perform texture filtering processing on the image after enhancement processing to generate an optimized image; Semantic segmentation processing: input the optimized image into a multi-scale feature fusion segmentation model for semantic segmentation processing, focus on the pipe wall area by using a spatial attention mechanism, and strengthen the defect features by using a channel attention, and output a preliminary defect segmentation map; Morphological constraint optimization: perform connected component analysis and rule verification on the preliminary defect segmentation map according to a pre-set pipeline defect physical rule library, correct or remove defect areas that violate the continuity constraint and structural integrity constraint, and generate an optimized defect segmentation map; Defect quantitative analysis: perform defect parameter quantitative processing based on the optimized defect segmentation map, associate the pipeline odometer data to generate a defect database, and output a structural defect recognition result with spatial position; Defect optimization feedback: dynamically update the local enhancement parameters based on the position information of the corrected or removed defect areas and the type of the pipeline sub-regions to which the defect areas belong.

2. The method according to claim 1, wherein the method is characterized by: The specific construction method of the semantic segmentation processing is as follows: Use an encoder network with multiple convolution and down-sampling layers to extract feature maps of the optimized image at different resolutions; Fuse the feature maps of the optimized image at different resolutions through a feature pyramid structure to generate multi-scale fusion features; Apply a spatial attention mechanism to the multi-scale fusion features to focus on the key areas of the pipe wall; Strengthen the defect-related feature channels through a channel attention mechanism, and output a pixel-level defect classification result based on the attention-weighted features.

3. The method according to claim 1, characterized in that: The specific analysis method of the connected component analysis and rule verification is as follows: Perform connected component analysis on the preliminary defect segmentation map, aggregate all spatially connected and pixel value identical pixel points in the map into a defect area, and then obtain a defect area set, which is composed of various defect areas; Match and verify the defect area set with the continuity constraint rules in the pipeline defect physical rule library, and simultaneously match and verify the defect area set with the structural integrity constraint rules in the pipeline defect physical rule library; Correct or remove the defect areas that violate the constraint rules according to the verification results; Perform boundary smoothing processing on the defect areas that meet the constraint rules.

4. The method according to claim 3, wherein the method further comprises: The specific content of the continuity constraint rule is as follows: Calculate the ratio of the long side and the short side of the minimum circumscribed rectangle of the defect region, and determine whether it is greater than or equal to the preset long-short ratio threshold; Calculate the total number of pixel points of the defect region, and determine whether it is greater than or equal to the preset minimum continuous pixel number threshold; Calculate the main extension direction of the defect region by principal component analysis method, and compare it with the direction of the pipeline center axis obtained in advance to obtain the angle between the extension direction of the defect region and the axial direction of the pipeline, and compare it with the preset angle range threshold of the extension direction of the defect region and the axial direction of the pipeline to determine whether it is within the range threshold.

5. The method according to claim 3, characterized in that: The specific content of the structural integrity constraint rule is as follows: Identify the defect region marked as collapse, identify the defect region marked as deformation around it, obtain the area of the collapse defect region and the area of the deformation defect region, calculate the area ratio of the collapse defect region to the deformation defect region, and compare it with the preset area ratio threshold of the collapse defect region to the adjacent deformation region; Identify the defect region marked as rupture, calculate the shortest distance from each pixel point of the defect region to the pre-detected pipe wall boundary, and compare it with the preset minimum distance threshold of the rupture defect region to the pipe wall boundary; Identify the defect region marked as interface misalignment, analyze the geometric features of the pipeline profile on both sides of the defect region, calculate the symmetry index, and compare it with the preset symmetry index threshold of the interface misalignment defect.

6. The method according to claim 1, wherein the method is characterized by: It also includes a closed-loop feedback step, the specific content of which is as follows: Record the defect region position information corrected in the morphological constraint optimization process; Generate adaptive enhancement parameter adjustment instructions based on the corrected defect region position information; Feedback the adaptive enhancement parameter adjustment instructions to the adaptive enhancement processing step; Dynamically update the local enhancement parameters based on the adaptive enhancement parameter adjustment instructions.

7. The method according to claim 6, wherein the method further comprises: The adaptive enhancement parameter adjustment instructions include: Locate the corrected defect region position information, calculate the average brightness, brightness variance, and pixel gray histogram of the defect region; Establish a mapping relationship model between the regional lighting features and the enhancement parameters; Predict the enhancement parameters of the subsequent frames based on the current frame correction region features.

8. The method according to claim 1, characterized in that: The specific analysis method for defect parameter quantization processing is as follows: Calculate the geometric feature parameters of each type of defect in the optimized defect segmentation map; Match the defect level according to the preset defect evaluation standard; Integrate the odometer data recorded by the pipeline detection equipment to obtain the accurate longitudinal position coordinates of each image in the pipeline; Create a defect database associated with geographic information, and the table structure of the defect database includes pipeline segment number, defect longitudinal position, defect category, defect level, defect geometric feature parameters, and corresponding original image frame file path; Sort each defect region according to its longitudinal position in the pipeline and store it in the defect database; Generate a visual pipeline defect distribution quantization report, which displays the entire detection pipeline segment in the form of a two-dimensional expansion graph and labels the type, level and key parameters of each defect on the corresponding position on the graph, providing intuitive basis for pipeline repair and maintenance decision-making.

9. The method according to claim 8, wherein the method further comprises: determining the structural defects of the sewer pipeline based on the semantic segmentation result. The specific analysis manner of matching the defect level according to the preset defect evaluation standard is as follows: A defect evaluation standard rule base is established, the defect evaluation standard rule base presets multiple severity levels for each defect category, and defines corresponding geometric parameter threshold ranges for each level; The evaluated defect level is bound with the corresponding defect entity information to form a final structured defect identification result.

Images