Deep Learning-Based Sewage Monitoring System and Method in Pipelines

Abstract

This invention discloses a deep learning-based system and method for monitoring sewage in pipelines, comprising the following steps: S1, preprocessing acquired image data to generate a standardized sewage image sequence; S2, introducing radial and axial position codes to generate a structure-enhanced image feature sequence; S3, inputting the data into an improved Mask2Former image segmentation network, performing semantic query grouping and boundary refinement prediction to output an abnormal segmentation set; S4, applying radial distribution consistency constraints and axial continuity constraints to generate a sewage state feature vector; S5, performing state determination to generate an abnormal sewage state identifier; S6, matching the identifier with a preset sewage state threshold rule to enter the sewage discharge process determination mode; S7, repeating steps S1 to S4 and performing a recovery determination to form a monitoring result. This invention achieves stable identification and accurate determination of abnormal sewage in pipelines, improving monitoring reliability and sewage discharge control safety.

Description

Technical Field

[0001] This invention relates to the field of pipeline sewage monitoring technology, and in particular to a pipeline sewage monitoring system and method based on deep learning. Background Technology

[0002] Currently, the inspection of sewage pipelines mainly relies on manual visual inspection and periodic patrols. For example, maintenance personnel often use closed-circuit television (CCTV) endoscopic camera systems, placing cameras inside the pipeline to capture video, which is then manually reviewed to identify sediment, corrosion, or siltation problems inside the pipeline. For small and medium-diameter pipelines that cannot be accessed, CCTV inspection has become the most common method, but the inspection results still require manual interpretation and analysis. This traditional method has many shortcomings:

[0003] Significant human error: Relying primarily on visual inspection to identify pipeline defects is highly subjective. This manual method is not only time-consuming and labor-intensive, but also prone to subjective misjudgments and omissions. The operator's experience level directly affects the accuracy of the judgment; a single oversight may lead to missing potential hazards.

[0004] Low monitoring frequency: Because manual inspections require a significant investment of manpower and resources, pipelines are typically only inspected periodically (e.g., every six months or year), making it difficult to detect deposits, blockages, or corrosion that gradually develop between inspections. Insufficient monitoring frequency leads to problems accumulating to a severe stage before they are discovered.

[0005] High safety risks: Manually descending into manholes or entering pipelines for inspection poses significant safety risks. Underground sewage pipes are often filled with toxic and harmful gases (such as hydrogen sulfide), have low oxygen levels, and are located in confined spaces, posing a risk of poisoning and asphyxiation to maintenance personnel. Furthermore, the pipeline environment is harsh and slippery, making manual operation inherently dangerous.

[0006] Inability to continuously monitor: Existing manual or CCTV inspection methods cannot achieve continuous real-time monitoring of pipeline status, and are therefore passive and intermittent inspections. Once an anomaly occurs between two inspections (such as a sudden increase in siltation or aggravated corrosion), the system is unable to provide timely warnings and cannot provide all-weather automatic monitoring data support.

[0007] It is evident that traditional manual methods for monitoring sewage inside pipelines are significantly inadequate in terms of accuracy, timeliness, and safety, failing to meet the demands of modern pipeline network operation and maintenance for real-time, precise, and intelligent monitoring. Therefore, providing a deep learning-based sewage monitoring system and method for pipelines is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] One objective of this invention is to propose a deep learning-based system and method for monitoring sewage in pipelines. This invention employs an improved image segmentation network and a structural consistency determination mechanism to continuously sense and determine the status of abnormal sewage in pipelines, thereby achieving sewage discharge triggering and recovery control. It has the advantages of high monitoring accuracy, strong determination stability, and high operational reliability.

[0009] The deep learning-based method for monitoring sewage in pipelines according to embodiments of the present invention includes the following steps:

[0010] S1. Collect continuous image data of sewage in the pipeline and preprocess it to generate a standardized sewage image sequence;

[0011] S2. Introduce radial and axial position coding of pipelines into the standardized sewage image sequence, perform structural prior embedding operation, and generate a structure-enhanced image feature sequence.

[0012] S3. Input the structure-enhanced image feature sequence into the image segmentation network based on the Mask2Former architecture. In the mask query stage, construct several query groups according to the preset sewage anomaly semantic types. Perform group attention update operation on the query vector in each query group and the structure-enhanced image feature sequence to generate an updated query group set. In the mask prediction stage, use the updated query group set to generate anomaly region masks. At the same time, introduce anomaly boundary perception refinement branch in parallel to perform constraint correction processing on the boundary position of the anomaly region, forming an anomaly segmentation set predicted by joint constraints, which is used to characterize the spatial distribution of sewage anomalies in the pipeline.

[0013] S4. Based on the abnormal segmentation set, perform structural consistency region quantization operation according to the radial distribution consistency constraint and axial continuity constraint of the pipeline to generate sewage state feature vector;

[0014] S5. Perform wastewater state determination operation on the wastewater state feature vector to generate wastewater abnormal state identifier;

[0015] S6. Match the abnormal state identifier of sewage with the preset sewage state threshold rule. When the match is successful, generate a sewage discharge trigger state identifier and switch to the sewage discharge process judgment mode, and collect sewage image sequence of the sewage discharge process.

[0016] S7. Steps S1 to S4 are executed sequentially on the sewage image sequence during the sewage discharge process to generate a sewage state feature vector during the sewage discharge process. A recovery judgment operation is performed on the sewage state feature vector during the sewage discharge process. When the recovery judgment is successful, a stop sewage discharge status identifier is generated to form the sewage monitoring result in the pipeline.

[0017] Optionally, the continuous sewage image data collected in step S1 includes visible light image data continuously collected along the pipeline axis and the corresponding timestamp sequence; the preprocessing includes performing time alignment on the visible light image data according to the timestamp sequence to generate a time-synchronized sewage image sequence, performing joint operations of noise suppression and brightness correction to generate a brightness-stable sewage image sequence, and performing geometric consistency correction and unified resolution normalization operations to generate a standardized sewage image sequence.

[0018] Optionally, the process of introducing radial and axial position codes for the pipeline and performing structural prior embedding operations in step S2 includes:

[0019] S21. Perform grayscale conversion processing on each frame of the standardized sewage image sequence, perform edge detection operation on the grayscale image, and extract the set of continuous pipe wall edge pixels.

[0020] S22. Perform circular structure fitting processing on the set of pixels at the edge of the pipe wall to determine the center position of the pipe and the contour range of the inner wall of the pipe in the corresponding image.

[0021] S23. Using the center position of the pipe as a reference, perform radial distance mapping on the relative position of each pixel in the image coordinate system, and classify the radial distance according to the contour range of the inner wall of the pipe to generate a radial position distribution map consistent with the image space size.

[0022] S24. Assign different position labels to different radial hierarchical regions in the radial position distribution map in sequence, and expand the position labels along the image channel direction to form a radial position coding matrix.

[0023] S25. Based on the frame order of the standardized sewage image sequence, establish an axial position index sequence, divide the axial position index sequence into levels according to the acquisition order, and generate an axial position identifier sequence.

[0024] S26. Extend the axial position identifier sequence according to the frame index mapping to be consistent with the spatial size of the corresponding frame image, forming an axial position encoding matrix;

[0025] S27. Perform channel-by-channel interactive mapping processing on the radial position coding matrix and the axial position coding matrix to generate a structural prior coding matrix that simultaneously represents the radial and axial positional relationships of the pipeline.

[0026] S28. Perform feature fusion processing on the structure prior coding matrix and the standardized sewage image sequence according to the channel dimension to generate a structure-enhanced image feature sequence.

[0027] Optionally, the process of constructing a set of query groups oriented towards sewage anomaly semantics and performing group attention update operations in step S3 during the mask query phase includes:

[0028] S31. Expand the feature sequence of the structure-enhanced image into a feature vector sequence according to its spatial location, and record the pixel spatial location index corresponding to each feature vector in the feature vector sequence.

[0029] S32. Establish a wastewater anomaly semantic set, and assign a query group identifier to each wastewater anomaly semantic based on the wastewater anomaly semantic set to form a query group identifier set;

[0030] S33. Generate an initial set of query vectors according to the set of query group identifiers, divide the initial set of query vectors into a set of query groups according to the set of query group identifiers, and arrange the query vectors of each query group in the set of query groups in a fixed order to form a query sequence within the group.

[0031] S34. Perform intra-group attention matching processing on each query group in the query group set. The intra-group attention matching processing includes matching the intra-group query sequence with the feature vector sequence one by one, extracting the local feature subset associated with the query vector from the feature vector sequence according to the matching result, and rearranging the local feature subset into an intra-group context sequence according to the pixel spatial position index.

[0032] S35. Perform intra-group fusion update processing on the intra-group query sequence and intra-group context sequence to generate the updated intra-group query sequence.

[0033] S36. After completing the intra-group fusion update process for each query group, inter-group isolation processing is performed between different query groups. The inter-group isolation processing includes writing the updated intra-group query sequences of different query groups into independent query caches, and reading the intra-group query sequences only from the corresponding query caches in the query update loop.

[0034] S37. Repeat steps S34 to S36 sequentially for the query group set until the preset update round is reached, and output the updated query group set.

[0035] Optionally, in step S3, an anomaly boundary-aware refinement branch is introduced in the mask prediction stage, and the process of performing joint constraint prediction includes:

[0036] S38. Input the structure-enhanced image feature sequence into the mask generation path, and perform query-by-query mask generation processing on the structure-enhanced image feature sequence according to the updated query group set to obtain the initial abnormal region mask set.

[0037] S39. Input the structure-enhanced image feature sequence into the abnormal boundary perception refinement branch in parallel, and perform boundary response extraction processing on the structure-enhanced image feature sequence to obtain the abnormal boundary response map.

[0038] S310. Perform thresholding and connectivity filtering on the abnormal boundary response map to generate an abnormal boundary binary map.

[0039] S311. Perform pixel-by-pixel alignment processing on the initial abnormal region mask set and the abnormal boundary binary map according to the pixel spatial position index to determine the abnormal region mask boundary position set.

[0040] S312. Within the set of abnormal region mask boundaries, perform boundary consistency correction processing on the initial abnormal region mask set based on the abnormal boundary binary map to form a boundary-refined abnormal region mask set.

[0041] S313. Perform hole filling and small connected component removal processing on the boundary refinement abnormal region mask set to generate an abnormal region consistent mask set.

[0042] S314. Merge the set of consistency masks for abnormal regions according to the query group identifier to form a set of abnormal segmentation quantities used to characterize the abnormal spatial distribution of sewage in the pipeline.

[0043] Optionally, the generation of the wastewater state feature vector includes:

[0044] S41. Based on the abnormal segmentation set, extract the abnormal region pixel set in each frame image, and map the abnormal region pixel set to the corresponding radial partition according to the pipeline radial position encoding to form an abnormal radial distribution sequence.

[0045] S42. Count the number of abnormal pixels in each radial partition in the abnormal radial distribution sequence, perform consistency judgment on the abnormal pixel distribution changes of adjacent radial partitions, perform suppression processing on the abnormal pixel set corresponding to the radial partition that fails the consistency judgment, and generate an abnormal region pixel set under radial consistency constraint.

[0046] S43. Based on the radial consistency constraint, the abnormal region pixel set is matched with the position of the abnormal region pixel set between adjacent image frames according to the frame order of the standardized sewage image sequence, forming an axial distribution sequence of abnormal regions.

[0047] S44. Perform continuity determination on the axial distribution sequence of abnormal regions between adjacent image frames, and perform elimination processing on the set of abnormal region pixels that fail the continuity determination to generate a set of abnormal region pixels under structural consistency constraints.

[0048] S45. Summarize the distribution of abnormal areas in each radial partition and axial segment to generate a wastewater state feature vector.

[0049] Optionally, step S5 specifically includes:

[0050] S51. Read the wastewater state feature vector and extract the value of each feature component in sequence according to the dimensional order of the wastewater state feature vector;

[0051] S52. Compare each feature component value with the corresponding threshold condition in the preset wastewater state threshold rule to generate a feature comparison result set.

[0052] S53. Perform state determination processing based on the feature comparison result set. When at least one feature component value meets the corresponding abnormal threshold condition, generate a sewage abnormal state identifier. When none of the feature component values ​​meet the abnormal threshold condition, generate a normal state identifier and output the sewage abnormal state identifier.

[0053] Optionally, step S6 specifically includes: reading the sewage abnormal state identifier, retrieving the trigger judgment condition corresponding to the sewage abnormal state identifier from the preset sewage state threshold rules; statistically analyzing the occurrence of the sewage abnormal state identifier within several consecutive judgment periods, and comparing the statistical results with the persistence requirement in the trigger judgment condition; when the sewage abnormal state identifier meets the trigger judgment condition within several consecutive judgment periods, generating a sewage discharge trigger state identifier, and switching the current monitoring process to the sewage discharge process judgment mode; in the sewage discharge process judgment mode, collecting sewage image data of the sewage discharge process according to the preset acquisition frequency to form a sewage image sequence of the sewage discharge process.

[0054] Optionally, the process of performing the recovery determination operation and generating the stop sewage discharge status indicator in step S7 includes:

[0055] S71. Steps S1 to S4 are executed sequentially on the sewage image sequence of the sewage discharge process to generate a sewage state feature vector sequence arranged in chronological order.

[0056] S72. According to the preset judgment period, select the corresponding sewage state feature vectors from the sewage state feature vector sequence of the sewage discharge process in sequence, and compare each sewage state feature vector with the abnormal threshold conditions in the preset sewage state threshold rules item by item.

[0057] S73. When a feature component in the wastewater state feature vector within a certain judgment period meets the abnormal threshold condition, the judgment period is marked as an unrecovered period. When none of the feature components meet the abnormal threshold condition, the judgment period is marked as a candidate recovery period.

[0058] S74. Statistically analyze the marking results of several consecutive judgment cycles. When several consecutive judgment cycles are marked as candidate recovery cycles, generate a stop sewage discharge status indicator, end the sewage discharge process judgment mode, and form the sewage monitoring results in the pipeline.

[0059] The deep learning-based pipeline sewage monitoring system according to an embodiment of the present invention includes the following modules:

[0060] The wastewater image acquisition and preprocessing module is used to acquire continuous image data of wastewater in the pipeline and perform preprocessing to generate a standardized wastewater image sequence;

[0061] The structure prior embedding module is used to introduce radial and axial position codes of pipes into the standardized sewage image sequence to generate a structure-enhanced image feature sequence.

[0062] The anomaly segmentation analysis module is used to input the structure-enhanced image feature sequence into the improved Mask2Former image segmentation network and output a set of anomaly segmentation quantities that characterize the abnormal spatial distribution of sewage in the pipeline.

[0063] The structural consistency quantification module is used to generate wastewater state feature vectors based on the abnormal segmentation set, according to the radial distribution consistency constraint and axial continuity constraint of the pipeline.

[0064] The wastewater status determination module is used to perform wastewater status determination operations on the wastewater status feature vector and generate wastewater abnormal status identifiers.

[0065] The sewage discharge trigger control module is used to match the sewage abnormal status identifier with the preset sewage status threshold rules. When the match is successful, a sewage discharge trigger status identifier is generated and the sewage discharge process judgment mode is switched.

[0066] The recovery judgment and result output module is used to perform recovery judgment calculation on the sewage state feature vector of the sewage discharge process. When the recovery judgment is successful, a stop sewage discharge status flag is generated, forming the sewage monitoring result in the pipeline.

[0067] The beneficial effects of this invention are:

[0068] (1) This invention introduces radial and axial position coding of pipelines in the process of sewage image analysis, and combines structural consistency region quantization operation to achieve stable expression of sewage anomalies in spatial structure, so that the anomaly segmentation results are consistent with the actual geometric shape of the pipeline, reduce the impact of noise interference on the anomaly identification results, and improve the accuracy of sewage anomaly spatial positioning.

[0069] (2) In the abnormal segmentation stage, the present invention adopts an improved image segmentation network. By using a query group set and group attention update mechanism oriented towards sewage abnormal semantics, combined with the abnormal boundary perception refinement branch, the ability to distinguish between different abnormal semantics is enhanced, making the boundary of the abnormal area clearer and improving the stability and consistency of the abnormal segmentation results in complex sewage flow scenarios.

[0070] (3) The present invention constructs a state determination, discharge triggering and recovery determination process based on sewage state feature vector. It determines the abnormality persistence and recovery process through unified threshold rules, forming a complete monitoring and control closed loop, reducing the frequent switching of discharge triggers, and improving the reliability and operational safety of sewage monitoring process in pipelines. Attached Figure Description

[0071] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0072] Figure 1 This is an overall flowchart of the deep learning-based pipeline sewage monitoring method proposed in this invention.

[0073] Figure 2 This is a block diagram of the deep learning-based sewage monitoring system for pipelines proposed in this invention.

[0074] Figure 3 This is a schematic diagram of the structure of the improved Mask2Former image segmentation network proposed in this invention during the mask query stage and the mask prediction stage. Detailed Implementation

[0075] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0076] refer to Figures 1-3 A deep learning-based method for monitoring sewage in pipelines includes the following steps:

[0077] S1. Collect continuous image data of sewage in the pipeline and preprocess it to generate a standardized sewage image sequence;

[0078] S2. Introduce radial and axial position coding of pipelines into the standardized sewage image sequence, perform structural prior embedding operation, and generate a structure-enhanced image feature sequence.

[0079] S3. Input the structure-enhanced image feature sequence into the image segmentation network based on the Mask2Former architecture. In the mask query stage, construct several query groups according to the preset sewage anomaly semantic types. Perform group attention update operation on the query vector in each query group and the structure-enhanced image feature sequence to generate an updated query group set. In the mask prediction stage, use the updated query group set to generate anomaly region masks. At the same time, introduce anomaly boundary perception refinement branch in parallel to perform constraint correction processing on the boundary position of the anomaly region, forming an anomaly segmentation set predicted by joint constraints, which is used to characterize the spatial distribution of sewage anomalies in the pipeline.

[0080] S4. Based on the abnormal segmentation set, perform structural consistency region quantization operation according to the radial distribution consistency constraint and axial continuity constraint of the pipeline to generate sewage state feature vector;

[0081] S5. Perform wastewater state determination operation on the wastewater state feature vector to generate wastewater abnormal state identifier;

[0082] S6. Match the abnormal state identifier of sewage with the preset sewage state threshold rule. When the match is successful, generate a sewage discharge trigger state identifier and switch to the sewage discharge process judgment mode, and collect sewage image sequence of the sewage discharge process.

[0083] S7. Steps S1 to S4 are executed sequentially on the sewage image sequence during the sewage discharge process to generate a sewage state feature vector during the sewage discharge process. A recovery judgment operation is performed on the sewage state feature vector during the sewage discharge process. When the recovery judgment is successful, a stop sewage discharge status identifier is generated to form the sewage monitoring result in the pipeline.

[0084] In this embodiment, the continuous sewage image data collected in step S1 includes visible light image data collected continuously along the pipeline axis and the corresponding timestamp sequence; the preprocessing includes performing time alignment on the visible light image data according to the timestamp sequence to generate a time-synchronized sewage image sequence, performing joint operations of noise suppression and brightness correction to generate a brightness-stable sewage image sequence, and performing geometric consistency correction and unified resolution normalization operations to generate a standardized sewage image sequence.

[0085] In this embodiment, the process of introducing the radial and axial position codes of the pipeline and performing the structural prior embedding operation in step S2 includes:

[0086] S21. Perform grayscale conversion processing on each frame of the standardized sewage image sequence, perform edge detection operation on the grayscale image, and extract the set of continuous pipe wall edge pixels.

[0087] S22. Perform circular structure fitting processing on the set of pixels at the edge of the pipe wall to determine the center position of the pipe and the contour range of the inner wall of the pipe in the corresponding image.

[0088] S23. Using the center position of the pipe as a reference, perform radial distance mapping on the relative position of each pixel in the image coordinate system. Divide the radial distance into levels according to the pipe inner wall contour range to generate a radial position distribution map consistent with the image space size. Specifically, using the pipe inner wall contour range as the upper limit of the radial direction, divide the radial region between the pipe center position and the pipe inner wall contour into several concentric radial partitions according to a preset number of levels. For each pixel in the standardized sewage image sequence, determine the corresponding radial partition identifier based on the concentric radial partition where the pixel is located. Map the radial partition identifier corresponding to each pixel to a discrete position value and generate a radial position distribution map consistent with the image space size according to the image space arrangement.

[0089] S24. Assign different position labels to different radial hierarchical regions in the radial position distribution map in sequence, and expand the position labels along the image channel direction to form a radial position coding matrix.

[0090] S25. Establish an axial position index sequence based on the frame order of the standardized sewage image sequence, and divide the axial position index sequence into hierarchical segments according to the acquisition order to generate an axial position identifier sequence; specifically, based on the acquisition order of the standardized sewage image sequence, divide the continuously acquired image frames into several axial segments according to a preset number of frames, assign a unique axial level identifier to each axial segment, and assign the corresponding axial level identifier to the image frame according to the axial segment to which each image frame belongs, thus forming an axial position identifier sequence that corresponds one-to-one with the standardized sewage image sequence.

[0091] S26. Extend the axial position identifier sequence according to the frame index mapping to be consistent with the spatial size of the corresponding frame image, forming an axial position encoding matrix;

[0092] S27. Perform channel-by-channel interactive mapping processing on the radial position coding matrix and the axial position coding matrix to generate a structural prior coding matrix that simultaneously represents the radial and axial positional relationships of the pipeline. Specifically, perform pixel-by-pixel alignment processing on the radial position coding matrix and the axial position coding matrix in the same image frame according to the pixel spatial correspondence. For each pixel position, read the corresponding radial partition identifier and axial level identifier, and concatenate the radial partition identifier and axial level identifier into a two-dimensional index pair in a fixed order. Retrieve the coding entry that matches the two-dimensional index pair in the pre-established index pair coding table to obtain the joint position coding value. Write the joint position coding value into the corresponding pixel position and rearrange it in the image space to generate the structural prior coding matrix.

[0093] S28. Perform feature fusion processing on the structure prior coding matrix and the standardized sewage image sequence according to the channel dimension to generate a structure-enhanced image feature sequence.

[0094] In this embodiment, the process of constructing a query group set oriented towards sewage anomaly semantics and performing group attention update operation in step S3 during the mask query stage includes:

[0095] S31. Expand the feature sequence of the structure-enhanced image into a feature vector sequence according to its spatial location, and record the pixel spatial location index corresponding to each feature vector in the feature vector sequence.

[0096] S32. Establish a wastewater anomaly semantic set, and assign a query group identifier to each wastewater anomaly semantic based on the wastewater anomaly semantic set to form a query group identifier set;

[0097] S33. Generate an initial set of query vectors according to the set of query group identifiers, divide the initial set of query vectors into a set of query groups according to the set of query group identifiers, and arrange the query vectors of each query group in the set of query groups in a fixed order to form a query sequence within the group.

[0098] S34. Perform intra-group attention matching processing on each query group in the query group set. The intra-group attention matching processing includes matching the intra-group query sequence with the feature vector sequence one by one, extracting the local feature subset associated with the query vector from the feature vector sequence according to the matching result, and rearranging the local feature subset into an intra-group context sequence according to the pixel spatial position index.

[0099] S35. Perform intra-group fusion update processing on the intra-group query sequence and intra-group context sequence to generate an updated intra-group query sequence. The update processing specifically includes: S351. For each query vector in the intra-group query sequence, read the intra-group context sequence corresponding to the query vector; S352. Arrange the context feature vectors in the intra-group context sequence according to the pixel spatial position index order to form a context feature set that corresponds one-to-one with the query vector; S353. Perform channel-wise alignment processing on the query vector and the corresponding context feature set to map the context feature set to a feature representation with the same channel dimension as the query vector; S354. Merge the aligned context feature representation and the query vector according to the channel dimension to form a fused feature representation; S355. Perform nonlinear transformation and normalization processing on the fused feature representation to generate an updated query vector; S356. Replace the original query vector in the intra-group query sequence with the updated query vector to form an updated intra-group query sequence.

[0100] S36. After completing the intra-group fusion update process for each query group, inter-group isolation processing is performed between different query groups. The inter-group isolation processing includes writing the updated intra-group query sequences of different query groups into independent query caches, and reading the intra-group query sequences only from the corresponding query caches in the query update loop.

[0101] S37. Repeat steps S34 to S36 sequentially for the query group set until the preset update round is reached, and output the updated query group set.

[0102] In this embodiment, step S3, which introduces an anomaly boundary-aware refinement branch in the mask prediction stage and performs joint constraint prediction, includes the following steps:

[0103] S38. Input the structure-enhanced image feature sequence into the mask generation path, and perform a query-by-query mask generation process on the structure-enhanced image feature sequence according to the updated query group set to obtain an initial abnormal region mask set. In the mask generation path, the query-by-query mask generation process specifically includes: selecting the query vector in each query group in sequence according to the query group order in the updated query group set; for the selected single query vector, performing position-by-position association matching between the query vector and the structure-enhanced image feature sequence to generate a response feature map corresponding to the query vector; for each spatial position in the response feature map, generating a corresponding pixel-level mask value according to the response intensity at that position to form a single query mask map with the same spatial size as the original image; superimposing and merging multiple single query mask maps in the same query group according to the query order to obtain the initial abnormal region mask of the corresponding query group; repeating the above process for each query group to form an initial abnormal region mask set.

[0104] S39. Input the structure-enhanced image feature sequence into the anomaly boundary perception refinement branch in parallel, and perform boundary response extraction processing on the structure-enhanced image feature sequence to obtain an anomaly boundary response map; specifically, this includes: inputting the structure-enhanced image feature sequence into the anomaly boundary perception refinement branch in parallel, and performing layer-by-layer feature mapping on the structure-enhanced image feature sequence in the branch; during the feature mapping process, detecting feature changes in adjacent spatial positions and extracting boundary candidate features that reflect changes in the contour of the anomaly region; aggregating the boundary candidate features in the spatial dimension to generate a boundary response map with the same spatial size as the original image; in the boundary response map, the boundary response value reflects the probability that the corresponding pixel position belongs to the boundary of the anomaly region, and is used for boundary consistency constraint processing;

[0105] S310. Perform thresholding and connectivity filtering on the abnormal boundary response map to generate an abnormal boundary binary map. Specifically, for each pixel position in the abnormal boundary response map, read the corresponding boundary response value and compare it with a preset boundary threshold. When the boundary response value is not lower than the preset boundary threshold, mark the corresponding pixel position as a boundary candidate pixel. When the boundary response value is lower than the preset boundary threshold, mark the corresponding pixel position as a non-boundary pixel, thereby generating a boundary candidate binary map. Perform connectivity analysis on the boundary candidate pixels in the boundary candidate binary map, and divide the interconnected boundary candidate pixels into several boundary connected regions according to the pixel spatial adjacency relationship. Count the number of boundary candidate pixels contained in each boundary connected region, and determine the boundary connected regions with a number of pixels lower than the preset connectivity threshold as noise regions and remove them. Mark the pixel positions in the retained boundary connected regions as abnormal boundary pixels to generate an abnormal boundary binary map for mask boundary consistency correction processing.

[0106] S311. Perform pixel-by-pixel alignment processing on the initial abnormal region mask set and the abnormal boundary binary map according to the pixel spatial position index to determine the abnormal region mask boundary position set.

[0107] S312. Within the set of boundary positions of the abnormal region mask, perform boundary consistency correction processing on the initial abnormal region mask set based on the abnormal boundary binary map to form a boundary-refined abnormal region mask set. The boundary consistency correction processing specifically includes: determining the set of boundary pixel positions in the abnormal region mask based on the abnormal boundary binary map; for each boundary pixel in the boundary pixel position set, reading the mask value of the corresponding position in the initial abnormal region mask; performing consistency judgment between the mask value corresponding to the boundary pixel position and the boundary marker at that position in the abnormal boundary binary map; when the mask value is inconsistent with the boundary marker, performing correction processing on the mask value of the boundary pixel position to make the mask state at that position consistent with the boundary marker; after completing the correction processing of all boundary pixel positions, updating the initial abnormal region mask to generate the boundary-refined abnormal region mask.

[0108] S313. Perform hole filling and small connected component removal processing on the boundary refinement abnormal region mask set to generate an abnormal region consistent mask set.

[0109] S314. Merge the set of consistency masks for abnormal regions according to the query group identifier to form a set of abnormal segmentation quantities used to characterize the abnormal spatial distribution of sewage in the pipeline.

[0110] In this embodiment, the generation of the wastewater state feature vector includes:

[0111] S41. Based on the abnormal segmentation set, extract the abnormal region pixel set in each frame image, and map the abnormal region pixel set to the corresponding radial partition according to the pipeline radial position encoding to form an abnormal radial distribution sequence.

[0112] S42. Count the number of abnormal pixels in each radial partition in the abnormal radial distribution sequence, perform consistency judgment on the abnormal pixel distribution changes of adjacent radial partitions, perform suppression processing on the abnormal pixel set corresponding to the radial partition that fails the consistency judgment, and generate an abnormal region pixel set under radial consistency constraint.

[0113] Specifically, when applying radial distribution consistency constraints to the abnormal radial distribution sequence, the abnormal pixel count statistics of each radial partition are read sequentially from the inside out, and the abnormal pixel count statistics of the inner and outer radial partitions adjacent to the current radial partition are read simultaneously. The abnormal pixel count statistics of the current radial partition are compared with the abnormal pixel count statistics of the adjacent radial partitions. When the difference exceeds the preset radial consistency range, the current radial partition is marked as a radially inconsistent partition. When the difference is within the preset radial consistency range, the current radial partition is marked as a radially consistent partition. After marking all radial partitions, the abnormal region pixel set of each frame image in the abnormal segmentation set is traversed. The radial partition identifier corresponding to each abnormal pixel is read according to the radial position code determined in step S2. When the radial partition identifier belongs to the radially inconsistent partition, the abnormal pixel is suppressed and the corresponding pixel position is updated to a non-abnormal pixel. When the radial partition identifier belongs to the radially consistent partition, the abnormal pixel's abnormal label is retained, and an abnormal region pixel set under radial consistency constraints is generated.

[0114] S43. Based on the radial consistency constraint, the abnormal region pixel set is matched with the position of the abnormal region pixel set between adjacent image frames according to the frame order of the standardized sewage image sequence, forming an axial distribution sequence of abnormal regions.

[0115] Specifically, when performing position mapping on the set of pixels in abnormal regions, adjacent image frames are selected sequentially according to the frame order of the standardized sewage image sequence, and the set of pixels in the abnormal regions in the adjacent image frames is read respectively; the set of pixels in the abnormal regions in each frame is divided into several abnormal connected regions according to the spatial adjacency relationship of pixels, and the corresponding region center position is determined for each abnormal connected region; for each abnormal connected region in the current frame, abnormal connected regions in the set of pixels in the next frame that are within the adjacent spatial range of the region center position are searched, and the inter-frame mapping relationship is established according to the principle of minimum spatial distance; after the mapping relationship is established for all abnormal connected regions, an axial distribution sequence of abnormal regions for axial continuity determination is formed.

[0116] S44. Perform continuity determination on the axial distribution sequence of abnormal regions between adjacent image frames, and perform elimination processing on the set of abnormal region pixels that fail the continuity determination to generate a set of abnormal region pixels under structural consistency constraints.

[0117] Specifically, when performing continuity determination on the axial distribution sequence of abnormal regions, the correspondence between each abnormal connected region in the axial distribution sequence of abnormal regions in adjacent image frames is traversed along the frame order of the standardized sewage image sequence. When an abnormal connected region appears only in a single frame or a non-continuous image frame and does not form a continuous distribution in adjacent image frames, the abnormal connected region is determined to be an axially discontinuous region. When an abnormal connected region maintains a correspondence in multiple consecutive image frames, the abnormal connected region is determined to satisfy axial continuity. The pixel positions corresponding to the axially discontinuous regions are removed, and the pixel positions corresponding to the abnormal connected regions that satisfy axial continuity are retained, generating a set of abnormal region pixels under structural consistency constraints.

[0118] S45. Summarize the distribution of abnormal areas in each radial partition and axial segment to generate a wastewater state feature vector.

[0119] In this embodiment, step S5 specifically includes:

[0120] S51. Read the wastewater state feature vector and extract the value of each feature component in sequence according to the dimensional order of the wastewater state feature vector;

[0121] S52. Compare each feature component value with the corresponding threshold condition in the preset wastewater state threshold rule to generate a feature comparison result set.

[0122] S53. Perform state determination processing based on the feature comparison result set. When at least one feature component value meets the corresponding abnormal threshold condition, generate a sewage abnormal state identifier. When none of the feature component values ​​meet the abnormal threshold condition, generate a normal state identifier and output the sewage abnormal state identifier.

[0123] In this invention, the anomaly threshold condition is used to determine the state of the wastewater state feature vector generated in step S4. It serves as the basis for distinguishing between normal and abnormal wastewater states. The anomaly threshold condition is preset in the form of threshold rules and corresponds one-to-one with each feature component in the wastewater state feature vector. During the wastewater state determination operation, the wastewater state is determined to meet the anomaly determination condition by comparing each item. Specifically, as follows:

[0124] Anomaly threshold condition description based on radial distribution characteristics:

[0125] When the abnormal region is concentrated in adjacent partitions in the radial direction of the pipeline, and this distribution is stable, the corresponding feature component is determined to meet the abnormal threshold condition; when the abnormal region is scattered in the radial direction and does not form a concentrated distribution in adjacent partitions, the corresponding feature component is determined not to meet the abnormal threshold condition; through these abnormal threshold conditions, pipe wall adhesion anomalies and pseudo-anomalies caused by random noise can be distinguished.

[0126] Anomaly threshold condition description based on axial continuity features:

[0127] When the same abnormal region maintains a stable positional correspondence in adjacent frames of a standardized wastewater image sequence and appears repeatedly in multiple consecutive frames, the corresponding feature component is determined to meet the abnormal threshold condition. When the abnormal region appears only in a single frame or non-continuous frames and does not form an axially continuous distribution, the corresponding feature component is determined not to meet the abnormal threshold condition. This type of abnormal threshold condition is used to suppress misjudgments caused by transient interference or environmental noise.

[0128] Description of anomaly threshold conditions based on the size characteristics of the abnormal region:

[0129] When the pixel range occupied by the abnormal region in a single frame or multiple consecutive frames exceeds the preset size threshold, the corresponding feature component is determined to meet the abnormal threshold condition; when the abnormal region is always in a small range and does not reach the preset size threshold, the corresponding feature component is determined not to meet the abnormal threshold condition. Through this type of abnormal threshold condition, it is possible to avoid misjudging minor anomalies as sewage anomalies that require subsequent processing.

[0130] When performing wastewater state determination calculation on the wastewater state feature vector, each feature component is compared item by item according to the above-mentioned abnormal threshold conditions. When at least one feature component meets the corresponding abnormal threshold conditions, an abnormal wastewater state identifier is generated; when none of the feature components meet the abnormal threshold conditions, a normal state identifier is generated. Through the combined application of the above-mentioned abnormal threshold conditions, a stable and reliable determination of the wastewater state in the pipeline is achieved.

[0131] In this embodiment, step S6 specifically includes: reading the sewage abnormal state identifier, retrieving the trigger judgment condition corresponding to the sewage abnormal state identifier from the preset sewage state threshold rules; statistically analyzing the occurrence of the sewage abnormal state identifier within several consecutive judgment periods, and comparing the statistical results with the persistence requirement in the trigger judgment condition; when the sewage abnormal state identifier meets the trigger judgment condition within several consecutive judgment periods, generating a sewage discharge trigger state identifier, and switching the current monitoring process to the sewage discharge process judgment mode; in the sewage discharge process judgment mode, collecting sewage image data of the sewage discharge process according to the preset acquisition frequency to form a sewage image sequence of the sewage discharge process.

[0132] In this invention, the preset wastewater state threshold rules refer to a set of anomaly judgment rules pre-defined for each feature component in the wastewater state feature vector, used to uniformly manage the judgment conditions for abnormal wastewater states. The preset wastewater state threshold rules may include anomaly threshold conditions based on the radial distribution characteristics of the abnormal region, anomaly threshold conditions based on the axial continuity characteristics of the abnormal region, and anomaly threshold conditions based on the scale characteristics of the abnormal region. When performing wastewater state judgment and discharge trigger judgment, the wastewater state feature vector or wastewater abnormal state identifier is matched with the preset wastewater state threshold rules to determine the abnormal wastewater state and the discharge trigger timing.

[0133] The matching and triggering process of the trigger judgment condition is executed as follows: The abnormal wastewater state identifier is read, and the trigger judgment condition corresponding to that identifier is retrieved from the preset wastewater state threshold rules; a statistical window is established on a judgment cycle basis, and at the end of each judgment cycle, the current abnormal wastewater state identifier is recorded and the state sequence within the statistical window is updated; a consistency check is performed on the state sequence within the statistical window, and if a state identifier different from the current abnormal wastewater state identifier appears in the statistical window, the statistical window is cleared and recording begins again; provided the consistency check passes, a persistence check is performed on the state sequence within the statistical window, and if the abnormal wastewater state identifier appears continuously for multiple consecutive judgment cycles and meets the persistence requirement, the state is judged to meet the trigger judgment condition; if the persistence requirement is not met, the statistical window continues to be updated until the persistence requirement or trigger suppression condition is met; when the trigger judgment condition is met, a discharge trigger state identifier is generated and the system switches to discharge process judgment mode.

[0134] In this embodiment, the process of performing the recovery determination calculation and generating the stop sewage discharge status indicator in step S7 includes:

[0135] S71. Steps S1 to S4 are executed sequentially on the sewage image sequence of the sewage discharge process to generate a sewage state feature vector sequence arranged in chronological order.

[0136] S72. According to the preset judgment period, select the corresponding sewage state feature vectors from the sewage state feature vector sequence of the sewage discharge process in sequence, and compare each sewage state feature vector with the abnormal threshold conditions in the preset sewage state threshold rules item by item.

[0137] S73. When a feature component in the wastewater state feature vector within a certain judgment period meets the abnormal threshold condition, the judgment period is marked as an unrecovered period. When none of the feature components meet the abnormal threshold condition, the judgment period is marked as a candidate recovery period.

[0138] S74. Statistically analyze the marking results of several consecutive judgment cycles. When several consecutive judgment cycles are marked as candidate recovery cycles, generate a stop sewage discharge status indicator, end the sewage discharge process judgment mode, and form the sewage monitoring results in the pipeline.

[0139] The deep learning-based pipeline sewage monitoring system according to an embodiment of the present invention includes the following modules:

[0140] The wastewater image acquisition and preprocessing module is used to acquire continuous image data of wastewater in the pipeline and perform preprocessing to generate a standardized wastewater image sequence;

[0141] The structure prior embedding module is used to introduce radial and axial position codes of pipes into the standardized sewage image sequence to generate a structure-enhanced image feature sequence.

[0142] The anomaly segmentation analysis module is used to input the structure-enhanced image feature sequence into the improved Mask2Former image segmentation network and output a set of anomaly segmentation quantities that characterize the abnormal spatial distribution of sewage in the pipeline.

[0143] The structural consistency quantification module is used to generate wastewater state feature vectors based on the abnormal segmentation set, according to the radial distribution consistency constraint and axial continuity constraint of the pipeline.

[0144] The wastewater status determination module is used to perform wastewater status determination operations on the wastewater status feature vector and generate wastewater abnormal status identifiers.

[0145] The sewage discharge trigger control module is used to match the sewage abnormal status identifier with the preset sewage status threshold rules. When the match is successful, a sewage discharge trigger status identifier is generated and the sewage discharge process judgment mode is switched.

[0146] The recovery judgment and result output module is used to perform recovery judgment calculation on the sewage state feature vector of the sewage discharge process. When the recovery judgment is successful, a stop sewage discharge status flag is generated, forming the sewage monitoring result in the pipeline.

[0147] Example 1:

[0148] To verify the feasibility of this invention in practice, it was applied to a typical closed-loop drainage pipeline operation monitoring scenario. These pipelines operate in a high-humidity, high-corrosion, and high-turbidity environment, with complex internal water flow patterns, and the pipe walls are prone to deposits, abnormal obstruction, and localized corrosion. Traditional methods relying on manual inspection or single-frame image analysis are insufficient to reflect changes in the internal sewage state in a timely and stable manner, easily leading to problems such as discontinuous anomaly identification, false triggering of sewage discharge, or delayed recovery judgment. This embodiment addresses these issues by employing the deep learning-based pipeline sewage monitoring method proposed in this invention to continuously perceive, structure, and determine the operating status of sewage within the pipeline.

[0149] In this scenario, imaging devices are deployed along the inside of the pipeline to continuously acquire a sequence of images of wastewater flow. The acquired raw images suffer from uneven brightness, local occlusion, and water surface reflection, which can easily introduce noise if used directly for analysis. A wastewater image acquisition and preprocessing module performs uniform scaling, brightness correction, and noise suppression on the continuous images to form a stable, standardized wastewater image sequence, providing consistent input for subsequent analysis. Based on this, a structure prior embedding module is introduced to overlay radial and axial position codes of the pipeline onto the standardized wastewater image sequence, allowing image features to simultaneously carry pipeline geometric information. In this way, the model can distinguish between the region near the pipe wall and the central flow region when processing images, avoiding the aliasing of features from different spatial regions.

[0150] The enhanced image features are input into the anomaly segmentation and analysis module. This module employs an improved image segmentation network. During the mask query stage, a set of query groups oriented towards the semantics of wastewater anomalies is constructed. Different query groups focus on different spatial manifestations, such as attached anomalies and flow-induced occlusion anomalies. Feature aggregation is performed through a group attention update mechanism. In the mask prediction stage, an anomaly boundary perception refinement branch is introduced to constrain and correct the boundaries of anomaly regions, making the segmentation results more continuous and clear at the pipe wall edges. Through these processes, the output anomaly segmentation set can stably reflect the true spatial distribution of anomalies within the pipe, avoiding single-frame false detections or boundary breakage issues.

[0151] The structural consistency quantization module performs radial distribution consistency constraints and axial continuity constraints based on the abnormal segmentation set. Abnormal regions are first statistically analyzed and filtered according to their radial positions, and abnormal pixels that show abrupt changes in the radial direction and do not conform to the pipe attachment rules are removed. Then, the abnormal regions in adjacent image frames are matched and their continuity is determined in the axial direction, and interference regions that only appear in single frames or non-continuous frames are suppressed. After the above quantization process, a sewage state feature vector is generated, which can comprehensively reflect the scale, distribution stability and persistence characteristics of abnormal regions.

[0152] During the wastewater status determination stage, the wastewater status determination module compares the wastewater status feature vector item by item to generate an abnormal wastewater status identifier. Unlike the traditional single determination, in this embodiment, the abnormal status must remain stable within multiple consecutive determination cycles to be recognized as a valid abnormality, thereby reducing the false alarm rate. When the wastewater abnormal status identifier matches the preset wastewater status threshold rule, the discharge trigger control module generates a discharge trigger status identifier and enters the discharge process determination mode. During the discharge process, the system continuously collects wastewater image sequences, repeatedly executes the aforementioned image analysis and structure quantization process, and generates the wastewater status feature vector of the discharge process in real time.

[0153] During the recovery determination phase, the recovery determination and result output module continuously determines the sewage state feature vector of the discharge process. When the sewage state feature vector does not meet the abnormal threshold condition within multiple consecutive determination cycles, the system determines that the sewage state has stabilized, generates a stop discharge status flag, and completes a complete closed loop of sewage monitoring in the pipeline. In this way, premature termination or ineffective extension of discharge is avoided, and operational stability is improved.

[0154] In this embodiment, statistical analysis was performed on continuous running data by comparing the method of the present invention with conventional image segmentation analysis methods that do not introduce structural priors and consistency constraints. The results show that the present invention has significant advantages in terms of anomaly segmentation stability, false detection suppression capability, and accuracy of pollution discharge triggering and recovery determination. The consistency of abnormal regions in continuous frames is significantly improved, the number of false pollution discharge triggers is significantly reduced, the pollution discharge duration is more reasonable, and the overall monitoring process is more in line with the actual operating state.

[0155] Table 1 presents a comparison of key operational data based on this embodiment, which visually demonstrates the beneficial effects of the present invention in practical applications.

[0156] Table 1: Comparison of Application Effects of Sewage Monitoring Methods in Pipelines

[0157]

[0158] As can be seen from Table 1, from the perspective of anomaly segmentation stability, conventional image analysis methods only perform at a moderate level in terms of the consistency rate of anomaly segmentation in consecutive frames. The shape and position of the anomaly region in adjacent images are prone to jumps, resulting in discontinuous segmentation results. The method of this invention, by introducing structural prior embedding and structural consistency quantification mechanism, maintains stable constraints on the anomaly segmentation results in both radial and axial spatial dimensions, thereby improving the consistency rate of anomaly segmentation in consecutive frames. This indicates that the spatial representation of anomalies is more consistent with the actual operating state inside the pipeline.

[0159] In terms of false detection suppression, conventional methods still have a relatively high false detection trigger ratio in a single frame. The main reason is that water surface reflection, floating objects, or momentary occlusion are easily misidentified as anomalies. This invention effectively suppresses abnormal regions that only appear in a single frame or non-continuous frames through a grouped attention mechanism and axial continuity determination, which significantly reduces the false detection trigger ratio in a single frame and makes the anomaly determination results more reliable.

[0160] Regarding the integrity of abnormal region boundaries, conventional methods often generate abnormal region boundaries that are broken or blurred, making it difficult to support subsequent accurate quantitative analysis. This invention introduces an abnormal boundary perception refinement branch in the abnormal segmentation stage to perform joint constraint correction on the boundary region, making the abnormal region outline more complete and continuous, improving the boundary integrity index, and providing higher quality input for subsequent structural consistency quantification.

[0161] Analysis of the sewage discharge triggering and control effects shows that conventional methods have significant shortcomings in terms of sewage discharge triggering accuracy, and are prone to false triggering or delayed triggering. This invention, by matching the sewage state feature vector with the preset sewage state threshold rules and combining it with the determination of abnormal persistence, makes sewage discharge triggering more in line with actual operational needs, significantly improves the accuracy of sewage discharge triggering, and reduces the incidence of premature termination of sewage discharge, indicating that the control logic is more stable.

[0162] Regarding the operational effectiveness after sewage discharge, conventional methods, due to insufficient recovery judgment, are prone to recurring abnormal states shortly after sewage discharge, resulting in repeated sewage discharge. This invention, by performing continuous recovery judgment on the sewage state feature vector during the sewage discharge process, effectively confirms the true recovery status of the sewage state, significantly reduces the abnormal recurrence rate, and improves the overall monitoring stability score, demonstrating the comprehensive advantages of this invention under long-term operating conditions.

[0163] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based method for monitoring sewage in pipelines, characterized in that, Includes the following steps: S1. Collect continuous image data of sewage in the pipeline and preprocess it to generate a standardized sewage image sequence; S2. Introduce radial and axial position coding of pipelines into the standardized sewage image sequence, perform structural prior embedding operation, and generate a structure-enhanced image feature sequence. S3. Input the structure-enhanced image feature sequence into the image segmentation network based on the Mask2Former architecture. In the mask query stage, construct several query groups according to the preset sewage anomaly semantic type. Perform group attention update operation on the query vector in each query group and the structure-enhanced image feature sequence to generate the updated query group set. In the mask prediction stage, the updated query set is used to generate the mask for the abnormal region. At the same time, an abnormal boundary perception refinement branch is introduced in parallel to perform constraint correction processing on the boundary position of the abnormal region, forming a set of abnormal segmentation quantities predicted by joint constraints, which is used to characterize the spatial distribution of sewage anomalies in the pipeline. S4. Based on the abnormal segmentation set, perform structural consistency region quantization calculation according to the radial distribution consistency constraint and axial continuity constraint of the pipeline to generate a wastewater state feature vector; specifically including: S41. Based on the abnormal segmentation set, extract the abnormal region pixel set in each frame image, and map the abnormal region pixel set to the corresponding radial partition according to the pipeline radial position encoding to form an abnormal radial distribution sequence. S42. Count the number of abnormal pixels in each radial partition in the abnormal radial distribution sequence, perform consistency judgment on the abnormal pixel distribution changes of adjacent radial partitions, perform suppression processing on the abnormal pixel set corresponding to the radial partition that fails the consistency judgment, and generate an abnormal region pixel set under radial consistency constraint. S43. Based on the radial consistency constraint, the abnormal region pixel set is matched with the position of the abnormal region pixel set between adjacent image frames according to the frame order of the standardized sewage image sequence, forming an axial distribution sequence of abnormal regions. S44. Perform continuity determination on the axial distribution sequence of abnormal regions between adjacent image frames, and perform elimination processing on the set of abnormal region pixels that fail the continuity determination to generate a set of abnormal region pixels under structural consistency constraints. S45. Summarize the distribution of abnormal areas in each radial partition and axial segment to generate a wastewater state feature vector. S5. Perform wastewater state determination operation on the wastewater state feature vector to generate wastewater abnormal state identifier; S6. Match the abnormal state identifier of sewage with the preset sewage state threshold rule. When the match is successful, generate a sewage discharge trigger state identifier and switch to the sewage discharge process judgment mode, and collect sewage image sequence of the sewage discharge process. S7. Steps S1 to S4 are executed sequentially on the sewage image sequence during the sewage discharge process to generate a sewage state feature vector during the sewage discharge process. A recovery judgment operation is performed on the sewage state feature vector during the sewage discharge process. When the recovery judgment is successful, a stop sewage discharge status identifier is generated to form the sewage monitoring result in the pipeline.

2. The deep learning-based method for monitoring sewage in pipelines according to claim 1, characterized in that, The continuous sewage image data collected in step S1 includes visible light image data collected continuously along the pipeline axis and the corresponding timestamp sequence; the preprocessing includes performing time alignment on the visible light image data according to the timestamp sequence to generate a time-synchronized sewage image sequence, performing joint operations of noise suppression and brightness correction to generate a brightness-stable sewage image sequence, and performing geometric consistency correction and unified resolution normalization operations to generate a standardized sewage image sequence.

3. The deep learning-based method for monitoring sewage in pipelines according to claim 2, characterized in that, The process of introducing radial and axial position codes for the pipeline and performing structural prior embedding calculations in step S2 includes: S21. Perform grayscale conversion processing on each frame of the standardized sewage image sequence, perform edge detection operation on the grayscale image, and extract the set of continuous pipe wall edge pixels. S22. Perform circular structure fitting processing on the set of pixels at the edge of the pipe wall to determine the center position of the pipe and the contour range of the inner wall of the pipe in the corresponding image. S23. Using the center position of the pipe as a reference, perform radial distance mapping on the relative position of each pixel in the image coordinate system, and classify the radial distance according to the contour range of the inner wall of the pipe to generate a radial position distribution map consistent with the image space size. S24. Assign different position labels to different radial hierarchical regions in the radial position distribution map in sequence, and expand the position labels along the image channel direction to form a radial position coding matrix. S25. Based on the frame order of the standardized sewage image sequence, establish an axial position index sequence, divide the axial position index sequence into levels according to the acquisition order, and generate an axial position identifier sequence. S26. Extend the axial position identifier sequence according to the frame index mapping to be consistent with the spatial size of the corresponding frame image, forming an axial position encoding matrix; S27. Perform channel-by-channel interactive mapping processing on the radial position coding matrix and the axial position coding matrix to generate a structural prior coding matrix that simultaneously represents the radial and axial positional relationships of the pipeline. S28. Perform feature fusion processing on the structure prior coding matrix and the standardized sewage image sequence according to the channel dimension to generate a structure-enhanced image feature sequence.

4. The deep learning-based method for monitoring sewage in pipelines according to claim 3, characterized in that, The process of constructing a set of query groups oriented towards sewage anomaly semantics and performing group attention update operations in step S3 during the mask query phase includes: S31. Expand the feature sequence of the structure-enhanced image into a feature vector sequence according to its spatial location, and record the pixel spatial location index corresponding to each feature vector in the feature vector sequence. S32. Establish a wastewater anomaly semantic set, and assign a query group identifier to each wastewater anomaly semantic based on the wastewater anomaly semantic set to form a query group identifier set; S33. Generate an initial set of query vectors according to the set of query group identifiers, divide the initial set of query vectors into a set of query groups according to the set of query group identifiers, and arrange the query vectors of each query group in the set of query groups in a fixed order to form a query sequence within the group. S34. Perform intra-group attention matching processing on each query group in the query group set. The intra-group attention matching processing includes matching the intra-group query sequence with the feature vector sequence one by one, extracting the local feature subset associated with the query vector from the feature vector sequence according to the matching result, and rearranging the local feature subset into an intra-group context sequence according to the pixel spatial position index. S35. Perform intra-group fusion update processing on the intra-group query sequence and intra-group context sequence to generate the updated intra-group query sequence. S36. After completing the intra-group fusion update process for each query group, inter-group isolation processing is performed between different query groups. The inter-group isolation processing includes writing the updated intra-group query sequences of different query groups into independent query caches, and reading the intra-group query sequences only from the corresponding query caches in the query update loop. S37. Repeat steps S34 to S36 sequentially for the query group set until the preset update round is reached, and output the updated query group set.

5. The deep learning-based method for monitoring sewage in pipelines according to claim 4, characterized in that, In step S3, the process of introducing an anomaly boundary-aware refinement branch in the mask prediction stage and performing joint constraint prediction includes: S38. Input the structure-enhanced image feature sequence into the mask generation path, and perform query-by-query mask generation processing on the structure-enhanced image feature sequence according to the updated query group set to obtain the initial abnormal region mask set. S39. Input the structure-enhanced image feature sequence into the abnormal boundary perception refinement branch in parallel, and perform boundary response extraction processing on the structure-enhanced image feature sequence to obtain the abnormal boundary response map. S310. Perform thresholding and connectivity filtering on the abnormal boundary response map to generate an abnormal boundary binary map. S311. Perform pixel-by-pixel alignment processing on the initial abnormal region mask set and the abnormal boundary binary map according to the pixel spatial position index to determine the abnormal region mask boundary position set. S312. Within the set of abnormal region mask boundaries, perform boundary consistency correction processing on the initial abnormal region mask set based on the abnormal boundary binary map to form a boundary-refined abnormal region mask set. S313. Perform hole filling and small connected component removal processing on the boundary refinement abnormal region mask set to generate an abnormal region consistent mask set. S314. Merge the set of consistency masks for abnormal regions according to the query group identifier to form a set of abnormal segmentation quantities used to characterize the abnormal spatial distribution of sewage in the pipeline.

6. The deep learning-based method for monitoring sewage in pipelines according to claim 5, characterized in that, Step S5 specifically includes: S51. Read the wastewater state feature vector and extract the value of each feature component in sequence according to the dimensional order of the wastewater state feature vector; S52. Compare each feature component value with the corresponding threshold condition in the preset wastewater state threshold rule to generate a feature comparison result set. S53. Perform state determination processing based on the feature comparison result set. When at least one feature component value meets the corresponding abnormal threshold condition, generate a sewage abnormal state identifier. When none of the feature component values ​​meet the abnormal threshold condition, generate a normal state identifier and output the sewage abnormal state identifier.

7. The deep learning-based method for monitoring sewage in pipelines according to claim 6, characterized in that, Step S6 specifically includes: reading the sewage abnormal state identifier, retrieving the trigger judgment condition corresponding to the sewage abnormal state identifier from the preset sewage state threshold rules; statistically analyzing the occurrence of the sewage abnormal state identifier within several consecutive judgment periods, and comparing the statistical results with the persistence requirement in the trigger judgment condition; when the sewage abnormal state identifier meets the trigger judgment condition within several consecutive judgment periods, generating a sewage discharge trigger state identifier, and switching the current monitoring process to the sewage discharge process judgment mode; in the sewage discharge process judgment mode, collecting sewage image data of the sewage discharge process according to the preset acquisition frequency to form a sewage image sequence of the sewage discharge process.

8. The deep learning-based method for monitoring sewage in pipelines according to claim 7, characterized in that, The process of performing the recovery determination operation and generating the stop sewage discharge status indicator in step S7 includes: S71. Steps S1 to S4 are executed sequentially on the sewage image sequence of the sewage discharge process to generate a sewage state feature vector sequence arranged in chronological order. S72. According to the preset judgment period, select the corresponding sewage state feature vectors from the sewage state feature vector sequence of the sewage discharge process in sequence, and compare each sewage state feature vector with the abnormal threshold conditions in the preset sewage state threshold rules item by item. S73. When a feature component in the wastewater state feature vector within a certain judgment period meets the abnormal threshold condition, the judgment period is marked as an unrecovered period. When none of the feature components meet the abnormal threshold condition, the judgment period is marked as a candidate recovery period. S74. Statistically analyze the marking results of several consecutive judgment cycles. When several consecutive judgment cycles are marked as candidate recovery cycles, generate a stop sewage discharge status indicator, end the sewage discharge process judgment mode, and form the sewage monitoring results in the pipeline.

9. A deep learning-based pipeline sewage monitoring system, applied to any one of the deep learning-based pipeline sewage monitoring methods according to claims 1 to 8, characterized in that, Includes the following modules: The wastewater image acquisition and preprocessing module is used to acquire continuous image data of wastewater in the pipeline and perform preprocessing to generate a standardized wastewater image sequence; The structure prior embedding module is used to introduce radial and axial position codes of pipes into the standardized sewage image sequence to generate a structure-enhanced image feature sequence. The anomaly segmentation analysis module is used to input the structure-enhanced image feature sequence into the improved Mask2Former image segmentation network and output a set of anomaly segmentation quantities that characterize the abnormal spatial distribution of sewage in the pipeline. The structural consistency quantification module is used to generate wastewater state feature vectors based on the abnormal segmentation set, according to the radial distribution consistency constraint and axial continuity constraint of the pipeline. The wastewater status determination module is used to perform wastewater status determination operations on the wastewater status feature vector and generate wastewater abnormal status identifiers. The sewage discharge trigger control module is used to match the sewage abnormal status identifier with the preset sewage status threshold rules. When the match is successful, a sewage discharge trigger status identifier is generated and the sewage discharge process judgment mode is switched. The recovery judgment and result output module is used to perform recovery judgment calculation on the sewage state feature vector of the sewage discharge process. When the recovery judgment is successful, a stop sewage discharge status flag is generated, forming the sewage monitoring result in the pipeline.

Images