A sewer pipe video defect processing method based on deep learning and sequence interpretation

Abstract

The application discloses a kind of based on deep learning and sequence interpretation sewer video defect processing method, including the new video sequence obtained by video stream pre-processing of CCTV video, sequence frame-by-frame identification model is constructed and is used to frame-by-frame identification new video sequence, form video sequence with defect identification, with defect identification video sequence is interpreted by video sequence interpretation method, the video sequence after interpretation is limited to form final video sequence by video sequence constraint strategy, then construct the key frame of visual and output representative corresponding defect basic morphology, solved the difficult problem of current CCTV video interpretation work, low operational efficiency, provide solid theoretical basis and technical support for the scientific maintenance and wisdom of city sewer.

Description

Technical Field

[0001] This invention relates to a method for processing defects in drainage pipe videos based on deep learning and sequence interpretation. Background Technology

[0002] Drainage pipes are prone to defects and have complex internal environments, making CCTV inspection one of the most widely used technologies for pipe defect detection. Robots equipped with high-resolution cameras and data transmission systems inspect internal pipe defects; however, current CCTV video interpretation and review rely on labor-intensive and costly manual methods. For example, 1 km of drainage pipe generates approximately 20GB of CCTV inspection data, taking about 18.8 hours in total, with a single person taking up to 27 hours for interpretation. Furthermore, under the sixteen defect classification systems, the similarity between defects, coupled with differences in human subjective interpretation, leads to varying inspection results from different personnel, resulting in low accuracy.

[0003] Most studies are based on CCTV images, and research on fast and accurate identification frameworks for video sequences is relatively insufficient. In actual engineering projects with complex situations, current intelligent detection technology for drainage pipeline defects cannot meet the needs of drainage pipeline condition assessment. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a video defect processing method for drainage pipelines. This method realizes the process from video input to outputting the number of defects, time range, and representative graphics in complex scenarios, promoting the transformation of drainage pipeline defect detection and diagnosis from experience-based to data-driven, and providing a solid theoretical foundation and technical support for the scientific maintenance and intelligent upgrading of urban drainage pipelines.

[0005] To achieve the above objectives, the technical solution provided in this application is as follows: A method for processing video defects in drainage pipes, comprising the following steps: S1: CCTV video is preprocessed to obtain a new video sequence; S2: Construct a frame-by-frame sequence recognition model and use it to recognize new video sequences frame by frame, forming video sequences with defect identifiers; S3: Decode video sequences with defect markers using video sequence decoding methods; S4: The decoded video sequence is constrained by a video sequence constraint strategy to form the final video sequence; S5: Construct and output keyframes that represent the basic form of the corresponding defects.

[0006] Furthermore, step S1 includes the following steps: S11: Set the downsampling rate and combine it with the image blur detection method to select N representative frames from CCTV video. The set of N representative frames forms a preliminary video sequence. The attribute of the representative frames is the collection of CCTV video frame numbers and content between adjacent representative frames. S12: Establish a lightweight Pipe Focus Net model to mark and remove frames in the initial video sequence that show the external environment of the pipeline and the manhole to form a simplified video sequence; S13: The simplified video sequence is obtained by frame-by-frame denoising using the Non-Local Means algorithm.

[0007] Furthermore, step S2 includes the following steps: S21: Establish a defect database based on 16 defect types and form a training dataset with polygon annotations; S22: Construct a YOLO v12 object detection network, use a dataset to train and validate the model, and adjust the model structure and parameters to form a frame-by-frame recognition model; S23: The frame-by-frame recognition model identifies all representative frames in a new video sequence and records the defect type of each representative frame. All representative frames are classified according to their defect types and form multiple different defect sequence windows. These multiple different defect sequence windows form a video sequence with defect identifiers.

[0008] Furthermore, step S3, the video interpretation method, includes: Occasional false positives are removed using a multi-threshold cross-screening strategy; Based on the visual characteristics analysis of defects, the sequence windows of easily confused defects are sorted out. Easily confused defects include at least one of the following: hidden connection of branch pipe and foreign object penetration, deposit and obstacle, misalignment and disconnection, corrosion and scale, and falling off and tree root.

[0009] Furthermore, the consolidation specifically includes: ① In cases of confusion between concealed branch pipe connections and foreign object penetration, if the defect sequence windows of the two are no more than M seconds apart, they are determined to be the same defect, and the defect in the second to last second is taken as the final defect type. ② To address the confusion between sediments and obstacles, all data within M seconds between windows (referring to the time between sediment sequence windows and obstacle sequence windows) are classified as sediments; ③ Regarding the confusion between misaligned and disconnected windows, any window within M seconds of each other is classified as a disconnected window; ④ Regarding the confusion between corrosion and scaling, any mixture of corrosion and scaling occurring within a window tolerance of M seconds is classified as corrosion; ⑤ In cases of confusion between detachment and root, the defect type is determined by the second-to-last second within an M-second tolerance.

[0010] Furthermore, step S4 includes: S41: Construct a "pipe wall-pipeline" scene recognition based on a PFN network, and mark the pipe wall scenes in the video sequence to form a pipe wall scene sequence window; S42: Establish pipe wall scene and pipeline material restriction strategies for different defect sequence windows, and connect the pipe wall scene sequence windows to remove, merge and / or not output different defect sequences.

[0011] Furthermore, step S5 specifically includes: S51: Establish a visualization map using the final video sequence, defect type, and scene classification as visualization content; S52: Based on the defect identification results and sequence status, output the corresponding frame of the basic defect morphology through the keyframe output strategy.

[0012] Furthermore, the key frame output strategy includes: firstly, the corresponding defect must be identified in the frame of the final video sequence; then, from the middle frame of the corresponding defect sequence window, key frames are searched. If a key frame is a frame in the defect sequence window where multiple frames of the same type of defect are identified, or if there are no multiple frames of the same type of defect in the defect sequence window, then the image frame with the most identified defect types is selected.

[0013] Furthermore, the lightweight Pipe Focus Net model includes a backbone network and a detection head. The backbone network consists of three feature extraction blocks, each of which is composed of a convolutional layer, a rectified linear unit, and a max pooling layer. The detection head is designed with a two-stage output structure for binary classification.

[0014] Beneficial effects: The drainage pipeline defect processing method based on deep learning and sequence interpretation proposed in this application adopts video stream preprocessing, frame-by-frame recognition model, video sequence interpretation method, video sequence constraint strategy, visualization and keyframe output method to solve the problems of difficult, complex, time-consuming and costly CCTV video recognition. It realizes an integrated process from video input to defect output, promotes the transformation of drainage pipeline defect detection and diagnosis from experience-based to data-driven, and provides a solid theoretical foundation and technical support for the scientific maintenance and intelligent organic upgrading of urban drainage pipelines.

[0015] The video stream preprocessing in this application sets the downsampling rate, simplifies the image by using a lightweight Pipe Focus Net model and performs image denoising, removes useless sequences from the video stream, enhances image quality, and lays a good foundation for subsequent sequence interpretation.

[0016] The frame-by-frame recognition model in this application is fine-tuned based on training to further optimize the model's running efficiency.

[0017] The video sequence interpretation method proposed in this application establishes five pairs of easily confused defect identification logics based on visual feature similarity analysis between different defects and the perception ability of computer vision models to the existing sixteen-category defect classification system. It merges the complex situation of "one defect, multiple identification results" to achieve higher accuracy identification results at the video level.

[0018] The video sequence constraint strategy proposed in this application establishes a recognition model for pipe wall surround shooting scenes based on PFN network, and formulates a targeted constraint strategy for the visual manifestation of different defects in this scene; and finally, using the pipe material as a constraint, unreasonable defect recognition results are deleted.

[0019] The proposed visualization and keyframe output method establishes a visualization template to intuitively display information such as the time range of defect occurrence, defect type, and number of defects, facilitating secondary verification of results by engineers. A detailed keyframe extraction strategy is formulated to ensure that no defects are missed and that defects are centered, which is convenient for subsequent report production. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is a diagram of the Pipe Focus Net (PFN) network structure. Figure 3 The images show typical scenes inside pipelines, external environments of pipelines and well scenes, as well as circumferential shooting scenes of pipeline walls. Figure 4 Training and validation results for the recognition model of pipeline external environment and well scene; Figure 5 Identify logic diagrams for pipeline external and manhole scenarios; Figure 6 This is a schematic diagram of the NL-Means denoising algorithm; Figure 7 This is a diagram of the YOLO v12 network architecture. Figure 8 This is a schematic diagram of the Window Attention mechanism. Figure 9 To address the confusion between concealed branch pipe connections and foreign object penetration defects; Figure 10 For scenarios where confusion arises between deposits and obstacle defects; Figure 11 To identify the confusion between misalignment and disconnection defects; Figure 12To address the confusion between corrosion and scaling defects; Figure 13 To identify the confusion between detachment and root defects; Figure 14 The training and validation results of the pipeline wall circumferential shooting scene recognition model; Figure 15 Example of logical constraints for a pipe wall scenario; Figure 16 Example of a visualization of video recognition results. Detailed Implementation

[0021] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0022] For reference Figure 1 As shown, this application provides a method for handling defects in drainage pipes based on deep learning and sequence interpretation, including the following steps: S1: CCTV video is preprocessed to obtain a new video sequence; S2: Construct a frame-by-frame sequence recognition model and use it to recognize new video sequences frame by frame, forming video sequences with defect identifiers; S3: Decipher video sequences with defect markers using video sequence interpretation methods; S4: The decoded video sequence is constrained by a video sequence constraint strategy to form the final video sequence; S5: Construct and output keyframes that represent the basic form of the corresponding defects.

[0023] The specific steps for S1 include: A downsampling rate is set, and N relatively clear images are selected from CCTV video as representative frames using an image blur detection method. This set of N representative frames forms a preliminary video sequence, where the representative frame attribute is the sum of the number of CCTV video frames corresponding to adjacent representative frames. Then, a lightweight Pipe Focus Net (PFN) model is established to mark and remove frames of the external environment of the pipeline and manholes appearing in the CCT video, forming a simplified video sequence. Finally, based on the Non-Local Means (NL-means) algorithm, the simplified video sequence is denoised frame by frame to form a new video sequence. Specifically, taking a video with a frame rate of 24 as an example, a frame is selected at 12-frame intervals, and relatively clear images are selected as representative frames based on the Plas operator to construct the preliminary video sequence.

[0024] Where △I(x, y) is the response value of the image after convolution with the Laplacian operator; I(x, y) is the brightness value of the image. When the blurriness of the image is higher than the set threshold of 2500, the image is considered unsuitable for subsequent recognition processing, and a relatively clearer frame is searched between the two preceding and following frames. If there are no frames with blurriness lower than the set threshold of 2500, the frame with the lowest blurriness is used.

[0025] Then, a lightweight Pipe Focus Net (PFN) model is built. See model details below. Figure 2 The PFN backbone network consists of three feature extraction blocks, each composed of a convolutional layer, a rectified linear unit (ReLU), and a max pooling layer. For all modules in the backbone, the core size, padding, and stride are set to 3×3, 1, and 2, respectively. Therefore, the output feature map after passing through the backbone network has a shape of 128, 18, 18. In the detection head, a two-level structure is designed for binary classification output. Furthermore, the model implements an Early Stop Policy (ESP), allowing training to stop if the model's performance metrics do not improve after 5 training epochs. This case study collected 300 images of the interior of pipes and 200 images of the external environment and manholes; see examples below. Figure 3 Adam was set as the optimizer, and the learning rate was adjusted to 0.001. The model was trained on a Windows system using an AMD EPYC 9534 GPU and two NVIDIA GeForce RTX A6000 GPUs (48GB RAM) connected to NVLink. See [link to training results] for the final training results and curves. Figure 4 The results show that the model triggers the ESP policy around 30 times, thus stopping training. On the validation set, the model achieved an accuracy of approximately 0.9 and a recall of approximately 0.7. However, in long-sequence video detection, a small number of errors is acceptable, as the loss of a few frames does not significantly affect the overall detection results.

[0026] Secondly, based on the PFN-based pipeline exterior / manhole model, the system starts identifying scenes from the first and last frames of the video sequence towards the middle. If five consecutive frames are identified as scenes inside the pipeline, the process stops. The middle sequence window represents the shooting scene inside the pipeline. (See [link to relevant documentation]). Figure 5 .

[0027] Finally, based on the Non-Local Means (NL-means) algorithm, frame-by-frame denoising is performed on the video sequence. See [link / reference]. Figure 6 :

[0028] Where x is the target region; y is the adjacent region; I is the search window, which is set to 9 in this case; u(x) is the pixel value after denoising; w(x,y) is the similarity weight between regions x and y; and v(y) is the pixel value of the noisy region y. is the Euclidean distance; h is the smoothing parameter, which is set to 0.8 in this case.

[0029] The specific content of step S2 includes: S21: Establish a defect database based on 16 defect types and form a training dataset with polygon annotations; S22: Construct a YOLO v12 object detection network, use a dataset to train and validate the model, and adjust the model structure and parameters to form a frame-by-frame recognition model; S23: The frame-by-frame recognition model identifies all representative frames in a new video sequence and records the defect type of each representative frame. All representative frames are categorized according to defect type, forming multiple different defect sequence windows. These multiple defect sequence windows form a video sequence with defect labels. Specifically, a large number of images of sixteen defect categories are collected (see Table 1), totaling 29,656. LabelMe software is used to annotate all defects with bounding boxes, creating a standard dataset in VOC format. 80% of the data is used for training, and 20% is used for validation.

[0030] Then, a YOLO v12-based object detection network was built to identify defects in the video sequence frame by frame. (See [link to relevant documentation]). Figure 7 In the network, we optimized the original Area Attention model mechanism to a Window Attention mechanism (see...). Figure 8To reduce the number of computational parameters in redundant regions, the RAdam optimizer was used, whose dynamic parameters make model training smoother, with a standard learning rate of 0.001. Model training was based on the pre-trained YOLO12s model, and the training results are shown in Table 2. The results show that for 16 defect categories, the average precision is 0.576 and the average recall is 0.726. These metrics are relatively low because there are many similar scenarios in the 16-level defect system, such as misalignment and dislocation. Specifically, the precision and recall of foreign object penetration, leakage, and detachment are relatively low because their visual features are complex, difficult to extract, and easily confused. For misalignment and dislocation, only the precision is relatively low because the two defects are not only easily confused but also identified simultaneously in the same joint problem, resulting in high recall and low precision. Since the degree of fluctuation is difficult to define and its visual features cover a wide range, it poses a challenge to the accurate recognition of detection models. Therefore, fluctuation has a high recall rate and a low precision rate. Breaks show relatively low precision and recall rate because they occur in complex scenes and their visual features are very small, making them difficult to capture at the global level.

[0031] Table 1

[0032] Table 2

[0033] The specific content of step S2 includes: Since confusion can easily occur among numerous defects, a video sequence interpretation algorithm is constructed, and a multi-threshold cross-screening strategy is established to remove occasional misidentification scenarios. Based on the visual feature analysis of defects, easily confused defect sequences are reorganized and summarized. First, depending on the type of defect, the Tolerance parameter is set to 2.0 or 1.5. 2.0 is for corrosion, scum, and scaling issues, while 1.5 is for other defect categories. For example, two frames identified as disconnected with a 1-second interval are still considered to belong to the same defect; the 1-second interval is attributed to image blurring or viewpoint changes. Furthermore, according to inspection specifications and operational practices, defects generally appear in the video for at least 3 seconds. Then, the Window_Size parameter is used to filter areas where defects appear for less than 6 seconds or 3 seconds respectively. 6 seconds is for corrosion, scum, and scaling, and 3 seconds is for other defect categories. Because corrosion, scum, and scaling often span almost the entire pipe, larger Tolerance and Window_Size parameters are set to improve detection accuracy and remove certain scene sequence windows.

[0034] Then, based on the visual characteristics analysis of the defects, the easily confused defect sequences were reorganized and summarized. The easily confused defects include 5 pairs: hidden branch pipe connection and foreign object penetration, deposit and obstacle, misalignment and disconnection, corrosion and scale, and detachment and tree root.

[0035] See Figure 9 To address the confusion between concealed pipe connections and foreign object penetration, since concealed pipe connections and foreign object penetration have very similar visual characteristics at a distance, and closer inspection is more accurate, if the video sequence windows for concealed pipe connections and foreign object penetration differ by no more than 1 second, they are considered to represent the same defect, and the defect identified in the second-to-last second is taken as the final result (see [link]). Figure 9 The situation suggests the result should be a concealed connection of the branch pipe.

[0036] See Figure 10 To address the confusion between sediments and obstacles, since both sediments and obstacles can be materials settled at the bottom of the pipe in some cases, making them difficult for the model to distinguish, any confusion between sediments and obstacles within 1 second between windows is completely classified as sediment (see [link]). Figure 10 The situation suggests that the result should be sedimentation.

[0037] See Figure 11 To address the confusion between misalignment and disconnection, since misalignment and disconnection can be interpreted differently at different shooting angles and distances, any confusion between the two within one second between shots is completely classified as disconnection. This is because a higher degree of disconnection may have more serious consequences, preventing missed judgments (see [link]). Figure 11 The situation should result in a disconnect.

[0038] See Figure 12 To address the confusion between corrosion and scaling, since both exhibit uneven visual characteristics on the pipe wall, any mixture of corrosion and scaling is classified as corrosion within a 1-second window tolerance to avoid overlooking more severe structural defect corrosion (see [link to relevant documentation]). Figure 12 The situation suggests that the result should be corrosion.

[0039] join Figure 13 To address the confusion between detached parts and tree roots, since both appear visually as a hard material intruding into the conduit from a distance, but can be clearly distinguished at close range, the final result is based on the recognition result in the penultimate second within a 1-second window margin (see [link]). Figure 13 The situation suggests the result should be tree roots.

[0040] The specific content of step S4 includes: S41: Construct a "pipe wall-pipeline" scene recognition based on a PFN network, and mark the pipe wall scenes in the video sequence to form a pipe wall scene sequence window; S42: Establish pipe wall scene and pipeline material restriction strategies for different defect sequence windows, and connect the pipe wall scene sequence windows to remove, merge and / or not output different defect sequences.

[0041] Specifically, 300 images of the interior of pipes and 300 images of circumferential scenes were collected. On the validation set, a "pipe wall-pipe" scene recognition model was built. The model achieved an accuracy of approximately 0.9 and a recall of approximately 0.9. The training process was relatively rapid and stable. See [link to documentation]. Figure 14 In long video sequence detection, a small number of errors are acceptable because the loss of a few frames will not significantly affect the overall detection results.

[0042] Then, establish a sequence restriction strategy for pipe wall scenarios, see [link / reference]. Figure 15 As shown: Except for foreign object penetration, corrosion, scaling and branch pipe hidden connection, if other types of defect sequence windows are completely detected in a pipe wall imaging scene sequence window, they are considered non-existent and the corresponding window is removed; then, the sequence windows of the same defect are separated by pipe wall scene sequence windows. If the redundancy between windows is 1 second, and the interval between "defect sequence-pipe wall scene-defect sequence" is 1 second, then two unconnected defects are considered to belong to the same target object and should be merged to avoid the model from outputting results repeatedly.

[0043] Finally, a pipe material restriction strategy was established, including no longer outputting corrosion-related identification results for plastic pipes, such as PVC pipes; and no longer outputting deformation-related identification results for rigid pipes, such as concrete pipes. This application randomly selected 50 CCTV videos, totaling 158 defect instances. Based on steps S1-S4 above, the identification results in Table 3 were obtained. The results show that all 158 defects, except for one case of a hidden connection in a branch pipe, were successfully detected, with an average recall rate of 0.99. 42 defects were misidentified, with an average precision of 0.79. By utilizing pipe wall scene recognition and confusion identification, most discontinuous and isolated sequence windows were deleted or merged, minimizing over-detection. Simultaneously, multi-frame continuous identification of a single defect significantly improved the recall rate. Compared with the image-level detection results in Table 2, this application not only achieved complete recall of all defects but also significantly improved precision. Furthermore, by introducing a multi-threaded parallel processing strategy, this invention can complete the entire processing of 50 videos totaling 7 hours in just 3 hours, greatly saving time costs.

[0044] Table 3

[0045] For the specific content of step S5, a visualization and keyframe output method is constructed. The final video sequence, defect type and scene classification are used as visualization content to establish a visualization map. Based on the defect identification results and sequence conditions, keyframe output is established, and the output is a frame that can represent the basic form of the defect.

[0046] Specifically, a visualization is created using video sequences as the horizontal axis and defect type and scene as the vertical axis. This chart marks the video sequence windows for each defect, thus revealing the time periods in which defects appear in the video and the number of defects. See example. Figure 16 .

[0047] The keyframe output strategy then outputs keyframes. First, the corresponding defect must be identified in a frame of the final video sequence. However, some frames may be on the pipe wall where the defect is not visible. Next, from the middle frame of the corresponding defect sequence window, keyframes are searched. A keyframe is either a frame in the defect sequence window where multiple frames of the same defect are identified, or a frame where multiple frames of the same defect are not identified. The frame with the most identified defect types is selected. For example, when outputting keyframes for deviated defects, the deviated defect must first be identified in a frame of the final video sequence. Then, a judgment is made within the corresponding defect sequence window. If some frames in the defect sequence window identify two or more deviated defects, that frame is selected as the keyframe. If all frames in the defect sequence window identify one deviated defect or none, the frame with the most identified defect types is output as the keyframe.

Claims

1. A method for processing defects in drainage pipe videos based on deep learning and sequence interpretation, characterized in that: Includes the following steps: S1: CCTV video is preprocessed to obtain a new video sequence; S2: Construct a frame-by-frame sequence recognition model and use it to recognize new video sequences frame by frame, forming video sequences with defect identifiers; S3: Decipher video sequences with defect markers using video sequence interpretation methods; S4: The decoded video sequence is constrained by a video sequence constraint strategy to form the final video sequence; S5: Construct and output keyframes that represent the basic form of the corresponding defects.

2. The method for processing drainage pipe video defects based on deep learning and sequence interpretation according to claim 1, characterized in that: Step S1 includes the following steps: S11: Set the downsampling rate and combine it with the image blur detection method to select N representative frames from CCTV video. The set of N representative frames forms a preliminary video sequence. The attribute of the representative frames is the collection of CCTV video frame numbers and content between adjacent representative frames. S12: Establish a lightweight Pipe Focus Net model to mark and remove frames in the initial video sequence that show the external environment of the pipeline and the manhole to form a simplified video sequence; S13: The simplified video sequence is obtained by frame-by-frame denoising using the Non-Local Means algorithm.

3. The method for processing drainage pipe video defects based on deep learning and sequence interpretation according to claim 2, characterized in that: Step S2 includes the following steps: S21: Establish a defect database based on 16 defect types and form a training dataset with polygon annotations; S22: Construct a YOLO v12 object detection network, use a dataset to train and validate the model, and adjust the model structure and parameters to form a frame-by-frame recognition model; S23: The frame-by-frame recognition model identifies all representative frames in a new video sequence and records the defect type of each representative frame. All representative frames are classified according to their defect types and form multiple different defect sequence windows. These multiple different defect sequence windows form a video sequence with defect identifiers.

4. The method for processing drainage pipe video defects based on deep learning and sequence interpretation according to claim 3, characterized in that: Step S3, the video interpretation method, includes: Occasional false positives are removed using a multi-threshold cross-screening strategy; Based on the visual characteristics analysis of defects, the sequence windows of easily confused defects are sorted out. Easily confused defects include at least one of the following: hidden connection of branch pipe and foreign object penetration, deposit and obstacle, misalignment and disconnection, corrosion and scale, and falling off and tree root.

5. The method for processing drainage pipe video defects based on deep learning and sequence interpretation according to claim 4, characterized in that: Consolidation specifically includes In cases of confusion between concealed branch pipe connections and foreign object penetration, if the defect sequence windows of the two differ by no more than M seconds, they are determined to be the same defect, and the defect in the second-to-last second is taken as the final defect type. To address the confusion between deposits and obstacles, all instances within M seconds between windows are classified as deposits; To address the confusion between incorrect connections and disconnections, any window within M seconds of each other is classified as a disconnection. In cases of confusion between corrosion and scaling, any mixture of corrosion and scaling occurring within a window tolerance of M seconds is classified as corrosion. For cases where there is confusion between detachment and root, the defect type is determined based on the second-to-last second within an M-second tolerance.

6. The method for processing drainage pipe video defects based on deep learning and sequence interpretation according to claim 5, characterized in that: Step S4 includes: S41: Construct a "pipe wall-pipeline" scene recognition based on a PFN network, and mark the pipe wall scenes in the video sequence to form a pipe wall scene sequence window; S42: Establish pipe wall scene and pipeline material restriction strategies for different defect sequence windows, and connect the pipe wall scene sequence windows to remove, merge and / or not output different defect sequences.

7. The method for processing drainage pipe video defects based on deep learning and sequence interpretation according to claim 6, characterized in that: Step S5 specifically includes: S51: Establish a visualization map using the final video sequence, defect type, and scene classification as visualization content; S52: Based on the defect identification results and sequence status, output the corresponding frame of the basic defect morphology through the keyframe output strategy.

8. The method for processing drainage pipe video defects based on deep learning and sequence interpretation according to claim 7, characterized in that: The key frame output strategy includes: first, the corresponding defect must be identified in the frame of the final video sequence; then, from the middle frame of the corresponding defect sequence window, key frames are searched. The key frame is either a frame in the defect sequence window where multiple frames of the same type of defect are identified, or a frame in the defect sequence window where multiple frames of the same type of defect are not identified. In this case, the image frame with the most identified defect types is selected.

9. The method for processing drainage pipe video defects based on deep learning and sequence interpretation according to claim 2 or 8, characterized in that: The lightweight Pipe Focus Net model includes a backbone network and a detection head. The backbone network consists of three feature extraction blocks, each of which is composed of a convolutional layer, a rectified linear unit, and a max pooling layer. The detection head is designed with a two-stage output structure for binary classification.

Images