Pipeline video disease detection data processing method and system, terminal and storage medium

Abstract

The invention discloses a pipeline video disease detection data processing method and system, a terminal and a storage medium, and the method comprises the steps: obtaining a pipeline disease data training set, carrying out the training processing of a built pipeline disease instance segmentation test model according to the pipeline disease data training set, and obtaining a pipeline disease instance segmentation model; and obtaining pipe network video data of the urban drainage pipeline, preprocessing the pipe network video data to obtain to-be-detected pipe network video data, inputting the to-be-detected pipe network video to the pipeline disease instance segmentation model, and outputting a pipeline disease detection result. According to the method, disease detection is performed on the pipeline video data through multi-scale feature fusion, cascade optimization, dynamic resource allocation and time sequence information utilization, so that the precision and efficiency in pipeline disease detection are improved, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of disease detection, and particularly relates to a method, a system, a terminal and a computer-readable storage medium for processing pipeline video disease detection data. Background Art

[0002] The disease detection of urban drainage pipelines is one of the core tasks of facility operation and maintenance. Traditional detection mainly relies on manual visual inspection or expert experience analysis based on CCTV (Closed Circuit Television) videos, which has problems such as low efficiency, high cost and high missed detection rate. With the gradual application of deep learning-based visual detection technology in this field, especially the instance segmentation model represented by Mask R-CNN (Mask Region-based Convolutional Neural Network), because it can simultaneously complete target localization and pixel-level segmentation, it has become a research hotspot for the detection of diseases such as pipeline cracks, corrosion, and sediments.

[0003] However, the video data of drainage pipelines has particularities, such as large differences in the scales of disease targets (for example, coexistence of fine cracks and large-area corrosion), complex shapes (for example, irregular edges), and many background interferences (for example, water accumulation reflection, stain noise), resulting in the following limitations of existing models: 1) The feature extraction network has insufficient representation ability for multi-scale targets; 2) The coupling of classification and regression tasks limits the positioning accuracy; 3) The missed detection rate of small-target diseases is high, resulting in low reliability of detection results.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a method, a system, a terminal and a storage medium for processing pipeline video disease detection data, aiming to solve the problem of low detection efficiency and reliability of the existing technology for the disease detection of urban drainage pipelines.

[0006] To achieve the above purpose, the present invention provides a method for processing pipeline video disease detection data, and the method for processing pipeline video disease detection data includes the following steps:

[0007] Obtain a training set of pipeline disease data, and perform training processing on the created pipeline disease instance segmentation test model according to the training set of pipeline disease data to obtain a pipeline disease instance segmentation model;

[0008] Obtain the network video data of urban drainage pipelines, preprocess the network video data to obtain the network video data to be detected, and input the network video data to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result.

[0009] Optionally, for the above-described pipeline video disease detection data processing method, the step of obtaining a pipeline disease data training set and training a created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model specifically includes:

[0010] Obtain pipeline disease data of urban drainage pipelines, perform annotation processing on the pipeline disease data to obtain a pipeline disease data training set;

[0011] Create a pipeline disease instance segmentation test model, and input a set of image samples in the pipeline disease data training set into the pipeline disease instance segmentation test model;

[0012] Extract features from the image samples to obtain a feature map, correct candidate boxes of the feature map to obtain a target feature map, and perform disease detection on the target feature map to obtain a disease prediction result;

[0013] Calculate the loss of the target prediction box of the disease prediction result according to a loss function to obtain a target loss value, and correct the parameters of the pipeline disease instance segmentation test model according to the target loss value;

[0014] Input the next set of image samples into the pipeline disease instance segmentation test model until the training situation of the pipeline disease instance segmentation test model meets a preset condition to obtain a pipeline disease instance segmentation model.

[0015] Optionally, for the above-described pipeline video disease detection data processing method, the disease prediction result includes a disease prediction category, a disease prediction area, and a disease prediction mask area;

[0016] The step of extracting features from the image samples to obtain a feature map, correcting candidate boxes of the feature map to obtain a target feature map, and performing disease detection on the target feature map to obtain a disease prediction result specifically includes:

[0017] Extract features from the image samples to obtain multiple image features, and perform multi-scale feature fusion on all the image features to obtain a feature map;

[0018] Divide the detection area of the feature map to obtain corresponding areas to be detected, determine candidate boxes of the areas to be detected, and correct the candidate boxes to obtain a target feature map;

[0019] Perform disease detection on the target feature map according to a fully convolutional neural network to obtain a disease prediction mask area, and perform disease detection on the target feature map according to a fully connected layer to obtain a disease prediction category and a disease prediction area.

[0020] Optionally, in the above pipeline video disease detection data processing method, when performing feature extraction on the image sample to obtain a feature map, correcting the candidate boxes of the feature map to obtain a target feature map, and performing disease detection on the target feature map to obtain a disease prediction result, the method further includes:

[0021] Inputting the disease prediction result into the pipeline disease instance segmentation test model, performing cascade processing on the disease prediction result to obtain a processing result, and determining a corresponding intersection over union (IoU) threshold according to the processing result.

[0022] Optionally, in the above pipeline video disease detection data processing method, when calculating the loss of the target prediction box of the disease prediction result according to a loss function to obtain a target loss value, and correcting the parameters of the pipeline disease instance segmentation test model according to the target loss value, the method specifically includes:

[0023] Generating candidate boxes for the disease prediction region of the disease prediction result to obtain prediction boxes, and optimizing the prediction boxes according to the IoU threshold to obtain target prediction boxes;

[0024] Calculating the loss of the target prediction boxes respectively according to a classification loss function, a bounding box regression loss function, and a mask loss function to obtain a first loss value, a second loss value, and a third loss value;

[0025] Summing up the first loss value, the second loss value, and the third loss value to obtain a target loss value, and correcting the parameters of the pipeline disease instance segmentation test model according to the target loss value.

[0026] Optionally, in the above pipeline video disease detection data processing method, the expression of the classification loss function is:

[0027] FL(p t ) = -α t (1 - p t ) γ log(p t );

[0028] The expression of the bounding box regression loss function is:

[0029]

[0030] The expression of the mask loss function is:

[0031]

[0032] Where FL(p t ) is the classification loss function, and α tis the true category, p t is the predicted category, γ is a constant factor, L smoothL1 (x) is the bounding box regression loss function, x is the boundary size of the target prediction box, L mask is the mask loss function, y i is the true mask value of the i-th pixel point, is the predicted mask value of the i-th pixel point, n is the number of pixel points.

[0033] Optionally, for the pipeline video disease detection data processing method, wherein, obtaining the network video data of the urban drainage pipeline, preprocessing the network video data to obtain the network video data to be detected, and inputting the network video data to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result, specifically including:

[0034] Obtaining the network video data of the urban drainage pipeline, performing frame extraction on the network video data to obtain a plurality of video key frames, and performing fusion processing on all the video key frames to obtain the network video data to be detected;

[0035] Inputting the network video data to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result.

[0036] Optionally, for the pipeline video disease detection data processing method, wherein, the pipeline video disease detection data processing system includes:

[0037] A model training module, configured to obtain a pipeline disease data training set, and perform training processing on a created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model;

[0038] A disease detection module, configured to obtain the network video data of the urban drainage pipeline, preprocess the network video data to obtain the network video data to be detected, and input the network video data to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result.

[0039] In addition, to achieve the above object, the present invention further provides a terminal, wherein, the terminal includes: a memory, a processor, and a pipeline video disease detection data processing program stored on the memory and executable on the processor, and when the pipeline video disease detection data processing program is executed by the processor, the steps of the pipeline video disease detection data processing method as described above are implemented.

[0040] In addition, to achieve the above object, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a pipeline video disease detection data processing program, and when the pipeline video disease detection data processing program is executed by a processor, the steps of the pipeline video disease detection data processing method as described above are implemented.

[0041] In the present invention, a pipeline disease data training set is obtained, and the created pipeline disease instance segmentation test model is trained according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model; the network video data of the urban drainage pipeline is obtained, the network video data is preprocessed to obtain the network video data to be detected, and the network video data to be detected is input into the pipeline disease instance segmentation model to output the pipeline disease detection result. The present invention performs disease detection on pipeline video data through multi-scale feature fusion, cascade optimization, dynamic resource allocation, and utilization of temporal information, which not only improves the accuracy and efficiency in pipeline disease detection, but also reduces the cost. Brief Description of the Drawings

[0042] Figure 1 is a flowchart of a preferred embodiment of the pipeline video disease detection data processing method of the present invention;

[0043] Figure 2 is a schematic diagram of the overall process of the pipeline video disease detection data processing method of the present invention;

[0044] Figure 3 is a schematic diagram of the process of the improved Mask-RCNN structure in the present invention;

[0045] Figure 4 is a schematic diagram of the process of the Cascade cascade structure in the present invention;

[0046] Figure 5 is a structural diagram of a preferred embodiment of the pipeline video disease detection data processing system of the present invention;

[0047] Figure 6 is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. Detailed Description of the Embodiments

[0048] To make the object, technical solution and advantages of the present invention clearer and more definite, the present invention will be further described in detail below with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] It should be noted that if there are directional indications (such as up, down, left, right, front, back...) involved in the embodiments of the present invention, the directional indications are only used to explain the relative positional relationship, movement conditions, etc. between components in a certain specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly.

[0050] In addition, if there are descriptions such as "first", "second", etc. involved in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those skilled in the art can implement them. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0051] The pipeline video disease detection data processing method described in the preferred embodiment of the present invention, as Figure 1 shown, the pipeline video disease detection data processing method includes the following steps:

[0052] Step S10: Obtain a pipeline disease data training set, and perform training processing on the created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model.

[0053] The step S10 includes:

[0054] Step S11: Obtain pipeline disease data of urban drainage pipelines, and perform annotation processing on the pipeline disease data to obtain a pipeline disease data training set;

[0055] Step S12: Create a pipeline disease instance segmentation test model, and input a set of image samples in the pipeline disease data training set into the pipeline disease instance segmentation test model;

[0056] Step S13: Extract features from the image samples to obtain a feature map, correct the candidate boxes of the feature map to obtain a target feature map, and perform disease detection on the target feature map to obtain a disease prediction result;

[0057] Step S14: Calculate the loss of the target prediction box of the disease prediction result according to the loss function to obtain a target loss value, and correct the parameters of the pipeline disease instance segmentation test model according to the target loss value;

[0058] Step S15: Input the next set of image samples into the pipeline disease instance segmentation test model until the training condition of the pipeline disease instance segmentation test model meets the preset condition, and obtain the pipeline disease instance segmentation model.

[0059] Specifically, in the embodiment of the present invention, in order to solve the problem of low detection efficiency and reliability of the existing technology for detecting pipeline diseases in urban drainage pipelines, a pipeline video disease detection data processing method is proposed. By introducing an optimized feature extraction network, a Cascade cascade structure strategy, and video frame-interval information fusion, the accuracy and efficiency of pipeline disease detection are significantly improved. The corresponding processing flow is as Figure 2 shown. Specifically, first, a pipeline disease data training set is constructed. By obtaining pipeline disease data of urban drainage pipelines and using the data annotation software Labelme to perform annotation processing on the pipeline disease data, a pipeline disease data training set is obtained. The corresponding label values include three categories, namely Bbox (bounding box), Class (category), and Mask (mask). Then, a pipeline disease instance segmentation test model is created. The core architecture of the pipeline disease instance segmentation test model includes ResNeSt (Residual Network with Split-Attention, an improved residual network): introducing the Split-Attention mechanism on the basis of ResNet to enhance feature diversity and improve the perception ability of complex pipeline textures (such as corrosion and cracks); FPN (Feature Pyramid Network): fusing multi-scale features to solve the problem of variable target sizes caused by differences in shooting distance and angle of pipeline diseases (such as the coexistence scenario of tiny cracks and large-area damage), and the Cascade cascade structure: introducing a Cascade cascade detection head on the basis of Mask R-CNN (Mask Region-based Convolutional Neural Network). Through multi-stage progressive bounding box regression and classification, the detection results are gradually refined to solve the problem of insufficient positioning accuracy caused by the coupling of classification and regression tasks. The mask prediction branch is deeply combined with the Cascade structure, and the cascade features are used to enhance the mask generation ability and improve the segmentation accuracy of complex edges (such as crack extension and corrosion area).

[0060] After that, a set of image samples in the pipeline disease data training set is input into the pipeline disease instance segmentation test model. The corresponding processing process is as Figure 3As shown in the figure, the ResNeSt+FPN network structure is adopted for full-image feature extraction. FPN is used to fuse the high-resolution information of low-level features and the semantic information of high-level features. FPN mainly solves the multi-scale problem in object detection. By simply changing the network connection, without significantly increasing the computational complexity of the original model, the performance of small object detection is greatly improved. Through upsampling of high-level features and top-down connection of low-level features, and prediction is performed on each layer. Specifically, feature extraction is performed on the image sample to obtain multiple image features, and all the image features are subjected to multi-scale feature fusion to obtain a feature map. Then, the obtained feature map is fed into the RPN (Region Proposal Network) to generate the region of interest (ROI) to be detected, and the bounding box of the ROI is corrected for the first time. Specifically, the feature map is divided into detection regions to obtain the corresponding regions to be detected, the candidate boxes of the regions to be detected are determined, the candidate boxes are corrected to obtain the target feature map, and bilinear interpolation is used to more accurately find the features corresponding to each candidate box. Finally, the obtained target feature map is fed into the fully convolutional neural network and the fully connected layer respectively. Specifically, disease detection is performed on the target feature map according to the fully convolutional neural network to obtain the disease prediction mask region, and disease detection is performed on the target feature map according to the fully connected layer to obtain the disease prediction category and the disease prediction region.

[0061] Furthermore, in the embodiment of the present invention, the single Mask-RCNN belongs to the Two-Stage object segmentation strategy. In order to improve the detection accuracy and solve the problem of selecting the IoU (Intersection over Union) threshold in the training stage, when using the Two-stage object detection method, there is a problem of how to select the appropriate IoU threshold. Therefore, the selection of the IoU threshold is a set of hyperparameters that need to be carefully selected. On the one hand, the higher the selected IoU threshold, the closer the obtained positive samples are to the target, so the detector trained is more accurate in positioning. However, blindly increasing the IoU threshold will cause two problems: one is the overfitting problem caused by too few positive samples, and the other is the decline in evaluation performance caused by using different thresholds for training and testing. On the other hand, the lower the selected IoU threshold, the richer the obtained positive samples, which is beneficial to the training of the detector, but will lead to a large number of false detections during testing. Therefore, the Cascade cascade structure is adopted, and the regression result of the previous stage is used as the input of the next stage, and the IoU threshold is gradually increased. Its structure is as Figure 4 shown. The disease prediction result is input into the pipeline disease instance segmentation test model, and the disease prediction result is cascaded to obtain a processing result. Among them, the cascading process is to use the regression result output after processing as the input of the next stage, and determine the corresponding intersection over union threshold according to the processing result.

[0062] After that, the detection head of each stage gradually optimizes the prediction boxes. In the initial stage, coarse-grained prediction boxes are generated, and in the latter two stages, they are successively screened through IoU thresholds (for example, 0.5 → 0.6 → 0.7) and the coordinates are refined. Specifically, candidate boxes are generated for the disease prediction regions of the disease prediction results to obtain prediction boxes, and the prediction boxes are optimized according to the intersection over union threshold to obtain target prediction boxes; the first loss value, the second loss value, and the third loss value are obtained by calculating the losses of the target prediction boxes according to the classification loss function, the bounding box regression loss function, and the mask loss function respectively; among them, the expression of the classification loss function is: FL(p t ) = -α t (1 - p t ) γ log(p t ); the expression of the bounding box regression loss function is:

[0063] The expression of the mask loss function is: Among them, FL(p t ) is the classification loss function, α t is the true category, p t is the predicted category, γ is a constant factor, L smoothL1 (x) is the bounding box regression loss function, x is the boundary size of the target prediction box, L mask is the mask loss function, y i is the true mask value of the i-th pixel point, is the predicted mask value of the i-th pixel point, and n is the number of pixel points; subsequently, the first loss value, the second loss value, and the third loss value are summed to obtain the target loss value, and the parameters of the pipeline disease instance segmentation test model are corrected according to the target loss value.

[0064] After that, the next group of image samples is input into the pipeline disease instance segmentation test model, and the above process is repeated, which will not be elaborated here; until the training situation of the pipeline disease instance segmentation test model meets the requirements, a pipeline disease instance segmentation model with optimized weight parameters is obtained.

[0065] Furthermore, in order to enable a dynamic RoI allocation strategy, through region saliency analysis, the region weights are calculated based on the feature map gradient magnitude and texture complexity, and the sampling density is increased to 2 times in the high corrosion areas (for example, joints). Adaptive grid partitioning is also used to partition the feature map by K-means clustering and dynamically adjust the RoI grid density to avoid resource waste caused by uniform sampling.

[0066] Step S20: Obtain the network video data of urban drainage pipelines, preprocess the network video data to obtain the pipeline network video data to be detected, and input the pipeline network video to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result.

[0067] The said step S20 includes:

[0068] Step S21: Obtain the network video data of urban drainage pipelines, perform frame extraction on the network video data to obtain multiple video key frames, and perform fusion processing on all the video key frames to obtain the pipeline network video data to be detected;

[0069] Step S22: Input the pipeline network video to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result.

[0070] Specifically, in the embodiment of the present invention, after obtaining the pipeline disease instance segmentation model, it is necessary to enter the actual application stage. Specifically, obtain the network video data of urban drainage pipelines, perform frame extraction on the network video data to obtain multiple video key frames. Among them, for the screening of video key frames, calculate the motion amplitude between video frames by the optical flow method. If the difference between consecutive frames is less than the preset ratio (for example, 5%), skip the detection and only perform full-scale processing on the video key frames; perform fusion processing on all the video key frames to obtain the pipeline network video data to be detected. By performing frame extraction on the pipeline network video data, the detection speed can be improved. Generally, the video frame rate is 25 - 30HZ, so sampling detection is performed at 3 - 5HZ to improve the processing efficiency; then, input the pipeline network video to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result, realizing pixel-level disease recognition.

[0071] Furthermore, as Figure 5 shown, based on the above pipeline video disease detection data processing method, the present invention also correspondingly provides a pipeline video disease detection data processing system. Among them, the pipeline video disease detection data processing system includes:

[0072] A model training module 51, configured to obtain a pipeline disease data training set, and perform training processing on the created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model;

[0073] A disease detection module 52, configured to obtain the network video data of urban drainage pipelines, preprocess the network video data to obtain the pipeline network video data to be detected, and input the pipeline network video to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result.

[0074] Furthermore, as Figure 6As shown, based on the above pipeline video disease detection data processing method, the present invention also correspondingly provides a terminal, which includes a processor 10, a memory 20, and a display 30. Figure 6 Only some components of the terminal are shown. However, it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0075] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as the hard disk or memory of the terminal. In some other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk equipped on the terminal, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 20 may also include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software installed on the terminal and various types of data, such as the program code for installing the terminal. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, a pipeline video disease detection data processing program 40 is stored on the memory 20, and this pipeline video disease detection data processing program 40 can be executed by the processor 10, thereby implementing the pipeline video disease detection data processing method in this application.

[0076] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chips, and is used to run the program code stored in the memory 20 or process data, such as executing the pipeline video disease detection data processing method, etc.

[0077] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other through a system bus.

[0078] In one embodiment, when the processor 10 executes the pipeline video disease detection data processing program 40 in the memory 20, the following steps are implemented:

[0079] Obtain a pipeline disease data training set, and perform training processing on the created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model;

[0080] Obtain the network video data of the urban drainage pipeline, preprocess the network video data to obtain the pipeline video data to be detected, and input the pipeline video to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection result.

[0081] Among them, to obtain the pipeline disease data training set, and perform training processing on the created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain the pipeline disease instance segmentation model, specifically including:

[0082] Obtain the pipeline disease data of the urban drainage pipeline, and perform annotation processing on the pipeline disease data to obtain the pipeline disease data training set;

[0083] Create a pipeline disease instance segmentation test model, and input a set of image samples in the pipeline disease data training set into the pipeline disease instance segmentation test model;

[0084] Extract features from the image samples to obtain a feature map, correct the candidate boxes of the feature map to obtain a target feature map, and perform disease detection on the target feature map to obtain a disease prediction result;

[0085] Calculate the loss of the target prediction box of the disease prediction result according to the loss function to obtain a target loss value, and correct the parameters of the pipeline disease instance segmentation test model according to the target loss value;

[0086] Input the next set of image samples into the pipeline disease instance segmentation test model until the training situation of the pipeline disease instance segmentation test model meets the preset conditions to obtain the pipeline disease instance segmentation model.

[0087] Among them, the disease prediction result includes a disease prediction category, a disease prediction area, and a disease prediction mask area;

[0088] The extracting features from the image samples to obtain a feature map, correcting the candidate boxes of the feature map to obtain a target feature map, and performing disease detection on the target feature map to obtain a disease prediction result specifically includes:

[0089] Extract features from the image samples to obtain multiple image features, and perform multi-scale feature fusion on all the image features to obtain a feature map;

[0090] Divide the detection area of the feature map to obtain the corresponding area to be detected, determine the candidate boxes of the area to be detected, and correct the candidate boxes to obtain a target feature map;

[0091] Perform disease detection on the target feature map according to the fully convolutional neural network to obtain a disease prediction mask region, and perform disease detection on the target feature map according to the fully connected layer to obtain a disease prediction category and a disease prediction region.

[0092] Among them, after performing feature extraction on the image sample to obtain a feature map, performing correction processing on the candidate boxes of the feature map to obtain a target feature map, and performing disease detection on the target feature map to obtain a disease prediction result, it further includes:

[0093] Input the disease prediction result into the pipeline disease instance segmentation test model, perform cascade processing on the disease prediction result to obtain a processing result, and determine a corresponding intersection over union (IoU) threshold according to the processing result.

[0094] Among them, calculating the loss of the target prediction box of the disease prediction result according to the loss function to obtain a target loss value, and correcting the parameters of the pipeline disease instance segmentation test model according to the target loss value specifically includes:

[0095] Generate candidate boxes for the disease prediction region of the disease prediction result to obtain prediction boxes, and perform optimization processing on the prediction boxes according to the IoU threshold to obtain target prediction boxes;

[0096] Calculate the loss of the target prediction box according to the classification loss function, the bounding box regression loss function, and the mask loss function respectively to obtain a first loss value, a second loss value, and a third loss value;

[0097] Sum the first loss value, the second loss value, and the third loss value to obtain a target loss value, and correct the parameters of the pipeline disease instance segmentation test model according to the target loss value.

[0098] Among them, the expression of the classification loss function is:

[0099] FL(p t )=-α t (1-p t ) γ log(p t );

[0100] The expression of the bounding box regression loss function is:

[0101]

[0102] The expression of the mask loss function is:

[0103]

[0104] Among them, FL(pt ) is the classification loss function, α t is the true class, p t is the predicted class, γ is a constant factor, and L smoothL1 (x) is the bounding box regression loss function, x is the boundary size of the target prediction box, and L mask is the mask loss function, y i is the true mask value of the i-th pixel, is the predicted mask value of the i-th pixel, and n is the number of pixels.

[0105] Among them, obtaining the network video data of the urban drainage pipeline, preprocessing the network video data to obtain the to-be-detected network video data, and inputting the to-be-detected network video into the pipeline disease instance segmentation model to output the pipeline disease detection result specifically includes:

[0106] Obtaining the network video data of the urban drainage pipeline, performing frame extraction on the network video data to obtain a plurality of video key frames, and performing fusion processing on all the video key frames to obtain the to-be-detected network video data;

[0107] Inputting the to-be-detected network video into the pipeline disease instance segmentation model to output the pipeline disease detection result.

[0108] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a pipeline video disease detection data processing program, and when the pipeline video disease detection data processing program is executed by a processor, the steps of the pipeline video disease detection data processing method described above are implemented.

[0109] In summary, the present invention provides a pipeline video disease detection data processing method, system, terminal, and storage medium. The method includes: obtaining a pipeline disease data training set, training a created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model; obtaining the network video data of the urban drainage pipeline, preprocessing the network video data to obtain the to-be-detected network video data, and inputting the to-be-detected network video into the pipeline disease instance segmentation model to output the pipeline disease detection result. The present invention performs disease detection on pipeline video data through multi-scale feature fusion, cascade optimization, dynamic resource allocation, and utilization of temporal information, not only improving the accuracy and efficiency in pipeline disease detection, but also reducing costs.

[0110] It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or terminal. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or terminal including such element.

[0111] Of course, those of ordinary skill in the art can understand that all or part of the processes of implementing the above-described embodiments of the method can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium readable by a computer. When the program is executed, it can include the processes of the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0112] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description. All such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for processing pipeline video disease detection data, characterized in that, The pipeline video disease detection data processing method includes: Obtain a pipeline disease data training set, and perform training processing on the created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model; Obtain the network video data of urban drainage pipelines, preprocess the network video data to obtain the to-be-detected network video data, and input the to-be-detected network video into the pipeline disease instance segmentation model to output the pipeline disease detection result.

2. The method for processing pipeline video disease detection data according to claim 1, wherein The obtaining of the pipeline disease data training set, and performing training processing on the created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain the pipeline disease instance segmentation model specifically includes: Obtain the pipeline disease data of urban drainage pipelines, and perform annotation processing on the pipeline disease data to obtain a pipeline disease data training set; Create a pipeline disease instance segmentation test model, and input a set of image samples in the pipeline disease data training set into the pipeline disease instance segmentation test model; Perform feature extraction on the image samples to obtain a feature map, correct the candidate boxes of the feature map to obtain a target feature map, and perform disease detection on the target feature map to obtain a disease prediction result; Calculate the loss of the target prediction box of the disease prediction result according to the loss function to obtain a target loss value, and correct the parameters of the pipeline disease instance segmentation test model according to the target loss value; Input the next set of image samples into the pipeline disease instance segmentation test model until the training situation of the pipeline disease instance segmentation test model meets the preset conditions to obtain a pipeline disease instance segmentation model.

3. The method for processing pipeline video disease detection data according to claim 2, wherein, The disease prediction result includes a disease prediction category, a disease prediction region, and a disease prediction mask region; The performing of feature extraction on the image samples to obtain a feature map, correcting the candidate boxes of the feature map to obtain a target feature map, and performing disease detection on the target feature map to obtain a disease prediction result specifically includes: Perform feature extraction on the image samples to obtain multiple image features, and perform multi-scale feature fusion on all the image features to obtain a feature map; Perform detection area division on the feature map to obtain corresponding to-be-detected areas, determine the candidate boxes of the to-be-detected areas, and correct the candidate boxes to obtain a target feature map; Perform disease detection on the target feature map according to the fully convolutional neural network to obtain a disease prediction mask region, and perform disease detection on the target feature map according to the fully connected layer to obtain a disease prediction category and a disease prediction region.

4. The method for processing pipeline video disease detection data according to claim 2, wherein, After the performing of feature extraction on the image samples to obtain a feature map, correcting the candidate boxes of the feature map to obtain a target feature map, and performing disease detection on the target feature map to obtain a disease prediction result, it further includes: Input the disease prediction result into the pipeline disease instance segmentation test model, perform cascade processing on the disease prediction result to obtain a processing result, and determine the corresponding intersection over union threshold according to the processing result.

5. The method for processing pipeline video disease detection data according to claim 4, wherein Performing loss calculation on the target prediction bounding boxes of the disease prediction results according to the loss function to obtain a target loss value, and correcting the parameters of the pipeline disease instance segmentation test model according to the target loss value, specifically including: Generating candidate bounding boxes for the disease prediction regions of the disease prediction results to obtain prediction bounding boxes, and performing optimization processing on the prediction bounding boxes according to the intersection over union (IoU) threshold to obtain target prediction bounding boxes; Performing loss calculation on the target prediction bounding boxes according to the classification loss function, the bounding box regression loss function, and the mask loss function respectively to obtain a first loss value, a second loss value, and a third loss value; Summing up the first loss value, the second loss value, and the third loss value to obtain a target loss value, and correcting the parameters of the pipeline disease instance segmentation test model according to the target loss value.

6. The method for processing pipeline video disease detection data according to claim 5, characterized in that The expression of the classification loss function is: FL(p t ) = -α t (1 - p t ) γ log(p t ) The expression of the bounding box regression loss function is: The expression of the mask loss function is: Among them, FL(p t ) is the classification loss function, α t is the true category, p t is the predicted category, γ is a constant factor, L smoothL1 (x) is the bounding box regression loss function, x is the boundary size of the target prediction box, L mask is the mask loss function, y i is the true mask value of the i-th pixel point, is the predicted mask value of the i-th pixel point, and n is the number of pixel points.

7. The method for processing pipeline video disease detection data according to claim 1, wherein, Obtaining the network video data of urban drainage pipelines, preprocessing the network video data to obtain the network video data to be detected, and inputting the network video data to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection results, specifically including: Obtaining the network video data of urban drainage pipelines, performing frame extraction on the network video data to obtain a plurality of video key frames, and performing fusion processing on all the video key frames to obtain the network video data to be detected; Inputting the network video data to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection results.

8. A pipeline video disease detection data processing system, characterized in that The pipeline video disease detection data processing system includes: A model training module, configured to obtain a pipeline disease data training set, and perform training processing on a created pipeline disease instance segmentation test model according to the pipeline disease data training set to obtain a pipeline disease instance segmentation model; A disease detection module, configured to obtain the network video data of urban drainage pipelines, preprocess the network video data to obtain the network video data to be detected, and input the network video data to be detected into the pipeline disease instance segmentation model to output the pipeline disease detection results.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a pipeline video disease detection data processing program stored on the memory and executable on the processor. When the pipeline video disease detection data processing program is executed by the processor, the steps of the pipeline video disease detection data processing method according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a pipeline video disease detection data processing program. When the pipeline video disease detection data processing program is executed by a processor, the steps of the pipeline video disease detection data processing method according to any one of claims 1-7 are implemented.

Images