Method for detecting defects of conveying pipeline

Abstract

The invention relates to a method for detecting defects of a conveying pipeline. The method comprises the following steps: firstly, extracting a video image of the pipeline and marking fracture, dislocation, corrosion and pipe tumor defect images; a model compression C3Ghost module is adopted to replace an ELAN module in a target detection algorithm YOLOv7 model so as to optimize the architecture design of the YOLOv7 model; the YOLOv7 model is trained and then analyzed layer by layer, and sorting is carried out according to floating point operand related to each layer from high to low so as to determine an ideal pruning ratio; carrying out pruning operation on a hierarchy which has the greatest contribution to reduction of floating point operand under an ideal pruning ratio and is relatively small in performance reduction degree of the YOLOv7 model, and optimizing the YOLOv7 model aiming at the defects of the conveying pipeline; and finally, analyzing the pipeline defect identification image in real time by using a YOLOv7 model and detecting the defect condition of the pipeline. According to the method, pipeline defects can be detected in real time by using the optimized target detection algorithm YOLOv7 model on small mobile equipment, and the YOLOv7 model is low in complexity, low in requirement on computing resources and high in robustness.

Description

Technical field:

[0001] The invention relates to a method for detecting pipeline defects, and in particular to a method for detecting conveying pipeline defects. Background technology:

[0002] For pipeline defect inspection, closed-circuit television (CCTV) systems are usually used to provide video data, and pipeline inspectors view the video data to identify pipeline defects and compile pipeline status reports. This inspection method lacks effective image analysis methods and still relies on manual processing of video data. Especially for long-distance transmission pipelines such as inter-city pipelines, the large amount of data processed significantly slows down the inspection speed of the pipeline, and the inspection results are easily affected by the subjective judgment of the inspector. Long-term work can easily cause visual fatigue to the staff, increasing the risk of misdetection and omission. Therefore, efficient, accurate and automated inspection methods are of great significance to improving the efficiency and accuracy of pipeline inspection.

[0003] In recent years, with the advancement of science and technology, deep learning technology has gradually been applied in pipeline defect detection. However, these applications require a lot of computing resources to support them during execution, usually involving a wide range of data processing, numerical analysis, image analysis and model training activities, such as convolution operations and recursive algorithms, which have higher requirements on the performance of CPU or GPU to ensure running speed and efficiency. Most of these applications are computationally intensive, that is, those deep learning, especially large convolutional neural networks (CNN), are regarded as typical computationally intensive applications due to their numerous internal parameters and extremely high demand for computing resources. Although existing technologies such as YOLOv3 and YOLOv5 have excellent performance in speed, accuracy and reliability, their high computing cost makes it difficult to directly apply such technologies to small embedded systems, which greatly limits their deployment and use on small mobile devices. How to achieve lightweight models while maintaining efficient detection has become an urgent problem to be solved. Summary of the invention:

[0004] The technical problem to be solved by the present invention is: to provide a method for detecting defects in a transmission pipeline, which can use an optimized target detection algorithm YOLOv7 model on a small mobile device to analyze the pipeline defect recognition image acquired in real time, and detect defects such as pipeline rupture, misalignment, corrosion and tube tumors. Compared with the prior art, the target detection algorithm YOLOv7 model has low complexity, and the demand for computing resources is reduced to a very low level. The overall efficiency is high, the system stability is good, and it has strong robustness.

[0005] The technical solution of the present invention:

[0006] A method for detecting defects in a conveying pipeline comprises the following steps:

[0007] Step 1: Use a visual sensor to capture videos of different long-distance pipelines. The visual sensor is placed inside the pipeline. The visual sensor can be a closed-circuit television (CCTV), a periscope, or a pipeline inspection robot equipped with an industrial camera.

[0008] Step 2: Perform image extraction processing on the video to obtain a set of related images of multiple pipelines;

[0009] Step 3: Filter out images that meet the defect criteria from the pipeline image collection; personnel who have received basic training mark the images that meet the defect criteria according to the defect type, marking rupture (PL), dislocation (CK), corrosion (FS) and tube tumor (GL) respectively, and finally obtain the pipeline defect identification image database (USDID);

[0010] Given the complexity and diversity of the internal environment of the pipeline, not all images obtained are suitable for network training and testing. Samples with obvious defect targets and high image quality should be selected to build a long-distance pipeline defect image database. The selection criteria mainly include: ensuring that the defect features can be easily identified by the naked eye; in order to enhance the system's ability to cope with different conditions, it is necessary to ensure that the selected images cover the defects that occur under a variety of lighting conditions; the defect types must be clear and accurate to avoid ambiguity.

[0011] Building a high-performance deep learning model depends on a dataset with a large number of high-quality samples. The quantity and quality of samples in the dataset directly affect the stability and detection accuracy of the model.

[0012] Step 4. On this basis, the architecture design of the target detection algorithm YOLOv7 model is further optimized: considering the actual needs of pipeline defect detection, the model compression C3Ghost module is used to replace the efficient aggregation network ELAN (Efficient Layer Aggregation Networks) module with more parameters in the target detection algorithm YOLOv7 model. The model compression C3Ghost module is a custom neural network module; this not only effectively reduces the number of parameters and computational complexity of the model, but also replaces the traditional convolutional layers of other parts with a simpler form to achieve lightweight transformation of the overall network structure;

[0013] In response to the needs of small embedded devices, the lightweight network GhostNet architecture is integrated into the target detection algorithm YOLOv7 model to reduce the complexity of the model. Compared with the traditional convolutional neural network (CNN), the lightweight network GhostNet can significantly reduce the number of parameters and computing requirements while keeping the output feature map unchanged. The computing resources required are only about 1 / s of the original requirements; "s" is a variable representing a scale or reduction factor. Here, "1 / s" means that the computing resource requirements are reduced to a very low level;

[0014] The process architecture of the model compression C3Ghost module contains a series of components connected in sequence, specifically: first, there are two parallel convolutional layers Conv_1, and the processing result of one convolutional layer Conv_1 is imported into a lightweight network component GhostBottleNeck, which contains convolutional layers Conv_2 and Conv_3; next, the information flow processed by another convolutional layer Conv_1 and the lightweight network component GhostBottleNeck is integrated through the concatenation layer Concat_1, and then the features are fused through the concatenation layer Concat_2 and an addition operation Add; finally, all these features pass through the concatenation layer Concat_3 and the final convolutional layer Conv_4 to complete the entire process;

[0015] Step 5: Use the pipeline defect recognition image database to train the target detection algorithm YOLOv7 model;

[0016] Step 6: Analyze the target detection algorithm YOLOv7 model layer by layer and calculate the floating-point operations involved in each layer; this can usually be achieved through tools or functions provided by deep learning frameworks (such as PyTorch, TensorFlow, etc.);

[0017] Step 7: Sort each layer from high to low according to the floating-point operations involved, and determine the ideal pruning ratio of each layer; the ideal pruning ratio is to reduce the size and amount of calculation of the model as much as possible while ensuring that the model performance (such as accuracy) is not significantly affected; compare the floating-point operations of each layer after sorting, and find out those layers with large floating-point operations and relatively small impact on model performance; these layers usually have greater optimization potential, because pruning these layers can significantly reduce the amount of calculation while having little impact on model performance;

[0018] Step 8, evaluate the impact of each layer on the floating-point operation performance under the ideal pruning ratio, so as to determine the n layers with the greatest optimization potential (the contribution to reducing the floating-point operation amount under the ideal pruning ratio is the greatest, and the performance degradation of the target detection algorithm YOLOv7 model is the smallest), so as to optimize the target detection algorithm YOLOv7 model for pipeline defects again; the method for determining n is: according to the goal of reducing the floating-point operation amount, a reasonable performance degradation threshold is set, starting from the layer with the highest floating-point operation amount after sorting, and pruning is performed layer by layer under the ideal pruning ratio. After each step of pruning, the target detection algorithm YOLOv7 model is tested with the verification data set to evaluate the change in the performance of the target detection algorithm YOLOv7 model until the set performance degradation threshold is reached or the floating-point operation amount can no longer be reduced. At this time, the number of pruned layers is the value of n, and the target detection algorithm YOLOv7 model is also optimized;

[0019] When determining the value of n, it is also necessary to comprehensively consider the requirements of the actual application scenario on model performance, inference speed, resource consumption, etc.;

[0020] The pruning operation can significantly reduce the number of model parameters, volume, and computing requirements, effectively solving common problems in current image processing algorithms, such as large model size, high computing cost, and poor cross-platform adaptability, and also improves the system's stability and detection accuracy.

[0021] Step 9: Use the target detection algorithm YOLOv7 model optimized in step 8 to analyze the pipeline defect recognition image acquired in real time to detect the defect condition of the pipeline.

[0022] In step 1, the video of the conveying pipeline is a video of the interior of the conveying pipeline.

[0023] In view of the small difference in image information between consecutive frames in the video, in step 2, the video is subjected to image extraction processing by extracting once every M frames, thereby obtaining a pipeline image set, where M is a natural number greater than or equal to 3.

[0024] In step 3, Lambelimg software is used to mark images that meet the defect criteria.

[0025] In step 5, according to the performance of the target detection algorithm YOLOv7 model on the test data set in the pipeline defect recognition image database, the training parameters such as learning rate and batch size are adjusted to optimize the training process.

[0026] In step 8, the pruning operation is performed using the Performance-Aware Global Channel Pruning (PAGCP) pruning algorithm.

[0027] The PAGCP pruning algorithm is a framework for global channel pruning of multi-task CNN models. This framework transforms the model compression problem into an optimization problem of joint channel saliency indicators, comprehensively considers the joint impact of inter-layer and intra-layer channels on the compression effect of multi-task models, and further develops into a sequential greedy pruning method based on performance-aware standards. In this way, without introducing regularization penalties, it can effectively identify and eliminate global redundant channels in multi-task models, thereby achieving an efficient performance optimization strategy.

[0028] Beneficial effects of the present invention:

[0029] 1. The present invention optimizes the architecture design of the target detection algorithm YOLOv7 model, and adopts the model compression C3Ghost module to replace the efficient aggregation network ELAN module with more parameters in the target detection algorithm YOLOv7 model, thereby achieving the lightweight of the overall network structure. While keeping the output feature map unchanged, the complexity of the model is reduced, the number of parameters and floating-point operations of the model are effectively reduced, and the demand for computing resources is reduced to a very low level.

[0030] 2. The present invention prunes n levels with the greatest optimization potential (the greatest contribution to reducing the amount of floating-point operations under an ideal pruning ratio, and the performance degradation of the target detection algorithm YOLOv7 model is relatively small), which can significantly reduce the number of parameters, volume and computing requirements of the model, effectively solving the problems of large volume, high computational complexity and poor cross-platform adaptability existing in current image processing algorithms, improving the stability and detection accuracy of the system, and is particularly suitable for small mobile devices.

[0031] 3. The present invention uses the performance-aware global channel pruning PAGCP pruning algorithm to implement pruning operations. Without introducing regularization penalty terms, it can effectively identify and eliminate global redundant channels in the multi-task model, and implement an efficient performance optimization strategy. It not only significantly reduces the computing cost, but also improves the overall efficiency of the model, and improves the robustness and detection accuracy of the model.

[0032] 4. The method for detecting pipeline defects of the present invention can be used on a small mobile device, and a single person can complete the detection of pipeline defects, which not only effectively reduces the manpower demand, but also greatly improves the detection work efficiency. Description of the drawings:

[0033] Figure 1 This is a schematic diagram of the principle of pipeline defect detection;

[0034] Figure 2 Corrosion images of pipeline defects;

[0035] Figure 3 It is the rupture image of the pipeline defect;

[0036] Figure 4 This is a schematic diagram of the architecture of the traditional target detection algorithm YOLOv7 model;

[0037] Figure 5 This is a schematic diagram of the architecture of the optimized target detection algorithm YOLOv7 model;

[0038] Figure 6 This is a schematic diagram of the process architecture of the C3Ghost module for model compression;

[0039] Figure 7 This is a schematic diagram of the pipeline defect image analysis and detection process;

[0040] In the figure, 1 is a pipeline inspection robot, 2 is a cable car, 3 is a decision terminal, 4 is a transmission pipeline image, 5 is a pipeline defect image, 6 is a feature extraction image, and 7 is a pipeline defect condition image. Specific implementation method:

[0041] The method for detecting defects in the transmission pipeline includes the following steps:

[0042] Step 1: Use a visual sensor to shoot videos of different long-distance pipelines. The visual sensor is arranged inside the pipeline. The visual sensor is a pipeline inspection robot 1 equipped with an industrial camera. These pipelines are mainly made of steel and PVC, and their diameters range from 400 to 1000 mm. The camera can obtain image data of the internal conditions of the pipeline in real time, and a total of 110 video clips were recorded under various environmental conditions. The duration of these video clips varies with the specific length of the pipeline and its condition, and the recording speed is 10 frames per second (FPS);

[0043] Step 2: Perform image extraction on the video to obtain a set of 64,000 images related to the transport pipeline;

[0044] Step 3: Filter out 3,000 images that meet the defect criteria from the image collection of the pipeline. The image database covers different types of problem images: 600 cases of corrosion problems, 600 cases of rupture, 1,000 cases of dislocation, and 800 cases of tube tumors. The corrosion and rupture defects are as follows: Figure 2 and Figure 3 As shown in the figure, personnel who have received basic training mark the images that meet the defect standards according to the defect type, marking rupture (PL), dislocation (CK), corrosion (FS) and tube tumor (GL) respectively, and finally obtain the pipeline defect identification image database (USDID);

[0045] Given the complexity and diversity of the internal environment of the transmission pipeline, not all images obtained are suitable for network training and testing. Samples with obvious defect targets and high image quality should be selected to build an inter-city long-distance transmission pipeline defect image database. The selection criteria mainly include: ensuring that the defect features can be easily identified by the naked eye; in order to enhance the system's ability to cope with different conditions, it is necessary to ensure that the selected images cover the defects that occur under a variety of lighting conditions; the defect types must be clear and accurate to avoid ambiguity.

[0046] In addition, if Figure 1 As shown, a cable car 2 is also configured. The cable car 2 adjusts the travel path of the pipeline inspection robot 1 through cables and a matching motion control system. At the same time, the sensor data sent by the pipeline inspection robot 1 is transmitted to the decision terminal 3 through the cable car 2. The decision terminal 3 also controls the operation of the cable car 2. As for the processing and decision-making part of data transmission, a model is constructed in the decision terminal 3 specifically for storing the conveying pipeline image 4 captured by the pipeline inspection robot 1 and analyzing and processing the conveying pipeline image 4. This model not only has a defect recognition function, but also integrates the management and control mechanism of the entire pipeline system.

[0047] Building a high-performance deep learning model depends on a large number of high-quality sample data sets. The number and quality of samples in the data set directly affect the stability and detection accuracy of the model. In this embodiment, a set of standardized inter-city long-distance pipeline defect recognition image libraries are created to train the target detection algorithm YOLOv7 model, aiming to develop a system with both high stability and excellent detection accuracy. Figure 4 shown.

[0048] Step 4: On this basis, further optimize the architecture design of the target detection algorithm YOLOv7 model, such as Figure 5 As shown in the figure, considering the actual needs of pipeline defect detection, the model compression C3Ghost module is used to replace the Efficient Layer Aggregation Networks (ELAN) module with more parameters in the target detection algorithm YOLOv7 model. The model compression C3Ghost module is a custom neural network module; this not only effectively reduces the number of model parameters and computational complexity, but also replaces the traditional convolutional layers of other parts with a simpler form to achieve a lightweight transformation of the overall network structure;

[0049] In response to the needs of small embedded devices, the lightweight network GhostNet architecture is integrated into the target detection algorithm YOLOv7 model to reduce the complexity of the model. Compared with the traditional convolutional neural network (CNN), the lightweight network GhostNet can significantly reduce the number of parameters and computing requirements while keeping the output feature map unchanged. The computing resources required are only about 1 / s of the original requirements; "s" is a variable representing a scale or reduction factor. Here, "1 / s" means that the computing resource requirements are reduced to a very low level;

[0050] The process architecture of the model compression C3Ghost module contains a series of components connected in sequence, such as Figure 6 As shown in the figure, specifically: first, there are two parallel convolutional layers Conv_1, and the processing result of one convolutional layer Conv_1 is imported into a lightweight network component GhostBottleNeck, which contains convolutional layers Conv_2 and Conv_3; next, the information flow processed by another convolutional layer Conv_1 and the lightweight network component GhostBottleNeck is integrated through the concatenation layer Concat_1, and then the features are fused through the concatenation layer Concat_2 and an addition operation Add; finally, all these features are concatenated through the concatenation layer Concat_3 and the final convolutional layer Conv_4 to complete the entire process;

[0051] Step 5: Use the pipeline defect recognition image database to train the target detection algorithm YOLOv7 model;

[0052] Step 6: Analyze the target detection algorithm YOLOv7 model layer by layer and calculate the floating-point operations involved in each layer; this can usually be achieved through tools or functions provided by deep learning frameworks (such as PyTorch, TensorFlow, etc.);

[0053] Step 7: Sort each layer from high to low according to the floating-point operations involved, and determine the ideal pruning ratio of each layer; the ideal pruning ratio is to reduce the size and amount of calculation of the model as much as possible while ensuring that the model performance (such as accuracy) is not significantly affected; compare the floating-point operations of each layer after sorting, and find out those layers with large floating-point operations and relatively small impact on model performance; these layers usually have greater optimization potential, because pruning these layers can significantly reduce the amount of calculation while having little impact on model performance;

[0054] Step 8, evaluate the impact of each layer on the floating-point operation performance under the ideal pruning ratio, so as to determine the n layers with the greatest optimization potential (the contribution to reducing the floating-point operation amount under the ideal pruning ratio is the greatest, and the performance degradation of the target detection algorithm YOLOv7 model is the smallest), so as to optimize the target detection algorithm YOLOv7 model for pipeline defects again; the method for determining n is: according to the goal of reducing the floating-point operation amount, a reasonable performance degradation threshold is set, starting from the layer with the highest floating-point operation amount after sorting, and pruning is performed layer by layer under the ideal pruning ratio. After each step of pruning, the target detection algorithm YOLOv7 model is tested with the verification data set to evaluate the change in the performance of the target detection algorithm YOLOv7 model until the set performance degradation threshold is reached or the floating-point operation amount can no longer be reduced. At this time, the number of pruned layers is the value of n, and the target detection algorithm YOLOv7 model is also optimized;

[0055] When determining the value of n, it is also necessary to comprehensively consider the requirements of the actual application scenario on model performance, inference speed, resource consumption, etc.;

[0056] The pruning operation can significantly reduce the number of model parameters, volume, and computing requirements, effectively solving common problems in current image processing algorithms, such as large model size, high computing cost, and poor cross-platform adaptability, and also improves the system's stability and detection accuracy.

[0057] Step 9: Figure 7 As shown, the pipeline defect recognition image 5 acquired in real time is input into the target detection algorithm YOLOv7 model optimized in step 8 for feature extraction to obtain a feature extraction image 6, and the pipeline defect recognition image 5 is analyzed using the optimized target detection algorithm YOLOv7 model to detect the defect condition of the pipeline and obtain a pipeline defect condition image 7.

[0058] In step 1, the video of the conveying pipeline is a video of the interior of the conveying pipeline.

[0059] In view of the small difference in image information between consecutive frames in the video, in step 2, the video is subjected to image extraction processing by extracting every three frames, thereby obtaining an image set of the conveying pipeline.

[0060] In step 3, Lambelimg software is used to mark images that meet the defect criteria.

[0061] In step 5, according to the performance of the target detection algorithm YOLOv7 model on the test data set in the pipeline defect recognition image database, the training parameters such as learning rate and batch size are adjusted to optimize the training process.

[0062] In step 8, the pruning operation is performed using the Performance-Aware Global Channel Pruning (PAGCP) pruning algorithm.

[0063] The PAGCP pruning algorithm is a framework for global channel pruning of multi-task CNN models. This framework transforms the model compression problem into an optimization problem of joint channel saliency indicators, comprehensively considers the joint impact of inter-layer and intra-layer channels on the compression effect of multi-task models, and further develops into a sequential greedy pruning method based on performance-aware standards. In this way, without introducing regularization penalties, it can effectively identify and eliminate global redundant channels in multi-task models, thereby achieving an efficient performance optimization strategy.

Claims

1. A method for detecting defects in a conveying pipeline, characterized in that: Contains the following steps: Step 1: Use a visual sensor to capture a video of the conveying pipeline; Step 2: Perform image extraction on the video to obtain a pipeline image set; Step 3: Filter out images that meet the defect criteria from the pipeline image collection, mark the images that meet the defect criteria according to the defect type, and mark rupture, dislocation, corrosion, and tube tumor respectively, and finally obtain a pipeline defect recognition image database; Step 4. Optimize the architecture of the target detection algorithm YOLOv7 model: Use the model compression C3Ghost module to replace the efficient aggregation network ELAN module in the target detection algorithm YOLOv7 model; the process architecture of the model compression C3Ghost module is as follows: first, there are two parallel convolutional layers Conv_1, and the processing result of one convolutional layer Conv_1 is imported into a lightweight network component GhostBottleNeck, which contains convolutional layers Conv_2 and Conv_3; next, the information flow processed by another convolutional layer Conv_1 and the lightweight network component GhostBottleNeck is integrated through the splicing layer Concat_1, and then the features are fused through the splicing layer Concat_2 and an addition operation Add; finally, all these features pass through the splicing layer Concat_3, and the whole process is completed through the final convolutional layer Conv_4; Step 5: Use the pipeline defect recognition image database to train the target detection algorithm YOLOv7 model; Step 6: Analyze the target detection algorithm YOLOv7 model layer by layer and calculate the floating-point operations involved in each layer; Step 7: Sort each layer from high to low according to the floating point operations involved, and determine the ideal pruning ratio of each layer; Step 8, evaluate the impact of each layer on the floating-point operation performance under the ideal pruning ratio, so as to determine the n layers with the greatest optimization potential, and thus optimize the target detection algorithm YOLOv7 model again; the method for determining n is: according to the goal of reducing the floating-point operation amount, a reasonable performance degradation threshold is set, starting from the layer with the highest floating-point operation amount after sorting, pruning is performed layer by layer under the ideal pruning ratio, and after each pruning step, the target detection algorithm YOLOv7 model is tested to evaluate the change in the performance of the target detection algorithm YOLOv7 model until the set performance degradation threshold is reached or the floating-point operation amount can no longer be reduced. At this time, the number of pruned layers is the value of n, and the target detection algorithm YOLOv7 model is also optimized; Step 9: Use the target detection algorithm YOLOv7 model to analyze the pipeline defect recognition image acquired in real time to detect the defect condition of the pipeline.

2. The method for detecting defects in a conveying pipeline according to claim 1 is characterized in that: In step 1, the video of the conveying pipeline is a video of the interior of the conveying pipeline.

3. The method for detecting defects in a conveying pipeline according to claim 1, characterized in that: In the step 2, the video is subjected to image extraction processing by extracting once every M frames, so as to obtain a pipeline image set, where M is a natural number greater than or equal to 3.

4. The method for detecting defects in a conveying pipeline according to claim 1, characterized in that: In step 3, Lambelimg software is used to mark images that meet the defect standards.

5. The method for detecting defects in a conveying pipeline according to claim 1, characterized in that: In step 5, according to the performance of the target detection algorithm YOLOv7 model on the test data set in the pipeline defect recognition image database, the training parameters such as learning rate and batch size are adjusted to optimize the training process.

6. The method for detecting defects in a conveying pipeline according to claim 1, characterized in that: In step 8, the pruning operation is implemented using the performance-aware global channel pruning (PAGCP) pruning algorithm.

Images