Pipeline defect detection and analysis method and system integrating labeling and recognition

Abstract

The invention provides a pipeline defect detection analysis method and system fusing labeling and recognition, and relates to the technical field of electrical digital data processing.The method is executed by a processor and comprises the steps that preprocessing pipeline inner wall information is subjected to execution analysis based on a deep network, and an analysis result is obtained; the analysis result is screened and stored based on the deep network; performing refined labeling on the screened and stored data to obtain standby data; and packaging and exporting the to-be-used data. According to the method and the device, the problems of data transmission delay, overlarge storage pressure and low redundant data processing efficiency in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the technical field of electrical digital data processing, and in particular, to a pipeline defect detection and analysis method and system that integrates annotation and recognition. Background Art

[0002] Currently, with the continuous development of industrial pipeline detection technologies, the detection of defects inside pipelines has become an important research field. Traditional pipeline detection methods usually rely on manual inspection, image acquisition, and backend data processing. Although these methods can detect defects in pipelines to a certain extent, there are still many problems. First, the amount of image data generated during pipeline detection is huge. Traditional processing methods need to upload this data to the cloud or a server for analysis, which not only increases the latency of data transmission but also requires a large amount of storage space and computing resources. Second, image data often contains a large amount of noise and redundant information. Traditional detection methods are inefficient in data preprocessing and are difficult to effectively filter out valuable information, thus affecting the accuracy and efficiency of detection.

[0003] To overcome these problems, in recent years, with the development of edge computing and deep learning technologies, more and more research has begun to attempt to introduce intelligent image processing methods in pipeline detection. Edge computing reduces the latency of data transmission by moving data processing to local devices and avoids the high latency and bandwidth bottlenecks brought by cloud computing. However, deep learning models are usually complex and have high computing requirements, and there are challenges in directly deploying them on resource-constrained edge devices. To solve this problem, model distillation technology has been widely used to transfer the knowledge of large, high-performance "teacher" models to small, lightweight "student" models, thereby reducing computing and storage requirements while maintaining high accuracy and adapting to the limitations of edge devices. Deep learning, especially methods based on convolutional neural networks, can automatically learn the features of pipeline defects on large-scale datasets, greatly improving the accuracy of defect detection. However, although these technologies have achieved good results in some applications, existing deep learning methods still face problems such as high data transmission latency, excessive storage pressure, and low efficiency in processing redundant data. For example, Patent CN114241282B proposes a method for scene recognition on edge devices based on knowledge distillation. This method pre-trains a teacher model on a large dataset and fine-tunes it on a scene recognition dataset to construct a high-performance teacher model. Subsequently, the student model is trained using knowledge distillation technology to achieve efficient and accurate scene recognition on edge devices. This method effectively solves the problem of difficult deployment of high-performance models on edge devices and improves the recognition speed and accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a pipeline defect detection and analysis method and system that integrates annotation and recognition, which solves the problems of data transmission delay, excessive storage pressure, and low efficiency in processing redundant data in the prior art.

[0005] The technical solution of this application: This application provides a pipeline defect detection and analysis method that integrates annotation and recognition. This method is executed by a processor and includes: Performing analysis on the preprocessed inner wall information of the pipeline based on a deep network to obtain an analysis result; Performing screening and storage on the analysis result based on a deep network; Performing refined annotation on the screened and stored data to obtain data for use; Performing encapsulation and export on the data for use.

[0006] Furthermore, the preprocessed inner wall information of the pipeline is at least a video of the inner wall of the pipeline that is collected by a camera and transmitted to an edge device for processing.

[0007] Furthermore, the preprocessing at least includes denoising, image enhancement, and key frame extraction.

[0008] Furthermore, the deep network at least includes a backbone network, a feature fusion module, and an output head.

[0009] Furthermore, the output head at least includes a first output head and a second output head; among them, the first output head identifies the spatial position of the defect and generates a mask region, and the second output head classifies the defect and outputs the probability distribution result of the category.

[0010] Furthermore, the performing screening and storage on the analysis result based on a deep network at least includes: Performing comprehensive analysis on the mask and category probability of the defect region, where: Setting dual screening conditions based on the mask region area and category probability: First, calculate the pixel ratio S of the defect mask region. If S is lower than the set threshold S min , it is considered that the defect region of this frame is too small, and it is determined as an invalid frame and discarded; Second, calculate the classification probability P of the defect category. If P is lower than the preset threshold P min , it is considered that the recognition of this frame is uncertain, and it is determined as an invalid frame and discarded; Only when the S and P of a certain frame simultaneously meet their respective threshold requirements, it is determined that this frame is a valid frame, and its original image, defect mask, defect category, and probability distribution are extracted, marked as key data, and stored; In addition, to adapt to the feature distributions of different types of defects, the thresholds S min and P min are dynamically adjusted based on statistical information or manual feedback to optimize the recognition accuracy and improve the data utilization rate.

[0011] Further, for valid frames, extract the corresponding original images, masks of defect regions, defect categories, and probability distributions, and integrate them into structured data.

[0012] Further, the refined annotation of the screened and stored data to obtain the data to be used at least includes: After completing data screening and storage, in the annotation interface, based on the generated mask regions and category probabilities, and based on the target requirements, perform manual annotation on the mask regions and category probabilities to obtain the data to be used.

[0013] On the other hand, the present application provides a pipeline defect detection and analysis system integrating annotation and recognition. The system at least includes a processor, and further includes: An analysis module, which is used to perform analysis on the preprocessed inner wall information of the pipeline based on a deep network to obtain an analysis result; A screening module, which is used to perform screening and storage on the analysis result based on a deep network; A marking module, which is used to perform refined marking on the screened and stored data to obtain the data to be used; An encapsulation module, which is used to encapsulate and export the data to be used.

[0014] According to the above technical features, the beneficial effects of the present application are: 1. Through edge computing technology and deep learning inference, the present application realizes efficient defect detection and data screening, greatly improving the detection efficiency: In traditional methods, images usually need to be uploaded to the cloud or the background server for processing, which not only increases the transmission delay but also requires powerful computing resources. In contrast, the present application directly preprocesses the images through edge devices and uses hardware acceleration for deep network inference, realizing real-time processing, reducing the time delay of data transmission and processing, and significantly improving the overall detection efficiency. Moreover, intelligent screening and data reduction. By intelligent screening, discard low-probability frames that do not meet the conditions and only retain high-quality image data, which not only reduces the storage burden but also makes subsequent processing and manual annotation work more focused on high-value data, thus improving the work efficiency.

[0015] 2. The present application achieves high-precision localization and classification of pipeline defects through the dual-head structure of the deep network. Different from general object detection tasks, pipeline defects have the characteristics of irregular morphology, fuzzy boundaries, and multi-scale distribution. Therefore, the dual-output head strategy of the present application integrates global and local information in the detection head to improve the accuracy of the defect mask, and combines context information in the classification head to optimize category judgment, so as to adapt to different types of pipeline defects. Compared with traditional methods, the present application can accurately locate the defect positions in the image and provide clear category judgments for each defect, greatly improving the detection accuracy: In the present application, the deep network inference uses the trained deep network to simultaneously detect the defect positions and classify the categories through the dual-head structure. One accurately identifies the spatial position of the defect and generates the mask region, and the other classifies the defect and outputs the probability distribution of the category. This joint processing method enables the system to comprehensively and accurately extract defect information from the image, avoiding the errors and missed detections that may be caused by manual calibration or simple pattern recognition in traditional methods.

[0016] 3. The present application reduces the storage cost and data transmission pressure by only storing valid defect data through intelligent screening and reducing unnecessary image data storage: Through intelligent screening, by comprehensively analyzing the mask of the defect area and the category probability, reasonable thresholds are set to screen valid image data. This process effectively avoids the storage of a large amount of irrelevant image data in traditional methods, making the finally stored data not only more concise but also more targeted, achieving the effect of greatly reducing the storage and transmission burden.

[0017] 4. The present application has a high degree of automation, reduces the need for manual intervention, and makes the defect detection process more automated and intelligent: Edge computing and deep network inference can automatically detect and classify defects in the pipeline through edge computing and deep learning inference, reducing the necessity of manual intervention. By combining intelligent screening and manual fine annotation, manual workers only need to verify and correct the inference results, rather than processing each image from scratch, greatly improving the overall detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic flow chart of the method provided by the present application; Figure 2 It is a schematic diagram of the network structure in the method provided by the present application; Figure 3 It is a schematic diagram of the structure of the system provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0020] Embodiment Please refer to Figures 1 - 3 , an embodiment of the present application provides a pipeline defect detection and analysis method integrating annotation and recognition, which relates to the technical field of electronic digital data processing. This method is executed by a processor. Among them, the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.; the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the corresponding terminal device, and uses various interfaces and lines to connect various parts of the terminal device.

[0021] Refer to the attached Figure 1 , specifically including: Perform an analysis on the preprocessed pipeline inner wall information based on a deep network to obtain an analysis result; Perform screening and storage on the analysis result based on a deep network; Perform refined annotation on the screened and stored data to obtain data for use; Perform encapsulation and export on the data for use.

[0022] It should be noted that a pipeline defect detection and analysis method integrating annotation and recognition is provided. In the backend processing of this method, the trained deep neural network is used to infer the image, combining two tasks of position detection and category classification. It can not only accurately locate the defect position but also identify the type of the defect. Through this method, efficient and accurate defect detection can be achieved, and at the same time, the storage and calculation of redundant data in the traditional method are avoided, thus improving the work efficiency and detection accuracy. In this embodiment, the pipeline defect detection method combining edge computing and deep learning inference solves technical problems such as data transmission delay, excessive storage pressure, and low processing efficiency of redundant data while improving the detection accuracy. On the other hand, this method not only optimizes the entire process of pipeline detection but also provides a feasible technical solution for future promotion and application in other industrial fields.

[0023] Furthermore, the preprocessed pipeline inner wall information is at least the pipeline inner wall video that is collected by the camera and then transmitted to the edge device for processing. Specifically, the high-definition camera on the in-pipeline detection device is used for image collection to record the high-definition video data of the pipeline inner wall in real time; after the collection is completed, the data is transmitted to the edge device for preliminary preprocessing.

[0024] Furthermore, the preprocessing at least includes denoising, image enhancement, and key frame extraction. Specifically, a pipeline defect detection and analysis method integrating annotation and recognition provided processes video data in real time through the edge device. In the preprocessing operation, denoising, image enhancement, and key frame extraction are performed. Through denoising, the image noise interference is reduced. Through image enhancement, the key details are highlighted. Through a specific algorithm, key frames are extracted from the video stream, which significantly reduces the amount of image data and also reduces the storage and transmission pressure brought by the large amount of data in the traditional method.

[0025] Furthermore, the deep network at least includes a backbone network, a feature fusion module, and an output head. As Figure 2 shown, the network structure will include the following main parts: a backbone network (Backbone), a feature fusion module (Neck), and two output heads (Head). The content of the above main parts is described in detail: 1. Backbone (backbone network): The input is an RGB image (3 channels) with a size of 1920x1080. The pre-trained ResNet50 is used as the backbone network to extract the low-level to high-level features of the image; 2. Neck (feature fusion module): The features extracted by the backbone network are further processed and fused using FPN (Feature Pyramid Network) so as to be able to provide the required feature information for the subsequent two output heads at the same time; 3. Heads: The output of the network needs to include two different tasks: One is the Segmentation Head: This head is responsible for generating a mask image of the same size as the input image, representing the location of the defect area; using convolutional layers for upsampling and convolutional operations, and finally outputting a binary mask image of the same size as the input image, where the value of each pixel indicates whether that location is a defect area; The other is the Classification Head: This head is responsible for assigning a classification label (such as crack, corrosion, etc.) to each defect area; outputting the probability distribution of each defect category through a fully connected layer; 4. Output categories: The pipeline defect categories that can be recognized include at least: Category 1: Crack; Category 2: Corrosion; Category 3: Deformation; Category 4: Weld defect; Category 5: Blockage; Category 6: Drop-off; Category 7: Leakage; Category 8: Normal.

[0026] Details of the network structure training process: During the network training process, an image dataset containing labeled defect locations and defect categories needs to be used. Each image data should not only have the original image but also the corresponding binary mask image, which marks the specific location of the pipeline defect and also contains the defect category label. To improve the generalization ability of the network and enhance the robustness of the model, data augmentation techniques will be adopted during the training process, including random cropping, rotation, flipping, color change, etc. During training, the Segmentation Head and the Classification Head will be trained simultaneously. The Segmentation Head uses the binary cross-entropy loss function to calculate the defect prediction result of each pixel point, so as to generate a binary mask image of the defect area; the Classification Head uses the cross-entropy loss function to calculate the classification probability of each defect area for outputting the category of the defect. The final total loss function is the weighted sum of these two losses, and the weighting coefficient can be adjusted according to specific requirements to balance the optimization of the two tasks. During the training process, the convolutional layers in the Backbone are frozen to avoid repeated training, reduce the amount of calculation, utilize the features it has learned on a large-scale dataset, and improve the overall training efficiency of the network. At the same time, the Neck part, as a fixed feature fusion layer, does not need to be trained in this method. This can greatly reduce the time and computing resources required for network training.

[0027] Furthermore, the output head includes at least a first output head and a second output head; wherein, the first output head identifies the spatial position of the defect and generates a mask region, and the second output head classifies the defect and outputs the probability distribution result of the category. Specifically, after preprocessing, the key-frame image is input into an offline-trained deep network for inference analysis. This network contains two output heads (Head), namely the first output head and the second output head, which are responsible for different tasks respectively. Among them: the first output head is used to detect the position of the defect and output the mask map of the defect region, and these mask regions can accurately mark the specific positions where defects may exist in the pipeline; the second output head is used for the category classification of the defect and outputs the probability distribution of the defect category. Through this structural design of the dual-head network, spatial position information and semantic classification information can be obtained simultaneously; the inference process of the deep network is carried out in real time on the edge device using the hardware acceleration environment, which can quickly process large-scale key-frame data, and the output results are directly used for further intelligent judgment.

[0028] Furthermore, screening and storage are performed on the analysis results based on the deep network, including at least: Perform comprehensive analysis on the mask of the defect region and the category probability, where: When the classification probability of any frame reaches the preset threshold, it is determined that the frame is valid, which contains valid defect information, and it is marked as key data; when the classification probability of any frame does not reach the preset threshold, it is determined that the frame is invalid, which contains data redundancy, and it is discarded.

[0029] Specifically, after the inference is completed, intelligent screening continues according to the output results of the deep network, that is, comprehensive analysis is performed on the mask of the defect region and the category probability. Specifically: perform comprehensive analysis on the mask of the defect region and the category probability, where double screening conditions are set based on the mask region area and the category probability: First, calculate the pixel ratio S of the defect mask region. If S is lower than the set threshold S min , it is considered that the defect region of this frame is too small, and it is determined as an invalid frame and discarded; second, calculate the classification probability P of the defect category. If P is lower than the preset threshold P min , it is considered that the recognition of this frame is uncertain, and it is determined as an invalid frame and discarded; only when the S and P of a certain frame simultaneously meet their respective threshold requirements, it is determined that the frame is a valid frame, and its original image, defect mask, defect category and probability distribution are extracted, marked as key data and stored; in addition, to adapt to the feature distributions of different types of defects, the thresholds S min and P min can be dynamically adjusted based on statistical information or manual feedback to optimize the recognition accuracy and improve the data utilization rate.

[0030] Further, for the valid frames, extract the corresponding original images, masks of defect regions, defect categories, and probability distributions, and integrate them into structured data. Specifically, for the frames determined to be valid, extract the corresponding original images, masks of defect regions, defect categories, and probability distributions, and integrate this information into structured data. The screening process used in this method not only ensures the high quality and high value of the stored data, but also significantly reduces the burden of irrelevant data on storage and subsequent analysis.

[0031] Further, perform refined annotation on the screened and stored data to obtain data for use, including at least: After completing data screening and storage, in the annotation interface, based on the generated mask regions and category probabilities, and according to the target requirements, perform manual annotation on the mask regions and category probabilities to obtain data for use.

[0032] Specifically, after completing intelligent screening and data storage, push the selected data to the manual refinement annotation module. Specifically, in the annotation interface, the operator can clearly see the mask regions and category probabilities generated by the system, and confirm or adjust these regions and categories as needed. The manual annotation results will further improve the accuracy of the data and provide high-quality training data for model optimization. At the same time, the annotation interface provides intuitive operation tools and a friendly interaction design to ensure that the operator can complete the annotation task efficiently.

[0033] Finally, the preliminarily processed data and the defect images with mask information will be packaged and exported for subsequent users to use.

[0034] On the other hand, this application provides a pipeline defect detection and analysis system that integrates annotation and recognition. The system at least includes a processor, and further includes: An analysis module, which is used to analyze the preprocessed inner wall information of the pipeline based on a deep network to obtain an analysis result; A screening module, which is used to screen and store the analysis result based on a deep network; An annotation module, which is used to perform refined annotation on the screened and stored data to obtain data for use; A packaging module, which is used to package and export the data for use.

[0035] It is worth noting that The system is divided into one or more modules. One or more modules are stored in a memory and executed by a processor to complete the present application. One or more modules can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of a computer program. For example, a computer program can be divided into an analysis module, a screening module, a labeling module, and a packaging module. The specific functions of each module are as follows: The analysis module is used to perform an analysis on the inner wall information of the pretreatment pipeline based on a deep network to obtain an analysis result; the screening module is used to perform screening and storage on the analysis result based on a deep network; the labeling module is used to perform refined labeling on the screened and stored data to obtain data for use; the packaging module is used to perform packaging and export on the data for use. The system provided in this embodiment is specifically a visual acquisition system for pipeline defects integrating labeling and intelligent recognition. In the prior art, traditional pipeline detection and analysis systems usually need to store a large amount of high-definition image data in on-board devices, and then export and perform recognition processing. Different from this, the system provided in this embodiment gives full play to the computing power of edge devices, performs preliminary preprocessing of high-definition images on edge devices, and only retains key frames suitable for backend analysis, significantly reducing the data explosion problem brought by massive data. Moreover, during the processing of edge devices, the hardware acceleration capability is used to quickly predict and roughly label, greatly improving the subsequent labeling efficiency and accuracy. Furthermore, not only the work efficiency is improved, but also the workload of manual labeling is reduced, and it can ensure that the data acquisition link has a certain defect detection and analysis ability.

[0036] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A pipeline defect detection and analysis method integrating labeling and recognition, the method being executed by a processor, characterized in that: Perform analysis on the inner wall information of the pre-processed pipeline based on the deep network to obtain the analysis results; Filter and store the analysis results based on the deep network; Perform refined annotation on the filtered and stored data to obtain the data to be used; Perform package export on the data to be used.

2. The method according to claim 1, characterized in that: The pre-processed pipeline inner wall information is at least a pipeline inner wall video that is collected by a camera and transmitted to an edge device for processing.

3. The method according to claim 2, characterized in that The preprocessing includes at least denoising, image enhancement and key frame extraction.

4. The method according to claim 1, characterized in that The deep network at least includes a backbone network, a feature fusion module and an output head.

5. The method according to claim 4, characterized in that The output head includes at least a first output head and a second output head; wherein the first output head identifies the spatial position of the defect and generates a mask area, and the second output head classifies the defects and outputs the probability distribution result of the category.

6. The method according to claim 1, characterized in that The screening and storage of the analysis results based on the deep network at least includes: A comprehensive analysis is performed on the mask and class probability of the defect area, where: Based on the mask area and category probability, dual screening conditions are set: First, the pixel ratio S of the defect mask area in any frame is calculated. If S is lower than the set threshold S min , the defect area of ​​the frame is considered too small, and it is judged as an invalid frame and discarded; secondly, the classification probability P of the defect category is calculated. If P is lower than the preset threshold P min , the frame recognition is considered uncertain, and it is judged as an invalid frame and discarded; only when S and P of a frame meet their respective threshold requirements at the same time, the frame is judged as a valid frame, and its original image, defect mask, defect category and probability distribution are extracted, marked as key data and stored; in addition, in order to adapt to the characteristic distribution of different types of defects, the threshold S min and P min Dynamically adjust based on statistical information or manual feedback to optimize recognition accuracy and improve data utilization.

7. The method according to claim 6, characterized in that The corresponding original image, mask of the defect area, defect category and probability distribution are extracted from the valid frame and integrated into structured data.

8. The method according to claim 1, characterized in that The performing of refined annotation on the screened and stored data to obtain the data to be used at least includes: After completing data screening and storage, in the annotation interface, manual annotation is performed on the mask area and category probability according to the generated mask area and category probability and based on target requirements to obtain the data to be used.

9. A pipeline defect detection and analysis system integrating labeling and recognition, the system at least comprising a processor, characterized in that: Also includes: An analysis module, the analysis module is used to perform analysis on the inner wall information of the pre-processed pipeline based on a deep network to obtain an analysis result; A screening module, the screening module is used to perform screening and storage on the analysis results based on the deep network; A labeling module, which is used to perform refined labeling on the screened and stored data to obtain data to be used; The encapsulation module is used to perform encapsulation export on the data to be used.

Images