A pipeline defect detection analysis method and system combining labeling and identification

Abstract

The application provides a pipeline defect detection analysis method and system fusing labeling and identification, and relates to the technical field of electric digital data processing. The method is executed by a processor and comprises the following steps: performing analysis on pretreated pipeline inner wall information based on a deep network to obtain an analysis result; performing screening and storage on the analysis result based on the deep network; performing fine labeling on the screened and stored data to obtain standby data; and performing packaging and exporting on the standby data. The application solves the problems of transmission delay, excessive storage pressure and low processing efficiency of redundant data in the prior art.

Description

Technical Field

[0001] The present application relates to the technical field of electrical digital data processing, and in particular to a pipeline defect detection and analysis method and system integrating labeling and identification. Background Art

[0002] Currently, with the continuous development of industrial pipeline inspection technology, the detection of defects within pipelines has become an important research area. Traditional pipeline inspection methods generally rely on manual inspection, image acquisition, and back-end data processing. While these methods can detect defects in pipelines to a certain extent, they still have many problems. First, the amount of image data generated during pipeline inspection is enormous. Traditional processing methods require uploading this data to the cloud or server for analysis, which not only increases data transmission delays but also requires a large amount of storage space and computing resources. Second, image data often contains a large amount of noise and redundant information. Traditional inspection methods are inefficient in data preprocessing, making it difficult to effectively filter out valuable information, thus affecting the accuracy and efficiency of inspection.

[0003] To overcome these issues, with the recent development of edge computing and deep learning technologies, a growing number of researchers have begun to explore the application of intelligent image processing methods in pipeline inspection. Edge computing reduces data transmission latency by moving data processing to local devices, avoiding the high latency and bandwidth bottlenecks associated with cloud computing. However, deep learning models are typically complex and computationally demanding, making direct deployment on resource-constrained edge devices challenging. To address this issue, model distillation techniques have been widely used to transfer knowledge from a large, high-performance "teacher" model to a small, lightweight "student" model. This approach maintains high accuracy while reducing computational and storage requirements, adapting to the limitations of edge devices. Deep learning, particularly methods based on convolutional neural networks, can automatically learn the characteristics of pipeline defects on large datasets, significantly improving defect detection accuracy. However, while these techniques have achieved promising results in certain applications, existing deep learning methods still face challenges with data transmission latency, excessive storage pressure, and inefficient processing of redundant data. For example, patent CN114241282B proposes a scene recognition method for edge devices based on knowledge distillation. This method constructs a high-performance teacher model by pre-training the teacher model on a large dataset and fine-tuning it on a scene recognition dataset. Subsequently, the student model is trained using knowledge distillation technology to achieve efficient and accurate scene recognition on edge devices. This method effectively solves the problem of high-performance models being difficult to deploy on edge devices, improving recognition speed and accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a pipeline defect detection and analysis method and system that integrates labeling and identification, which solves the problems of data transmission delay, excessive storage pressure and low efficiency in redundant data processing in the prior art.

[0005] The technical solution of this application:

[0006] The present application provides a pipeline defect detection and analysis method that integrates labeling and recognition. The method is executed by a processor and includes:

[0007] Perform analysis on the inner wall information of the pre-processed pipeline based on the deep network to obtain analysis results;

[0008] Filter and store the analysis results based on the deep network;

[0009] Perform refined annotation on the filtered and stored data to obtain the data to be used;

[0010] Perform package export on the data to be used.

[0011] Furthermore, the pre-processed pipeline inner wall information is at least a pipeline inner wall video that is captured by a camera and transmitted to an edge device for processing.

[0012] Furthermore, the preprocessing includes at least denoising, image enhancement and key frame extraction.

[0013] Furthermore, the deep network includes at least a backbone network, a feature fusion module and an output head.

[0014] Furthermore, the output head includes at least a first output head and a second output head; wherein the first output head identifies the spatial position of the defect and generates a mask area, and the second output head classifies the defect and outputs a probability distribution result of the category.

[0015] Furthermore, the screening and storage of the analysis results based on the deep network at least includes:

[0016] Perform a comprehensive analysis of the defect area mask and class probability, where:

[0017] Set dual screening conditions based on the mask area and category probability: First, calculate the pixel ratio S of the defect mask area. If S is lower than the set threshold S min , then the defect area of ​​the frame is considered too small, and it is judged as an invalid frame and discarded; secondly, the classification probability P of the defect category is calculated. If P is lower than the preset threshold P minIf S and P of a frame do not meet the threshold requirements, the frame is considered to be uncertain, and the frame is determined to be an invalid frame and discarded; only when S and P of a frame meet the threshold requirements, the frame is determined to be a valid frame, and the original image, the defect mask, the defect category and the probability distribution of the frame are extracted, and the frame is marked as key data and stored; in addition, in order to adapt to the feature distribution of different types of defects, the threshold S min and P min The threshold S

[0018] Further, the valid frame is executed to extract the corresponding original image, the defect mask, the defect category and the probability distribution, and is executed to integrate into structured data.

[0019] Further, the screening and storage data are executed to obtain the to-be-used data, and the to-be-used data at least includes:

[0020] After the data screening and storage are completed, in the labeling interface, the mask area and the category probability are generated according to the mask area and the category probability, and the mask area and the category probability are executed to be labeled based on artificial based on the target demand, and the to-be-used data is obtained.

[0021] Another aspect of the present application provides a pipeline defect detection and analysis system integrating labeling and recognition, the system at least includes a processor, and further includes:

[0022] An analysis module, the analysis module is used for executing analysis on the preprocessed pipeline inner wall information based on a deep network to obtain an analysis result;

[0023] A screening module, the screening module is used for executing screening and storage on the analysis result based on a deep network;

[0024] A labeling module, the labeling module is used for executing fine labeling on the screening and storage data to obtain to-be-used data;

[0025] An encapsulation module, the encapsulation module is used for executing encapsulation and export on the to-be-used data.

[0026] According to the above technical features, the beneficial effects of the present application are:

[0027] 1. This application uses edge computing technology and deep learning reasoning to achieve efficient defect detection and data screening, greatly improving detection efficiency: In traditional methods, images usually need to be uploaded to the cloud or backend server for processing, which not only increases transmission delays but also requires powerful computing resources. In contrast, this application directly pre-processes images through edge devices and uses hardware acceleration for deep network reasoning, achieving real-time processing, reducing the time delay for data transmission and processing, and significantly improving overall detection efficiency. In addition, intelligent screening and data reduction, through intelligent screening, low-probability frames that do not meet the conditions are discarded, and only high-quality image data is retained, which not only reduces the storage burden, but also makes subsequent processing and manual labeling work more focused on high-value data, thereby improving work efficiency.

[0028] 2. This application achieves high-precision positioning and classification of pipeline defects through a dual-head structure of a deep network. Unlike general target detection tasks, pipeline defects have the characteristics of irregular shapes, fuzzy boundaries, and multi-scale distribution. To this end, the dual-output head strategy of this application fuses global and local information in the detection head to improve the accuracy of the defect mask, and optimizes category judgment in the classification head in combination with contextual information to adapt to different types of pipeline defects. Compared with traditional methods, this application can accurately calibrate the defect position in the image and provide a clear category judgment for each defect, greatly improving the detection accuracy: the deep network inference in this application uses a trained deep network to simultaneously perform defect position detection and category classification through a two-head structure. One accurately identifies the spatial position of the defect and generates a mask area, while the other classifies the defect and outputs the probability distribution of the category. This joint processing method enables the system to comprehensively and accurately extract defect information from the image, avoiding errors and missed detections that may be caused by manual calibration or simple pattern recognition in traditional methods.

[0029] 3. This application reduces storage costs and data transmission pressure by intelligent screening, storing only valid defect data and reducing unnecessary image data storage: Intelligent screening, through comprehensive analysis of the mask and category probability of the defect area, sets a reasonable threshold to screen valid image data. This process effectively avoids the storage of a large amount of irrelevant image data in traditional methods, making the final stored data not only more streamlined but also more targeted, achieving the effect of greatly reducing the burden of storage and transmission.

[0030] 4. This application features a high degree of automation, reducing the need for human intervention and making the defect detection process more automated and intelligent: Edge computing and deep network reasoning enable automatic detection and classification of defects in pipelines, reducing the need for human intervention. By combining intelligent screening with refined manual annotation, humans only need to verify and correct the inference results, eliminating the need to process each image from scratch, significantly improving overall detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of the process provided in this application;

[0032] Figure 2 A schematic diagram of the network structure in the method provided in this application;

[0033] Figure 3 A schematic diagram of the structure of the system provided in this application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0035] Example

[0036] Please refer to Figure 1-Figure 3 Embodiments of the present application provide a pipeline defect detection and analysis method that integrates labeling and identification, relating to the field of electrical digital data processing technology. The method is executed by a processor, wherein the processor can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.; the general-purpose processor can be a microprocessor or any conventional processor, etc. The processor corresponds to the control center of a terminal device, which uses various interfaces and lines to connect various parts of the terminal device.

[0037] Reference Attachment Figure 1 , specifically including:

[0038] Perform analysis on the inner wall information of the pre-processed pipeline based on the deep network to obtain analysis results;

[0039] Filter and store the analysis results based on the deep network;

[0040] Perform refined annotation on the filtered and stored data to obtain the data to be used;

[0041] Perform package export on the data to be used.

[0042] It is worth noting that a pipeline defect detection and analysis method that integrates labeling and recognition is provided. During the back-end processing, this method uses a trained deep neural network to reason about the image, combining the two tasks of position detection and category classification. It can not only accurately locate the defect position, but also identify the type of defect. Through this method, efficient and accurate defect detection can be achieved, while avoiding the storage and calculation of redundant data in traditional methods, thereby improving work efficiency and detection accuracy. This embodiment combines edge computing with deep learning reasoning to improve the detection accuracy of pipeline defect detection methods while solving technical problems such as data transmission delays, excessive storage pressure, and inefficient redundant data processing. On the other hand, this method not only optimizes the entire process of pipeline detection, but also provides a feasible technical solution for future promotion and application in other industrial fields.

[0043] Furthermore, the pre-processed pipeline inner wall information includes at least video of the pipeline inner wall captured by a camera and transmitted to an edge device for processing. Specifically, a high-definition camera on the pipeline inspection device captures images and records high-definition video data of the pipeline inner wall in real time. After the acquisition is completed, the data is transmitted to the edge device for preliminary pre-processing.

[0044] Furthermore, preprocessing includes at least denoising, image enhancement, and keyframe extraction. Specifically, a pipeline defect detection and analysis method that integrates annotation and recognition processes video data in real time via edge devices. During preprocessing, denoising, image enhancement, and keyframe extraction are performed. Denoising reduces image noise interference, image enhancement highlights key details, and a specific algorithm extracts keyframes from the video stream, significantly reducing the amount of image data and alleviating the storage and transmission pressures associated with massive amounts of data in traditional methods.

[0045] Furthermore, the deep network includes at least a backbone network, a feature fusion module, and an output head. Figure 2 As shown in the figure, the network structure will include the following main parts: backbone network (Backbone), feature fusion module (Neck), and two output heads (Head). The contents of the above main parts are described in detail:

[0046] 1. Backbone: The input is a 1920x1080 RGB image (3 channels), and a pre-trained ResNet50 is used as the backbone network to extract low-level to high-level features of the image;

[0047] 2. Neck (feature fusion module): The features extracted by the backbone network are further processed and fused using FPN (Feature Pyramid Network) to provide the required feature information for the subsequent two output heads at the same time;

[0048] 3. Heads: The output of the network needs to include two different tasks:

[0049] The first is the Segmentation Head, which generates a mask of the same size as the input image to indicate the location of the defective area. It uses a convolutional layer to perform upsampling and convolution operations, ultimately outputting a binary mask of the same size as the input image. The value of each pixel indicates whether the location is a defective area.

[0050] The other is the defect classification head: This head is responsible for assigning a classification label (such as crack, corrosion, etc.) to each defect area and outputs the probability distribution of each defect category through a fully connected layer.

[0051] 4. Output categories: Identifiable pipeline defect categories, including at least: Category 1: Crack; Category 2: Corrosion; Category 3: Deformation; Category 4: Weld defect; Category 5: Blockage; Category 6: Falling off; Category 7: Leakage; Category 8: Normal.

[0052] Detailed description of the network structure training process: in the network training process, an image dataset containing labeled defect position and defect category needs to be used. Each image data not only has an original image, but also contains a corresponding binary mask image, in which the specific position of the pipeline defect is marked, and the category label of the defect is also included. In order to improve the generalization ability of the network and enhance the robustness of the model, data enhancement techniques will be used in the training process, including random cropping, rotation, flipping, color change, etc. During training, the segmentation head and the classification head will be trained at the same time. The segmentation head uses binary cross entropy loss function to calculate the defect prediction result of each pixel point, so as to generate the binary mask image of the defect area; the classification head uses cross entropy loss function to calculate the classification probability of each defect area, which is used to output the category of the defect. The final total loss function is the weighted sum of the two losses, and the weighting coefficient can be adjusted according to the specific requirements to balance the optimization of the two tasks. In the training process, the convolutional layers in the backbone are frozen to avoid repeated training and reduce the amount of calculation, and the features learned on a large-scale dataset are used to improve the overall training efficiency of the network. At the same time, the neck part as a fixed feature fusion layer does not need to be trained in this method. Therefore, the time and computing resources required for network training can be greatly reduced.

[0053] Further, the output head includes at least a first output head and a second output head; wherein the first output head identifies the spatial position of the defect and generates a mask area, and the second output head classifies the defect and outputs the probability distribution result of the category. In detail, after completing the preprocessing, the key frame image is input into an offline trained deep network for inference analysis. The network includes two output heads (Head), namely the first output head and the second output head, which are responsible for different tasks, wherein: the first output head is used to detect the position of the defect, and outputs the mask image of the defect area, which can accurately mark the specific position of the possible defect in the pipeline; the second output head is used for category classification of the defect, and outputs the probability distribution of the category to which the defect belongs. Through the design of the double-head network structure, spatial position information and semantic classification information can be obtained at the same time; the inference process of the deep network is carried out in real time on the edge device using hardware acceleration environment, which can quickly process large-scale key frame data, and the output result is directly used for further intelligent judgment.

[0054] Further, the analysis result is screened and stored based on the deep network, at least including:

[0055] The mask and category probability of the defect area are comprehensively analyzed, wherein:

[0056] When the classification probability of any frame reaches the preset threshold, the frame is judged to be valid, contains valid defect information, and is marked as critical data; when the classification probability of any frame does not reach the preset threshold, the frame is judged to be invalid, contains data redundancy, and is discarded.

[0057] In detail, after the inference is completed, the output results of the deep network are used to continue intelligent screening, that is, a comprehensive analysis of the mask and category probability of the defect area is performed. Specifically, a comprehensive analysis of the mask and category probability of the defect area is performed, wherein a dual screening condition is set based on the mask area area and category probability: First, the pixel ratio S of the defect mask area is calculated. If S is lower than the set threshold S min , then the defect area of ​​the frame is considered too small, and it is judged as an invalid frame and discarded; secondly, the classification probability P of the defect category is calculated. If P is lower than the preset threshold P min , the frame recognition is considered uncertain, and it is determined to be an invalid frame and discarded; only when the S and P of a frame meet the respective threshold requirements at the same time, the frame is determined to be a valid frame, and its original image, defect mask, defect category and probability distribution are extracted, marked as key data and stored; In addition, in order to adapt to the characteristic distribution of different types of defects, the threshold S min and P min It can be dynamically adjusted based on statistical information or human feedback to optimize recognition accuracy and improve data utilization.

[0058] Furthermore, for valid frames, the corresponding original image, defect area mask, defect category, and probability distribution are extracted and integrated into structured data. Specifically, for frames determined to be valid, the corresponding original image, defect area mask, defect category, and probability distribution are extracted and integrated into structured data. The screening process employed by this method not only ensures the high quality and value of the stored data, but also significantly reduces the burden of irrelevant data on storage and subsequent analysis.

[0059] Furthermore, fine-grained annotation is performed on the filtered and stored data to obtain the data to be used, which at least includes:

[0060] After completing data screening and storage, in the annotation interface, manual annotation is performed on the generated mask area and category probability based on the target requirements to obtain the data to be used.

[0061] Specifically, after intelligent screening and data storage, the selected data is pushed to the manual fine-grained annotation module. Specifically, within the annotation interface, the operator can clearly see the system-generated masked regions and category probabilities, and confirm or adjust these regions and categories as needed. Manual annotation results further improve data accuracy, providing high-quality training data for model optimization. Furthermore, the annotation interface provides intuitive operating tools and user-friendly interactive design, ensuring that operators can efficiently complete annotation tasks.

[0062] Finally, the preliminarily processed data and defect images with mask information will be packaged and exported for subsequent users.

[0063] Another aspect of the present application provides a pipeline defect detection and analysis system integrating labeling and recognition, the system comprising at least a processor and further comprising:

[0064] An analysis module is used to perform analysis on the inner wall information of the pre-processed pipeline based on a deep network to obtain analysis results;

[0065] The screening module is used to screen and store the analysis results based on the deep network;

[0066] The annotation module is used to perform refined annotation on the filtered and stored data to obtain the data to be used;

[0067] The encapsulation module is used to perform encapsulation export on the data to be used.

[0068] It is worth mentioning that

[0069] The system is divided into one or more modules, one or more modules are stored in the memory and executed by the processor to complete the present application. One or more modules can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program. For example, the computer program can be divided into an analysis module, a screening module, a labeling module, and a packaging module. The specific functions of each module are as follows: the analysis module is used to perform analysis on the pre-processed pipeline inner wall information based on the deep network to obtain analysis results; the screening module is used to perform screening and storage on the analysis results based on the deep network; the labeling module is used to perform refined labeling on the screened and stored data to obtain stand-by data; the packaging module is used to perform packaging and export on the stand-by data. The system provided in this embodiment is specifically a pipeline defect visual acquisition system that integrates labeling and intelligent recognition. Traditional pipeline detection and analysis systems in the prior art usually need to store a large amount of high-definition image data in airborne equipment, which is then exported and processed for recognition. In contrast, the system provided in this embodiment fully utilizes the computing power of edge devices, performs preliminary pre-processing of high-definition images on edge devices, and retains only key frames suitable for back-end analysis, significantly reducing the data explosion problem caused by massive data. Moreover, during the edge device processing, hardware acceleration capabilities are used to quickly predict and roughly mark, greatly improving the efficiency and accuracy of subsequent marking, thereby not only improving work efficiency, but also reducing the workload of manual marking, and ensuring that the data collection link has certain defect detection and analysis capabilities.

[0070] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A pipeline defect detection and analysis method integrating labeling and recognition, the method being executed by a processor, characterized in that: Perform analysis on the inner wall information of the pre-processed pipeline based on the deep network to obtain analysis results; Filter and store the analysis results based on the deep network; Perform refined annotation on the filtered and stored data to obtain the data to be used; Execute encapsulation export for the data to be used; The pre-processed pipeline inner wall information is at least a pipeline inner wall video that is collected by a camera and transmitted to an edge device for processing; The deep network at least includes a backbone network, a feature fusion module and an output head; The output head includes at least a first output head and a second output head; wherein the first output head identifies the spatial location of the defect and generates a mask area, and the second output head classifies the defect and outputs a probability distribution result of the category; The screening and storage of the analysis results based on the deep network at least includes: Perform a comprehensive analysis of the defect area mask and class probability, where: A double screening condition is set based on the mask area and the category probability: First, the pixel ratio S of the defect mask area of ​​any frame is calculated. If S is lower than the set threshold S min , then the defect area of ​​the frame is considered too small, and it is judged as an invalid frame and discarded; secondly, the classification probability P of the defect category is calculated. If P is lower than the preset threshold P min , the frame recognition is considered uncertain, and it is determined to be an invalid frame and discarded; only when the S and P of a frame meet the respective threshold requirements at the same time, the frame is determined to be a valid frame, and its original image, defect mask, defect category and probability distribution are extracted, marked as key data and stored; the threshold S min and P min Dynamic adjustment based on statistical information or manual feedback; Extract the corresponding original image, defect area mask, defect category and probability distribution of the valid frame and integrate them into structured data; The performing of refined annotation on the screened and stored data to obtain the ready-to-use data at least includes: After completing data screening and storage, in the annotation interface, manual annotation is performed on the generated mask area and category probability based on the target requirements to obtain the data to be used.

2. The method according to claim 1, characterized in that The preprocessing includes at least denoising, image enhancement and key frame extraction.

3. A pipeline defect detection and analysis system for integrating labeling and identification according to any one of claims 1 to 2, the system comprising at least a processor, characterized in that: include: An analysis module, configured to perform analysis on the inner wall information of the pre-processed pipeline based on a deep network to obtain analysis results; A screening module, configured to screen and store the analysis results based on a deep network; A labeling module, which is used to perform refined labeling on the screened and stored data to obtain data to be used; The encapsulation module is used to perform encapsulation export on the data to be used.

Images