Detection task-oriented enhanced sample feature adaptive fusion pipeline defect detection and intelligent report generation method
By enhancing the adaptive fusion of sample features and intelligent report generation methods, the problems of limited information and reliance on manual reporting in single-modal pipeline inspection are solved, achieving efficient and accurate pipeline defect detection and report generation, and adapting to complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU CHENGAN PIPE NETWORK TECHNOLOGY CO LTD
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-21
AI Technical Summary
Existing pipeline inspection methods rely on a single data modality, resulting in limited information, difficulty in providing a complete inspection perspective, challenges in feature fusion, poor accuracy and robustness of inspection results, and a lack of efficient reporting output mechanisms. Inspection reports often rely on manual analysis, which is prone to human error and low automation.
An enhanced sample feature adaptive fusion method is adopted, which removes image watermarks through generative adversarial networks. Combined with pre-trained models and manual annotation, data is filtered, normalized and preprocessed. A semi-human-machine autonomous learning optimization model is used to automatically generate standardized detection reports.
It achieves deep data fusion, improves detection accuracy and robustness, reduces manual operation, generates detailed detection reports, improves detection efficiency and automation level, and adapts to dynamic defect environments.
Smart Images
Figure CN121904449A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of industrial inspection, computer vision and artificial intelligence technology, and in particular relates to an enhanced sample feature adaptive fusion method for pipeline defect detection and intelligent report generation for inspection tasks. Background Technology
[0002] Current pipeline inspection methods often rely on single data modalities, such as CCTV video, sonar images, or sensor data. However, these single-modal data have significant limitations in defect detection, offering limited information and failing to provide a complete inspection perspective. Furthermore, feature fusion from different data sources is difficult, often resulting in insufficient information integration and consequently affecting the accuracy of the inspection results. The models also exhibit poor adaptability to complex pipeline defect scenarios, struggling to cope with varying environments and changing defect characteristics. Especially when facing constantly changing inspection conditions, detection accuracy and robustness can be significantly reduced.
[0003] On the other hand, current pipeline inspection technologies generally lack efficient and standardized reporting mechanisms. Inspection reports often rely on manual analysis, which not only reduces work efficiency but also makes them prone to human error. Furthermore, due to the low level of automation in traditional methods, the generated reports typically lack depth and consistency, making it difficult to provide accurate guidance for engineering decisions. Summary of the Invention
[0004] Purpose of the invention: The purpose of this invention is to provide a method for pipeline defect detection and intelligent report generation that is oriented towards detection tasks, and is based on the adaptive fusion of enhanced sample features. This method can improve the accuracy and efficiency of pipeline defect detection.
[0005] Technical solution: The present invention provides a method for adaptive fusion of enhanced sample features for pipeline defect detection and intelligent report generation, oriented towards detection tasks, comprising:
[0006] S1: Extract pipeline defect data from pipeline inspection reports, filter the data, and perform normalization and preprocessing operations to create a dataset;
[0007] S2: Based on the dataset, use an image inpainting method based on generative adversarial networks to remove the watermarks from the images in each dataset. Then, load a pre-trained object detection model to annotate the images, and manually correct the dataset based on the annotation results.
[0008] S3: Input the corrected dataset into the target detection model for fine-tuning, select a batch of pipeline inspection reports that have not been extracted, extract new data on pipeline defects, use the fine-tuned benchmark model to label the new data, and train the target detection model based on the labeling results and human feedback;
[0009] S4: Using the trained target detection model, perform defect detection and report generation on the target to be detected, obtain defect detection results and reports, and verify the validity of the defect detection results and reports and make corrections based on expert feedback.
[0010] Further, step S1 includes:
[0011] S11: By writing script code, data is extracted from several pre-acquired pipeline inspection reports to construct a preliminary dataset; the data includes images and corresponding text information;
[0012] S12: Manually screen the data in the preliminary dataset and use the pre-trained target detection model to assist in screening irrelevant or erroneous data; standardize the screened data; the manual screening specifically involves screening the data based on preset requirements, such as data category, accuracy, data quality, and scenario;
[0013] S13: Divide the initial dataset, after filtering and normalization, into several subsets, ensuring that each subset covers different defect types, defect severity, and scenario conditions;
[0014] S14: Preprocess the normalized data. Specifically, the preprocessing involves using image rotation, cropping, and flipping algorithms on the images in the data, and simultaneously using a denoising network based on a convolutional neural network to eliminate background noise in the images.
[0015] S15: Save the preprocessed data in a standardized JSON format to complete the dataset creation.
[0016] Furthermore, step S11 specifically includes: extracting text information using optical character recognition technology and extracting images using image processing algorithms.
[0017] Furthermore, the normalization process described in step S12 includes standardizing the format, size, and quality of the data.
[0018] Further, step S2 includes:
[0019] S21: Based on the dataset, use generative adversarial networks to remove watermarks from images in each dataset;
[0020] S22: Load the pre-trained target detection model to automatically annotate the watermark-free image according to the preset annotation specifications; the annotation content includes the defect location and defect category;
[0021] S23: Through manual review and correction of the annotation results by experts, data corresponding to images that do not conform to the annotation specifications are removed from the dataset, thus completing the fine-tuning of the dataset.
[0022] Further, step S3 includes:
[0023] S31: Input the corrected dataset into the object detection model for training;
[0024] S32: Select a batch of pipeline inspection reports that have not been extracted, extract new data on pipeline defects, and use the trained target detection model to automatically label the data to be inspected;
[0025] S33: Combine the annotation results generated by the object detection model with human feedback, periodically correct the annotation data, and feed the feedback back to the object detection model for training.
[0026] Furthermore, the target detection model described in step S32 employs a semi-human-machine autonomous learning sampling strategy during automatic annotation, utilizing the following uncertain sampling method:
[0027]
[0028] in, For the target detection model to test samples The uncertainty of prediction For conditional probability;
[0029] Select the sample with the maximum entropy:
[0030]
[0031] in, For the most uncertain sample selected, Samples used to find the maximum entropy.
[0032] Further, step S4 includes:
[0033] S41: Use a trained and fine-tuned model to detect defects in the target object and automatically generate defect detection results and reports;
[0034] S42: Compare the generated report with the report reviewed by experts to verify the accuracy of the test results and revise the report content accordingly.
[0035] Furthermore, step S4 also includes:
[0036] S43: Based on expert feedback, adjust the report generation strategy through the object detection model and continuously optimize the report content in subsequent detections.
[0037] Further, step S11 includes:
[0038] Each detection method acquires data corresponding to a modality. The following modality fusion algorithm is used for data from different modalities:
[0039]
[0040] in, It is the final fusion feature. It is the first Adaptive weights for each modal feature It is the first Modal features, The number of modes; where the weights are calculated using the following formula:
[0041]
[0042] in, It is the first A measure of the fitness or importance of a modal feature.
[0043] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: 1. Through adaptive feature fusion pipeline, the fusion weights of modal features can be dynamically adjusted according to defect type and data quality, achieving deep data fusion and improving detection accuracy and robustness; 2. This invention combines manual and automatic annotation, continuously optimizing the model through active learning, enabling the model to maintain high adaptability in dynamically changing defect environments; 3. It automatically generates standardized inspection reports, reducing manual operations, and the reports are detailed, providing important basis for subsequent maintenance and decision-making; 4. Through large-scale automated inspection and report generation, it greatly reduces labor costs and improves inspection efficiency; 5. The technical solution of this invention can effectively improve the accuracy, robustness, and automation level of pipeline defect detection, and has significant engineering application value and broad application prospects. Attached Figure Description
[0044] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;
[0045] Figure 2 This is a schematic diagram of a pipeline inspection report according to an embodiment of the present invention;
[0046] Figure 3 This is a schematic diagram of an image inpainting method based on Generative Adversarial Networks (GANs) according to an embodiment of the present invention;
[0047] Figure 4 This is a schematic diagram showing the before and after watermark removal in an embodiment of the present invention;
[0048] Figure 5 This is a schematic diagram of auxiliary annotation of the model after watermark removal in an embodiment of the present invention. Detailed Implementation
[0049] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.
[0050] As attached Figure 1 As shown in this embodiment, the enhanced sample feature adaptive fusion pipeline defect detection and intelligent report generation method for detection tasks includes:
[0051] S1: Extract data on pipeline defects from several pipeline inspection reports. Each data set includes images and corresponding text information. Filter the data and perform standardization and preprocessing operations to create a dataset.
[0052] S2: Based on the dataset, use an image inpainting method based on generative adversarial networks to remove watermarks from the images in each dataset. Then, load a pre-trained object detection model to annotate the images, and manually correct the dataset based on the annotation results.
[0053] S3: Select a batch of pipeline inspection reports that have not been extracted, extract new data on pipeline defects, input the corrected dataset into the target detection model for fine-tuning, use the fine-tuned benchmark model to label the new data, and train the target detection model based on the labeling results and human feedback;
[0054] S4: Using the trained target detection model, perform defect detection and report generation on the target to be detected, obtain defect detection results and reports, and verify the validity of the defect detection results and reports and make corrections based on expert feedback.
[0055] Specifically, in step S1, firstly, as Figure 2 As shown, key data, including images, labels, and defect levels, is automatically extracted from pipeline inspection reports using script code. This data is initially organized and converted into a structured format for subsequent processing. Next, the expert team rigorously screens this initial data, removing erroneous or irrelevant data and standardizing the remaining data to ensure consistency and high quality. The data is divided into multiple subsets according to different defect types and scenario conditions to further improve the model's generalization ability. To ensure data reliability, image enhancement techniques (such as rotation, flipping, and cropping) and denoising techniques are also employed to increase sample diversity and reduce background noise, ensuring the model performs well in more complex scenarios. This is achieved through the following sub-steps:
[0056] S11: Extract key information such as images, videos, sensor data, tags, and defect levels from pipeline inspection reports using script code to construct a preliminary dataset. Ensure that data from different modalities can be used synchronously; the following modality fusion algorithm is used for data from different modalities:
[0057]
[0058] in, It is the final fusion feature. It is the adaptive weight of each modal feature. Each modality feature is represented by a weight, calculated using the following formula:
[0059]
[0060] in, It is a measure of the fitness or importance of each modal feature. The weights are calculated based on fitness. This process provides standardized, structured data for subsequent processing.
[0061] S12: Manual data screening and correction to ensure reliable data quality. After expert review, irrelevant, erroneous, or incomplete data is removed. The goal at this stage is to standardize the data format, unify size and quality, and ensure the data provides effective input for model training and testing. Manual screening includes: 1. Confirming category coverage: including all categories of the target scene, with each category covering different angles / lighting / scales; 2. Confirming accurate annotation: no missing / multiple annotations / category errors, and standardized format; 3. Confirming sample quality: clear targets, occlusion ≤50%, no blur / overexposure / distortion; 4. Confirming scene matching: consistent with the deployment scene, with training / validation / test sets divided in a 7:2:1 ratio to prevent data leakage.
[0062] S13: The filtered and normalized data are divided according to defect type, defect degree and different scenario conditions to provide diverse training samples for model training.
[0063] S14: Preprocess the data, use image enhancement techniques (such as rotation, cropping, flipping, etc.) to expand the dataset, and use denoising techniques to remove background noise and optimize model training results.
[0064] S15: Save the preprocessed data in a standardized JSON format to ensure data consistency and facilitate subsequent model training and testing.
[0065] In step S2, during the data preparation stage, an image inpainting method based on Generative Adversarial Networks (GANs) is used to remove watermarks and other interfering elements from the images to ensure the purity of the training data. Based on the watermark-removed and preprocessed data, a trained benchmark object detection model is loaded for initial annotation. At this point, the automatic annotation results are reviewed and fine-tuned by experts to ensure the accuracy of the annotations, and the corrected data is fed back to the model, thus forming a closed loop of continuous optimization and learning. This process greatly improves the efficiency of annotation and the accuracy of the data.
[0066] S21: Use an image inpainting method based on Generative Adversarial Networks (GANs) to remove watermarks from the image, such as... Figure 3 As shown, removing interfering factors from the image reduces the interference of watermarks on model training and defect detection, ensuring data purity; a comparison before and after watermark removal is shown. Figure 4 As shown.
[0067] S22: Load the baseline target detection model and perform preliminary annotation on the watermark-free data, such as... Figure 5 As shown, the system automatically identifies and generates defect detection results. This stage provides initial annotations for subsequent manual fine-tuning.
[0068] S23: Experts manually review and correct the initial annotations generated by the benchmark model to ensure the accuracy of the annotations. They remove data corresponding to images that do not conform to the annotation specifications from the dataset and return the corrected data to the model to further optimize the model's annotation capabilities.
[0069] S3: After initial training of the baseline model, experts will manually fine-tune the data to help the model better adapt to the characteristics of different defects. The fine-tuned model will then be used to annotate new data, continuously optimizing its parameters using its self-learning capabilities. Simultaneously, human feedback and labeled data are continuously added to the model, forming a closed loop of active learning. Through this continuous feedback mechanism, the model continuously improves, enhancing detection accuracy and robustness. This is achieved through the following sub-steps.
[0070] S31: Input the manually corrected and enhanced dataset into the baseline target detection model for fine-tuning. At this point, the model will optimize its detection performance based on the latest labeled data, especially improving the accuracy of defect type and feature identification.
[0071] To facilitate user understanding and application, the fine-tuning system includes clearly explained textual descriptions of the parameters involved; these descriptions are as follows:
[0072]
[0073] S32: Select a batch of previously unextracted pipeline inspection reports, extract new data on pipeline defects, automatically label the new data using a fine-tuned model, and generate defect detection results. The fine-tuned model can continue to learn based on the new data and automatically adjust parameters to adapt to different types of defects and complex scenarios.
[0074] Maximum entropy sampling filters out samples that the model is "unsure" about in automatic annotation. The annotation quality of these samples is then optimized through manual review, iteratively improving the accuracy of automatic annotation and making the final annotation results more reliable. The semi-human-machine autonomous learning sampling strategy utilizes the following uncertain sampling method:
[0075]
[0076] in, For the target detection model to test samples Prediction uncertainty (entropy) For conditional probability;
[0077] Select the sample with the maximum entropy:
[0078]
[0079] in, For the most uncertain sample selected, Samples used to find the maximum entropy.
[0080] S33: Combine the annotation results generated by the model with human feedback, regularly correct the annotation data, form a feedback loop, and continuously optimize the model in this way to improve detection accuracy and robustness.
[0081] S4: After training and fine-tuning the model, pipeline defect detection tasks can be performed automatically. Whenever a pipeline defect is detected, it generates a detailed report including key information such as the defect's location, type, and severity. The generated report undergoes validity verification; experts review every detail to ensure the accuracy of the detection results. Based on expert feedback, the defect detection algorithm and report generation process are further optimized to continuously improve report quality. This is achieved through the following sub-steps:
[0082] S41: Uses a trained and fine-tuned model to detect defects in the target pipeline and automatically generates a detailed inspection report. The report will include information such as the location, type, and severity of the defects, and provide guidance for pipeline maintenance and decision-making.
[0083] The generated report format is as follows:
[0084] detect_report={
[0085] time_period : A,
[0086] coordinates : B,
[0087] defect_category : C,
[0088] defect_level : D,
[0089] confidence_level: E
[0090] }, where A is the time segment in the pipeline inspection video, B is the coordinate position of the detected defect in the video frame, in the format [x1, y1, x2, y2], and C represents the defect category, of which there are a total of 16 defect categories.
[0091] C={
[0092] Concealed branch pipe connection: 0; Deformation: 1; Deposition: 2; Misalignment: 3; Residual wall / dam root: 4; Foreign object insertion: 5; Corrosion: 6; Scum: 7; Scale: 8; Crack: 9; Undulation: 10; Tree root: 11; Leakage: 12; Disconnection: 13; Material detachment: 14; Obstacles: 15
[0093] }, D represents the pipeline defect level category E is the confidence level parameter, where .
[0094] The generation of the description report utilizes the following algorithm to optimize the objective:
[0095]
[0096] in, It is the first in the real report part, It is the corresponding part generated.
[0097] To ensure consistency between images and text when generating reports, this invention employs the following algorithm:
[0098]
[0099] This formula is used to ensure consistency between image and text content.
[0100] S42: Verify the validity of automatically generated test reports by comparing them with the results of manual review by experts to ensure the accuracy of the test results and assess the reliability of the automatically generated reports.
[0101] S43: Optimize the defect detection algorithm and report generation process based on expert feedback. Through a continuous feedback and correction mechanism, continuously improve the accuracy of the model and the quality of the generated reports.
[0102] Throughout the process, data annotation, model training, fine-tuning, and report generation are continuously cyclical. Each step relies on the output of the previous step, forming a complete feedback loop. Interaction between experts and this solution improves the accuracy and reliability of automated detection and report generation, enabling pipeline defect detection to maintain high efficiency and precision while minimizing manual intervention.
[0103] Through this progressively optimized implementation method, the present invention enables more efficient and accurate pipeline defect detection, significantly improves the level of automation in detection, and provides reliable support for pipeline maintenance and decision-making.
Claims
1. A method for adaptive fusion of enhanced sample features for pipeline defect detection and intelligent report generation, characterized in that, include: S1: Extract pipeline defect data from pipeline inspection reports, filter the data, and perform normalization and preprocessing operations to create a dataset; S2: Based on the dataset, use an image inpainting method based on generative adversarial networks to remove the watermarks from the images in each dataset. Then, load a pre-trained object detection model to annotate the images, and manually correct the dataset based on the annotation results. S3: Input the corrected dataset into the target detection model for fine-tuning, select a batch of pipeline inspection reports that have not been extracted, extract new data on pipeline defects, use the fine-tuned benchmark model to label the new data, and train the target detection model based on the labeling results and human feedback; S4: Using the trained target detection model, perform defect detection and report generation on the target to be detected, obtain defect detection results and reports, and verify the validity of the defect detection results and reports and make corrections based on expert feedback.
2. The method for enhanced sample feature adaptive fusion of pipeline defect detection and intelligent report generation for detection tasks according to claim 1, characterized in that, Step S1 includes: S11: By writing script code, data is extracted from several pre-acquired pipeline inspection reports to construct a preliminary dataset; the data includes images and corresponding text information; S12: Manually screen the data in the preliminary dataset and use the pre-trained target detection model to assist in screening irrelevant or erroneous data; standardize the screened data; the manual screening specifically involves screening the data based on preset requirements, such as category, accuracy, data quality, and scenario; S13: Divide the initial dataset, after filtering and normalization, into several subsets, ensuring that each subset covers different defect types, defect severity, and scenario conditions; S14: Preprocess the normalized data. Specifically, the preprocessing involves using image rotation, cropping, and flipping algorithms on the images in the data, and simultaneously using a denoising network based on a convolutional neural network to eliminate background noise in the images. S15: Save the preprocessed data in a standardized JSON format to complete the dataset creation.
3. The method for enhanced sample feature adaptive fusion of pipeline defect detection and intelligent report generation oriented towards detection tasks as described in claim 2, characterized in that, Step S11 specifically includes: extracting text information using optical character recognition technology and extracting images using image processing algorithms.
4. The method for enhanced sample feature adaptive fusion of pipeline defect detection and intelligent report generation for detection tasks according to claim 2, characterized in that, The normalization process described in step S12 includes standardizing the format, size, and quality of the data.
5. The method for adaptive fusion of enhanced sample features for pipeline defect detection and intelligent report generation according to claim 1, characterized in that, Step S2 includes: S21: Based on the dataset, use generative adversarial networks to remove watermarks from images in each dataset; S22: Load the pre-trained target detection model to automatically annotate the watermark-free image according to the preset annotation specifications; the annotation content includes the defect location and defect category; S23: Through manual review and correction of the annotation results by experts, data corresponding to images that do not conform to the annotation specifications are removed from the dataset, thus completing the fine-tuning of the dataset.
6. The method for enhanced sample feature adaptive fusion of pipeline defect detection and intelligent report generation for detection tasks according to claim 1, characterized in that, Step S3 includes: S31: Input the corrected dataset into the object detection model for training; S32: Select a batch of pipeline inspection reports that have not been extracted, extract new data on pipeline defects, and use the trained target detection model to automatically label the data to be inspected; S33: Combine the annotation results generated by the object detection model with human feedback, periodically correct the annotation data, and feed the feedback back to the object detection model for training.
7. The method for enhanced sample feature adaptive fusion of pipeline defect detection and intelligent report generation for detection tasks according to claim 6, characterized in that, The target detection model described in step S32 employs a semi-human-machine autonomous learning sampling strategy during automatic annotation, utilizing the following uncertain sampling method: ； in, For the target detection model to test samples The uncertainty of prediction For conditional probability; Select the sample with the maximum entropy: ； in, For the most uncertain sample selected, Samples used to find the maximum entropy.
8. The method for enhanced sample feature adaptive fusion of pipeline defect detection and intelligent report generation for detection tasks according to claim 1, characterized in that, Step S4 includes: S41: Use a trained and fine-tuned model to detect defects in the target object and automatically generate defect detection results and reports; S42: Compare the generated report with the report reviewed by experts to verify the accuracy of the test results and revise the report content accordingly.
9. The method for adaptive fusion of enhanced sample features for pipeline defect detection and intelligent report generation according to claim 8, characterized in that, Step S4 further includes: S43: Based on expert feedback, adjust the report generation strategy through the object detection model and continuously optimize the report content in subsequent detections.
10. The method for adaptive fusion of enhanced sample features for pipeline defect detection and intelligent report generation according to claim 2, characterized in that, Step S11 includes: Each type of detection method acquires data corresponding to a modality. The following modality fusion algorithm is used for data from different modalities: ； in, It is the final fusion feature. It is the first Adaptive weights for each modal feature It is the first Modal features, The number of modes; where the weights are calculated using the following formula: ； in, It is the first A measure of the fitness or importance of a modal feature.