Pipeline detection robot defect identification method based on image processing

Abstract

The invention relates to a pipeline detection robot defect identification method based on image processing, which comprises the following steps: carrying out continuous image acquisition on the inner wall of a pipeline through image acquisition equipment on a pipeline robot to obtain image data of the inner wall of the pipeline; performing image preprocessing on the pipeline inner wall image data; pipeline defects in the pipeline inner wall image data after image preprocessing are detected, and image segmentation is carried out on a defect area; and carrying out feature extraction on the defect area, carrying out defect classification, and outputting an identification classification result. According to the invention, based on algorithms of image processing, artificial intelligence and machine learning, defects of cracks, corrosion, leakage and the like in the pipeline can be automatically identified, the limitation of manual detection is solved, automatic and intelligent detection is realized, the technical effects are mainly embodied in the aspects of improving the detection precision and efficiency and the ability to adapt to different pipeline environments, and the method is suitable for popularization and application. High application and popularization values are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of visual recognition, and in particular to a pipeline inspection robot defect recognition method based on image processing. Background Art

[0002] With the continuous development of science and technology, robotics technology has penetrated into various fields. Among them, pipeline robots, as a robot that can operate in a small space, have been widely used. Pipeline inspection robots are an artificial intelligence application that has emerged in recent years. Using the flexibility and controllability of robots, they can successfully enter various pipelines to inspect and repair the internal conditions of the pipelines. Pipeline inspection robots can not only improve the efficiency of pipeline inspection and maintenance and shorten working hours, but also provide personnel with a safer and more innovative way of working. The practicality and superiority of this technology have been widely used and recognized in the pipeline industry. First of all, pipeline inspection robots can be applied to the inspection of various pipelines, including drainage pipes, rainwater pipes, sewage pipes, industrial pipelines, etc., and there are no restrictions on the diameter, material, and degree of curvature of the pipeline, and the adaptability is very high. Due to its unique structure and functional modules, it can accurately detect and identify hidden dangers and problems inside various pipelines, greatly improving the efficiency and safety of pipeline facilities. Secondly, the major advantage of the practicality of pipeline inspection robots is that they significantly improve the safety and efficiency of pipeline inspection and troubleshooting. Pipeline inspection robots can replace manual labor to perform pipeline inspection and troubleshooting, eliminating the need for human-to-human collaboration, reducing workers' exposure to dangerous situations such as gas poisoning and explosions, and ensuring workers' safety. At the same time, inspection robots can bring a higher level of inspection efficiency, reduce labor costs, and improve work efficiency.

[0003] As the service life of pipeline systems increases, pipeline corrosion, cracks, blockages and leaks become more and more common, requiring efficient and accurate detection methods to prevent significant losses caused by pipeline failure. Traditional detection of pipeline inner wall defects includes the following methods: manual inspection, manual entry into the pipeline for visual inspection, which is not only inefficient, but also poses a threat to the safety of operators, especially in toxic gas, low oxygen or narrow pipeline environments; static monitoring equipment, using static cameras, probes and other equipment can only be fixed in a certain position for monitoring, which is difficult to cover the entire area of ​​the pipeline, especially for long-distance pipelines. The detection efficiency is low; traditional detection equipment, such as pressure sensors or flow sensors, can only monitor outside the pipeline and cannot effectively detect corrosion, cracks, scaling, etc. inside the pipeline. In summary, in order to cope with the shortcomings of traditional detection methods, how to combine pipeline detection robot technology to realize intelligent and automatic identification of pipeline defects has become a problem that technicians in this field need to solve urgently. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art and provide a pipeline inspection robot defect recognition method based on image processing. Based on image processing, artificial intelligence and machine learning algorithms, it can automatically identify defects such as cracks, corrosion, and leakage in pipelines, solve the limitations of manual detection, and realize automated and intelligent detection.

[0005] The present invention is achieved through the following technical solutions:

[0006] A pipeline inspection robot defect recognition method based on image processing comprises the following steps:

[0007] S1. Continuously capture images of the inner wall of the pipeline through the image acquisition device on the pipeline robot to obtain image data of the inner wall of the pipeline;

[0008] S2. performing image preprocessing on the pipeline inner wall image data;

[0009] S3. Detecting pipeline defects in the pipeline inner wall image data after image preprocessing, and performing image segmentation on the defective area;

[0010] S4. Extract features from the defect area, classify the defects, and output the identification and classification results.

[0011] According to the above technical solution, preferably, the image acquisition device includes one or more of a high-definition camera or a multi-modal camera device, and the multi-modal camera device includes one or more of visible light, infrared, and ultrasonic imaging devices.

[0012] According to the above technical solution, preferably, the image preprocessing includes one or more of denoising, contrast enhancement, grayscale transformation, and edge enhancement.

[0013] According to the above technical solution, preferably, step S3 includes: using an edge detection algorithm to identify the defect edge on the pipeline surface; eliminating noise and filling the defect area through morphological operations, wherein the morphological operations include one or more of dilation, erosion, opening operation, and closing operation; using Otsu threshold method, adaptive threshold segmentation, or semantic segmentation through the U-Net model in deep learning to perform image segmentation on the defect area.

[0014] According to the above technical solution, preferably, step S4 includes: using SIFT or HOG algorithm to extract features of the defect area; using support vector machine, random forest classifier, or pre-trained convolutional neural network ResNet to perform feature classification, and output recognition and classification results.

[0015] According to the above technical solution, preferably, in step S4, the defect classification includes one or more of cracks, corrosion, and blockage.

[0016] According to the above technical solution, preferably, after step S4, defect analysis is performed on the identification and classification results, and the defect analysis includes one or more of defect size measurement, position calibration, and degree evaluation.

[0017] According to the above technical solution, preferably, when measuring the defect size, the pixel distance in the image is used in combination with the calibration information of the image acquisition device to accurately calculate the defect size; when calibrating the position, the position sensor or odometer of the pipeline robot is combined to calibrate the specific position of the defect; when evaluating the degree, a rule base or expert system is used to perform evaluation based on the defect size and the specific position of the defect.

[0018] The beneficial effects of the present invention are:

[0019] The present invention is based on algorithms of image processing, artificial intelligence and machine learning, which can automatically identify defects such as cracks, corrosion, and leakage in pipelines, solve the limitations of manual detection, and realize automated and intelligent detection. The technical effect is mainly reflected in improving detection accuracy, efficiency, and the ability to adapt to different pipeline environments. The specific advantages are as follows:

[0020] (1) High degree of automation and intelligence

[0021] Reduce manual intervention: Traditional pipeline inspection relies on manual observation and judgment, while image processing technology combined with machine learning and deep learning can realize automatic defect identification. This greatly reduces the possibility of manual operation and errors, and improves efficiency and accuracy;

[0022] Real-time monitoring: Through automated image processing technology, the inspection robot can identify defects and provide feedback in real time, and operators can obtain results without interrupting the inspection process.

[0023] (2) High detection accuracy

[0024] Detail detection: Image processing algorithms (such as edge detection and segmentation algorithms) can capture small defects such as tiny cracks or slight corrosion, thereby improving the accuracy of detection;

[0025] High robustness: Deep learning models (such as convolutional neural networks) can automatically extract image features and adapt to complex pipeline environments, including different lighting conditions and pipeline materials, so they can identify various defects more accurately.

[0026] (3) Adapt to various types of defects

[0027] Diversified defect detection: Through image processing methods, the robot can not only detect surface defects such as cracks and corrosion, but also identify different types of pipeline problems such as blockage, deformation, and sedimentation;

[0028] Application in complex environments: Image processing combined with multimodal sensors can adapt to a variety of pipeline materials and complex environmental conditions (such as insufficient light, narrow space, etc.), thereby expanding its application scenarios.

[0029] (4) Real-time feedback and data visualization

[0030] Instant feedback: The image processing system can analyze and feedback defect information in real time, without waiting for the inspection to be completed before processing the data, which helps to make maintenance decisions quickly;

[0031] Visualization results: Through the visualization of images and data, operators can intuitively see the damage and location of the pipeline, making it easier to judge the severity of the problem.

[0032] (5) Improve detection efficiency

[0033] Fast processing: Image processing technology can quickly analyze large quantities of image data, which is faster than traditional manual inspection. Combined with intelligent algorithms, the system can quickly traverse the entire pipeline and identify possible defects, greatly shortening the inspection time;

[0034] Automatic report generation: Image processing algorithms can automatically generate reports including defect type, location and severity, eliminating the need for operators to record them manually.

[0035] (6) Strong scalability

[0036] Adaptability of deep learning models: By using large-scale data sets for training, deep learning models have good scalability, can self-adjust and optimize according to different detection tasks and environments, and have the ability to learn and improve over the long term;

[0037] Combination with other technologies: Image processing methods can be combined with other detection technologies (such as ultrasound and laser scanning) to form a multimodal detection system, thereby further improving detection accuracy and comprehensiveness.

[0038] (7) Reduce testing costs

[0039] Reduced labor costs: The automated system reduces the reliance on professionals, lowering the high cost and risk of manual inspections, especially in difficult-to-access or dangerous pipeline environments;

[0040] Reduced maintenance costs: Through more accurate and timely defect identification, early maintenance and repair of pipelines can avoid more serious damage and reduce long-term pipeline maintenance costs.

[0041] (8) Improved safety

[0042] Avoiding human intervention in dangerous environments: Pipeline inspection robots can enter dangerous working environments (such as high pressure, toxic environments or underwater pipelines) to replace manual inspections, thereby reducing the risks for inspectors;

[0043] Early warning: Through high-precision image processing technology, defects can be detected at an early stage, avoiding safety hazards caused by serious damage to the pipeline system.

[0044] (9) Long distance and complex structure detection

[0045] Adapt to long-distance pipelines: Defect recognition methods based on image processing can be applied in long-distance pipelines, especially during long-term inspections. Automated image processing can handle large amounts of data without being affected by human fatigue.

[0046] Complex structure adaptation: For structures with complex shapes or variable pipeline layouts, image processing technology combined with sensor fusion can better adapt to the changes in these structures and accurately detect hidden defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and the best embodiment. Based on the embodiments in the invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the invention.

[0049] As shown in the figure, the present invention comprises the following steps:

[0050] Step S1. Continuously capture images of the inner wall of the pipeline by using the image acquisition device on the pipeline robot to obtain image data of the inner wall of the pipeline.

[0051] Use a high-definition camera or multi-modal camera equipment (such as visible light, infrared, and ultrasonic imaging equipment) installed on the pipeline inspection robot to continuously capture images of the inner wall of the pipeline while the robot moves in the pipeline. The camera can be fixed on the front of the robot, or the angle can be flexibly adjusted through the robot arm to obtain all-round image data. After the robot enters the pipeline, it starts to move along the pipeline, and the camera captures images of the inner wall of the pipeline in real time. For long-distance pipelines, the image acquisition system may need to store data in segments or transmit data to the background through a wireless communication module.

[0052] Step S2: performing image preprocessing on the pipeline inner wall image data.

[0053] The collected images are subjected to basic preprocessing operations using Gaussian filtering + mean filtering, including denoising, contrast enhancement, grayscale transformation, edge detection, etc., in order to better highlight the characteristics of defects and reduce unnecessary interference during image processing. Specifically:

[0054] Denoising: Use Gaussian filtering + mean filtering to eliminate noise in the image and ensure the smoothness of the image.

[0055] Contrast Enhancement: Use histogram equalization or adaptive histogram equalization to enhance image contrast, making defects easier to detect.

[0056] Grayscale transformation: Convert color images into grayscale images to reduce the amount of calculation.

[0057] Edge enhancement: Use edge detection operators such as Sobel+Canny to enhance the edge features of the image. Edge detection is very important, especially for small defects such as cracks.

[0058] Step S3: Detect pipeline defects in the pipeline inner wall image data after image preprocessing, and perform image segmentation on the defective area.

[0059] Among them, edge detection algorithms such as Canny+Sobel are used to identify the defect edges on the pipeline surface (defects such as cracks and corrosion often appear as prominent edges); morphological operations such as dilation, corrosion, opening and closing operations are used to eliminate noise and fill the defect area to ensure the continuity of the detection results; the Otsu threshold method, adaptive threshold segmentation, or semantic segmentation is performed through the U-Net model in deep learning to segment the defect area into images.

[0060] Step S4. Extract features from the defective area, classify the defects (such as cracks, corrosion, blockage, etc.), and output the identification and classification results.

[0061] Among them, SIFT or HOG algorithm is used to extract features of the defect area. The preferred steps of extracting features using SIFT (Scale Invariant Feature Transform) algorithm are as follows:

[0062] 1) Convert color images to grayscale images to reduce computational complexity;

[0063] 2) Use Gaussian filtering and other methods to remove image noise and improve feature extraction effects;

[0064] 3) Use different scales (i.e. different Gaussian blur levels) to build an image pyramid;

[0065] 4) Calculate the difference between Gaussian images at adjacent scales to generate a difference-of-Gaussian (DoG) image. Find local extrema (i.e., points that are extremely large in both scale and space), and use these points as candidate key points;

[0066] 5) Accurately locate candidate key points and eliminate key points with low contrast and edge response;

[0067] 6) Use the Hessian matrix of the second-order derivative to evaluate the stability of the key points;

[0068] 7) Calculate the main direction for each key point. By calculating the gradient direction histogram of the area around the key point, select the largest direction as the direction of the key point;

[0069] 8) Extract a fixed-size region (usually 16x16 pixels) around the location of the key point;

[0070] 9) Divide this area into small sub-regions (e.g. 4x4), and calculate the histogram of gradient direction (8 directions) in each sub-region;

[0071] 10) Form a 128-dimensional feature descriptor, represented by a 128-dimensional vector, by connecting the histograms of all sub-regions.

[0072] The preferred steps for extracting features using the HOG (Histogram of Oriented Gradients) algorithm are as follows:

[0073] 1) Convert the input image into a grayscale image to reduce computational complexity;

[0074] 2) Use the Sobel operator to calculate the horizontal and vertical gradients of the image. The gradient can be expressed as:

[0075] G x =I(x+1,y)-I(x-1,y)(horizontal gradient)

[0076] G y =I(x, y+1)-I(x, y-1)(vertical gradient)

[0077] 3) Calculate the gradient magnitude and direction of each pixel;

[0078] Amplitude

[0079] direction

[0080] 4) Divide the image into small cells (8x8 pixels) and calculate the gradient direction histogram in each cell;

[0081] 5) For each cell, a histogram is constructed based on the pixel gradient direction and magnitude within the cell. The commonly used direction bucket number is 9 (for example, 0° to 180°), and the width of each bucket is 20°;

[0082] 6) Combine cells into larger blocks (e.g. 2x2 cells) and perform normalization. The purpose of normalization is to improve robustness to changes in illumination;

[0083] 7) The L2 norm can be used to normalize the histogram in the block to ensure the stability of the feature under different lighting conditions;

[0084] 8) Concatenate the normalized histograms of all blocks together to form the final HOG feature vector.

[0085] In addition, feature classification is performed using a support vector machine, a random forest classifier, or a pre-trained convolutional neural network ResNet. The preferred steps for feature classification using a support vector machine (SVM) classifier are as follows:

[0086] 1) Divide the dataset into training and testing sets (usually 80% training and 20% testing);

[0087] 2) Concatenate or weightedly fuse SIFT and HOG features to form a comprehensive feature vector;

[0088] 3) Standardize the features so that they have a mean of 0 and a variance of 1. This is particularly important for the SVM model because it is sensitive to the scale of the features;

[0089] 4) To handle missing values ​​in feature vectors, you can use mean filling, median filling or delete samples with missing values;

[0090] 5) Select the appropriate kernel function according to the data characteristics, for example: linear kernel, suitable for linearly separable data; Gaussian kernel, suitable for nonlinearly separable data; polynomial kernel, suitable for data with polynomial characteristics;

[0091] 6) Set the hyperparameters of SVM: penalty parameters and kernel function parameters (such as the γ value of the RBF kernel);

[0092] 7) Use the training set data to train the SVM model;

[0093] 8) Select the best hyperparameter combination through cross-validation to avoid overfitting;

[0094] 9) Apply the trained SVM model to new data for classification prediction.

[0095] The optimal steps for feature classification using the Random Forest classifier are as follows:

[0096] 1) Divide the dataset into training and testing sets (usually 80% training and 20% testing);

[0097] 2) Concatenate or weightedly fuse SIFT and HOG features to form a comprehensive feature vector;

[0098] 3) To handle missing values ​​in feature vectors, you can use mean filling, median filling or delete samples with missing values;

[0099] 4) Choose an appropriate number of trees (e.g., 100 trees) to balance model complexity and computational efficiency;

[0100] 5) Train the random forest model using the training set data. During the training process, each tree is trained from a random subset of the training data and split using randomly selected features;

[0101] 6) Use cross-validation to optimize and tune hyperparameters, such as tree depth, minimum number of sample splits per tree, etc.

[0102] 7) Apply the trained random forest model to new data for classification prediction.

[0103] The preferred steps for feature classification using a pre-trained convolutional neural network (CNN) ResNet are as follows:

[0104] 1) Divide the dataset into training set, validation set and test set (e.g. 70% training, 15% validation, 15% test);

[0105] 2) Resize the input image to the input size required by ResNet (256x256);

[0106] 3) According to the requirements of ResNet, the image pixel values ​​are normalized to the interval [0, 1] and standardized using the mean and standard deviation of the pre-trained model (such as the mean and standard deviation of ImageNet);

[0107] 4) Select ResNet152;

[0108] 5) Use weights pre-trained on large datasets such as ImageNet to obtain better feature extraction capabilities;

[0109] 6) Modify the last fully connected layer of ResNet to ensure that the number of nodes in the output layer is consistent with the target category;

[0110] 7) Select the cross entropy loss function, optimizer (such as Adam, SGD, etc.) and set the learning rate, batch size 64 and number of training rounds 50;

[0111] 8) Perform model optimization, including: hyperparameter tuning, using grid search or random search to further optimize hyperparameters (such as learning rate, batch size, etc.); data enhancement, using data enhancement techniques (such as rotation, flipping, scaling, etc.) during training to improve the generalization ability of the model;

[0112] 9) Use the training set to train the model and the validation set to monitor model performance to avoid overfitting;

[0113] 10) After each training cycle, save the best model weights based on the performance of the validation set;

[0114] 11) Use the test set to evaluate the performance of the trained ResNet model and calculate the following metrics:

[0115] (A) Accuracy: the proportion of correctly classified samples to the total samples;

[0116] (B) Precision, recall and F1-score: evaluate the performance of the model on different categories;

[0117] 12) Apply the trained ResNet model to new data for classification prediction.

[0118] Step S5. Perform defect analysis on the identification and classification results, wherein the defect analysis includes one or more of defect size measurement, position calibration, and degree assessment, and generate a complete defect report.

[0119] Defect size measurement: Use the pixel distance in the image and the camera calibration information to accurately calculate the size of the defect, such as the width of the crack, the area of ​​the corrosion area, etc.

[0120] Position calibration: Combined with the position sensor or odometer of the pipeline inspection robot, the specific location of the defect is calibrated to facilitate subsequent maintenance work. Accurate positioning can be achieved through sensor fusion (such as LiDAR and visual data fusion).

[0121] Severity assessment: Based on the size, location, shape and development trend of the defect, a rule base or expert system is used to automatically evaluate and give a severity level of the defect.

[0122] Step S6. The processed defect data is fed back to the control center in real time for operators to view and analyze. At the same time, the detection results are stored in the cloud or local database to facilitate subsequent tracking and analysis.

[0123] Real-time feedback: Through the wireless communication module, defect images, inspection reports and other data are transmitted to the remote monitoring system in real time, and the operator can remotely view the robot's inspection progress and defects found.

[0124] Data storage: Using cloud computing technology, a large amount of test data can be stored in the cloud for subsequent analysis. Big data analysis technology can be combined to perform trend analysis and prediction on historical data.

[0125] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A pipeline inspection robot defect recognition method based on image processing, characterized in that: The steps include: S1. Continuously capture images of the inner wall of the pipeline through the image acquisition device on the pipeline robot to obtain image data of the inner wall of the pipeline; S2. performing image preprocessing on the pipeline inner wall image data; S3. Detecting pipeline defects in the pipeline inner wall image data after image preprocessing, and performing image segmentation on the defective area; S4. Extract features from the defect area, classify the defects, and output the identification and classification results.

2. According to the image processing-based pipeline inspection robot defect recognition method of claim 1, it is characterized in that: The image acquisition device includes one or more of a high-definition camera or a multi-modal camera device. The multi-modal imaging device includes one or more of visible light, infrared, and ultrasonic imaging devices.

3. According to the image processing-based pipeline inspection robot defect recognition method of claim 1, it is characterized in that: The image preprocessing includes one or more of denoising, contrast enhancement, grayscale conversion, and edge enhancement.

4. According to the image processing-based pipeline inspection robot defect recognition method of claim 1, it is characterized in that: Step S3 includes: Using edge detection algorithm to identify defective edges on the pipeline surface; Eliminate noise and fill defective areas through morphological operations, wherein the morphological operations include one or more of dilation, erosion, opening operation, and closing operation; Use Otsu threshold method, adaptive threshold segmentation, or semantic segmentation through U-Net model in deep learning to segment the defect area into images.

5. The pipeline inspection robot defect recognition method based on image processing according to claim 1 or 4, characterized in that: Step S4 includes: Use SIFT or HOG algorithm to extract features of defect areas; Use support vector machine, random forest classifier, or pre-trained convolutional neural network ResNet to perform feature classification and output recognition and classification results.

6. According to the image processing-based pipeline inspection robot defect recognition method of claim 5, it is characterized in that: In step S4, the defect classification includes one or more of cracks, corrosion, and blockage.

7. According to the image processing-based pipeline inspection robot defect recognition method of claim 1, it is characterized in that: After step S4, defect analysis is performed on the identification and classification results, and the defect analysis includes one or more of defect size measurement, position calibration, and degree assessment.

8. According to claim 7, a pipeline inspection robot defect recognition method based on image processing is characterized in that: When measuring defect size, the pixel distance in the image is used in combination with the calibration information of the image acquisition device to accurately calculate the defect size; When calibrating the position, the position sensor or odometer of the pipeline robot is combined to calibrate the specific position of the defect; When evaluating the degree, a rule base or an expert system is used to perform the evaluation based on the defect size and the specific location of the defect.

Images