A device and method for detecting cracks in underground pipelines based on deep learning

Abstract

The present invention relates to the detection field in the underground pipeline construction industry, and in particular to an underground pipeline crack detection device and identification method based on deep learning; the device comprises a main shaft (1), a stepping motor (2), a ball screw (3), a spring support crank (4) and a track wheel (5); the stepping motor (2) is mounted on one end of the main shaft (1); the output end of the stepping motor (2) is connected to the spring support crank (4) through the ball screw (3); the spring support crank (4) is mounted with a track wheel (5); the main shaft (1) is also mounted with a sensor module (6); and a battery pack is built in the track wheel (5); the labor cost of pipeline detection can be effectively reduced, and a safe transmission environment for power and communication can be provided; pipeline cracks can be identified by a deep learning algorithm as a means of judging pipeline cracks, thereby effectively reducing the probability of manual misjudgment and missed judgment, and greatly improving the detection efficiency.

Description

Technical Field

[0001] The present invention relates to the detection field in the underground pipeline construction industry, and in particular to an underground pipeline crack detection device and identification method based on deep learning. Background Art

[0002] With the rapid development of modern urban construction, more and more cable projects such as power and communications are built underground. Cable engineering construction projects are an important part of urban infrastructure. Due to the continuous increase in urban underground development space and environmental corrosion, it is easy to damage power pipelines and cable pipelines, forming cracks, leading to the paralysis of power and communications. In order to ensure the normal supply of electricity and the stability of communications, it is of great significance to detect cracks in power cable pipelines. Since power pipelines and cable pipelines have small diameters and are deeply buried underground, they cannot be checked manually. Therefore, it is very important to use intelligent detection devices to enter the pipelines to complete the identification and detection of cracks.

[0003] Combining the current research status at home and abroad, the existing foreign Piperdrone team has developed a cable-controlled pipeline inspection micro robot, which is controlled by a PLC drive motor, motion wheels, scanning laser and camera. When the robot is running in the pipeline, it detects pipeline deformation by scanning laser, and the camera allows the surveyor to identify the basic situation in the pipeline with human eyes. The domestic Tsinghua University Robotics Research Group has developed a small pipeline video acquisition robot equipped with a CCD camera, using PWM speed regulation and RS-422 communication. The above methods all use manual methods to determine whether there are pipeline cracks in the video data collected by the camera, but often due to unclear images, angle deviations, and human judgment errors, the detection results are not ideal, resulting in false detections, missed detections, etc., and the longer the manual detection time, the greater the probability of misjudgment. Summary of the invention

[0004] The purpose of the present invention is to provide an underground pipeline crack detection device and recognition method based on deep learning; combined with the existing image processing algorithm, the detection device identifies pipeline cracks, reduces the rate of manual misjudgment, and improves the accuracy of judgment.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a deep learning-based underground pipeline crack detection device, comprising a main shaft, a stepper motor, a ball screw, a spring-supported crank and a track wheel, wherein the stepper motor is installed at one end of the main shaft, and the output end of the stepper motor is connected to the spring-supported crank through a ball screw, and a track wheel is installed on the spring-supported crank, and a sensor module is also installed on the main shaft, and a battery pack is built into the track wheel.

[0006] Preferably, there are three track wheels, each of which is driven by an independent motor. The track adopts a double-sided tooth track, and the track wheels are evenly distributed radially around the main shaft.

[0007] Preferably, the sensor module includes an IMU module, an AI camera, a binocular camera and an MCU core control board, the AI ​​camera is used for visual AI perception, the binocular camera is used for 3D imaging and depth information acquisition, the IMU module is used for detection of motion angle and acceleration, and the MCU core board is used for video algorithm and logic processing.

[0008] Preferably, a 6-axis IMU is installed in the main shaft, specifically including a 3-axis accelerometer and a 3-axis gyroscope.

[0009] To achieve the above-mentioned object, the present invention provides the following technical solutions: a method for identifying an underground pipeline crack detection device based on deep learning, comprising the following steps: step 1, creating a video stream input, and reading the video stream from an AI camera through a function provided by Opencv;

[0010] Step 2: Load the detection model and load the trained crack key point marking model into the MCU. The specific steps for building the detection model are as follows:

[0011] (1) Convert the input image into Tensor format, use HyperNet as the backbone, and obtain the feature map;

[0012] (2) The lightweight network realizes candidate region generation, defines the loss function, calculates the classification loss and bounding box regression loss and saves them. The loss function is as follows:

[0013]

[0014] p i Represents the probability that the i-th anchor is predicted to be the true label.

[0015] It is 1 when it is a positive sample and 0 when it is a negative sample.

[0016] t i Represents the bounding box regression parameters for predicting the i-th anchor.

[0017] Indicates the GTBox corresponding to the i-th anchor.

[0018] N cls Indicates the number of samples in a mini-batch.

[0019] N reg Indicates the number of anchor positions.

[0020] (3) Access the fully connected layer to complete the final classification and regression;

[0021] Step 3, score prediction is performed for each frame to obtain labels and coordinates;

[0022] Step 4: Draw a prediction box on the detected object;

[0023] Step 5, write the processed frames into the output video stream;

[0024] Step 6: When the prediction box is detected, the car stops moving and transmits the data to the human-computer interaction PAD for manual confirmation.

[0025] As a preferred method, the method comprises the following steps: For classification loss, binary cross entropy loss is used for calculation, and the formula is as follows:

[0026]

[0027] in is the bounding box regression loss, and the specific parameters of the loss function are as follows:

[0028]

[0029]

[0030]

[0031] t x =(xx a ) / w a , t y =(yy a ) / h a t w =log(w / w a ), t h =log(h / h a )(7)

[0032]

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1. The present invention provides a detection device for underground power cable pipelines, which effectively reduces the labor cost of pipeline detection and provides a safe transmission environment for power and communication.

[0035] 2. The present invention uses a deep learning algorithm to identify pipeline cracks as a means of determining pipeline cracks, which effectively reduces the probability of manual misjudgment and missed judgment and greatly improves detection efficiency.

[0036] Instruction Manual

[0037] Figure 1 The structure of the present invention is schematically shown in FIG. Figure 1 ;

[0038] Figure 2 The structure of the present invention is schematically shown in FIG. Figure 2 ;

[0039] Figure 3 It is the framework diagram of the electronic control principle;

[0040] Figure 4 It is a flow chart of the identification detection method;

[0041] Figure 5 This is a diagram of the HyperNet network structure;

[0042] Figure 6 This is the original image of Example 1;

[0043] Figure 7 is the image preprocessing diagram of instance 1;

[0044] Figure 8 This is the prediction graph of the algorithm in Example 1;

[0045] Fig. 9 is the actual output graph of Example 1;

[0046] Fig.10 This is the original image of Example 2;

[0047] Fig.11 This is the image preprocessing diagram of instance 2;

[0048] Fig.12 This is the prediction graph of the algorithm in Example 2;

[0049] Fig.13 This is the actual output diagram of Example 2.

[0050] Description of the drawings: 1. Spindle; 2. Stepper motor; 3. Ball screw; 4. Spring-supported crank; 5. Track wheel; 6. Sensor module; 7. Camera module; 8. AI camera. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] In the specific implementation of the present invention, the structure of the underground pipeline crack detection device based on deep learning is as follows: Figure 1 and 2 , which is composed of gear train, sensor, wheel arm and other parts.

[0053] Gear train and drive device, such as Figure 1 , which is composed of a crawler wheel device and a motion motor. The wheel system device is composed of three crawler wheels 5, each crawler wheel 5 is independently driven by a motor to move, and the crawler adopts a double-sided tooth crawler. The crawler drive and support components are evenly distributed radially around the main shaft 1, including a base plate, a support seat, a sleeve, a synchronous pulley and other parts; when the robot walks in the pipeline, the three crawler wheels 5 of the wheel system device are driven by the stepper motor 2 of the wheel arm device, and are opened to a certain angle, so that the crawler fits on the inner wall of the pipeline and presses tightly. At this time, friction occurs between the crawler wheel 5 and the inner wall of the pipeline, generating a friction force consistent with the walking direction of the robot, which is the driving force for the robot to walk. When climbing in a vertical pipeline, the stepper motor 2 can provide a sufficiently large driving force to overcome gravity and various resistances, so that the robot can smoothly rise or fall close to the inner pipe wall. The driving devices of the three crawler wheels 5 are driven by their own motors, and do not affect each other. By controlling the walking speed of each crawler wheel 5, the speed difference is used to realize the steering of the robot.

[0054] Wheel arm device, such as Figure 1 and 2 As shown, it is composed of a gear train connection device and is suitable for different pipe diameters. The wheel arm device is equipped with components such as a ball screw 3, a nut, and a pressure sensor. The stepper motor 2 drives the ball screw 3 to rotate through the coupling to drive the nut to move, and then pushes the connecting rod to change the support angle of the active crank and the driven crank to adjust the opening and closing angle of the track wheel 5, and change the outer diameter of the robot to adapt to pipes of different diameters. A pressure sensor is installed on the ball screw 3 of the wheel arm device of the pipeline robot, which can ensure that the positive pressure between the robot's track wheel 5 and the inner wall of the pipeline is always in a stable range, thereby ensuring that the pipeline robot has a stable traction output.

[0055] Sensors such as Figure 1 and 2 As shown in the figure, it is composed of IMU module, AI camera, binocular camera and MCU core control board. The AI ​​camera is used for visual AI perception, the binocular camera is used for 3D imaging and depth information collection, the IMU module is used for motion angle and acceleration detection, and the MCU core board is used for video algorithm and logic processing.

[0056] The spindle 1 is equipped with a 6-axis IMU (a 3-axis accelerometer and a 3-axis gyroscope) and a core control board. The output of the IMU includes the output of three different types of sensors, namely the gyroscope, magnetometer and accelerometer, which are used to calculate and report the exact force, angular rate and direction of the body. The accelerometer is used to record the robot's motion acceleration, which is then used to calculate the robot's real-time position and measure the robot's vibration and motion acceleration; it is mainly used in the electronic stability control system of the pipeline robot to measure the actual movement of the robot.

[0057] Gyroscopes are used to determine the orientation of the pipeline robot in three-dimensional space, measuring angular velocity around three axes, namely pitch (X-axis), roll (Y-axis) and yaw (Z-axis). When integrated with sensor fusion, they are used to maintain a reference orientation and maintain stability.

[0058] Magnetometers are used to measure the direction, strength, and relative changes of the magnetic field at a specific location.

[0059] The AI ​​camera uses the OpenMV Cam H7 camera. The OpenMV Cam H7 uses stm32 as the core and has multiple IO pins, including ADC, DAC, PWM, serial port and SPI interface. It uses the MT9M114 camera module, supports 60FPS 640x480 8-bit grayscale images or 16-bit RGB565 images, and supports up to 120FPS 320x240 image acquisition. The AI ​​camera can effectively identify cracks in the pipeline by using the recognition method provided by the present invention, and the output is controlled by the core control board. The binocular camera uses the Shunxin binocular camera, which consists of two high-definition cameras, a visible light camera and an infrared camera, and can calculate the distance between the pipeline robot and the pipeline crack.

[0060] The core control board includes a core MCU, external components, etc., which are used for data processing, motor control, signal transmission, etc. of the pipeline robot. The present invention adopts the STM32F103C8T6 single-chip microcomputer. The STM32 main control chip is a high-speed, low-power chip launched by STMicroelectronics with an ARM Cortex-M3 as the core processor. It plays the role of receiving external signals, sending control instructions, and outputting AI camera data in the entire system. When the single-chip microcomputer module receives the information transmitted by the wireless communication module, it will output the corresponding drive signal according to the type of information. The core control board uses the CAN bus to connect interfaces such as external storage, AI camera input, binocular camera input, IMU input, video output, and control output, which can stably and quickly realize data transmission and has strong anti-interference ability.

[0061] The battery is composed of a lithium battery pack and a power source. The battery pack is a lithium manganese oxide battery pack, model MSL-19B, which is modified by nano-coating process, has a stable structure, excellent performance, high specific capacity and good cycle stability, as well as high compaction density and machinability. As the electrical power source of the pipeline robot, it can provide sufficient power and service life.

[0062] The electronic control principle of the detection device, such as Figure 3 , including AI video and binocular video input and output, inertial measurement unit (IMU) input, bus port connection, MCU core control board, and external storage.

[0063] Crack detection method based on deep learning of the present invention

[0064] The MCU core control board of the detection device realizes the recognition of cracks. The recognition method is constructed as follows:

[0065] Step 1: Define the input function, create a video stream input, call the function provided by OpenVC, read the video stream collected by the AI ​​camera, and perform image preprocessing on a single frame. The preprocessing includes: first edge detection, retention filtering and denoising, then conversion to a grayscale image, binarization, and then morphological operations, corrosion and expansion.

[0066] Step 2: Load the trained crack key point marking model. The specific steps for building the crack detection model are as follows:

[0067] (1) Using HyperNet as the backbone to extract feature maps, such as Figure 5 As shown, it contains 49 convolutional layers and 1 fully connected layer. The network is divided into 5 stages, stage0, stage1, stage2, stage3, and stage4. Stage0 corresponds to conv1 in layername, which contains 64 7×7 convolution kernels (with a stride of 2), BatchNorm, activation function and kernel 3×3, and a maximum pooling layer with a stride of 2. The input of (224, 224, 3) obtains (56, 56, 64) as the final output feature map a.

[0068] (2) Generation of candidate boxes: On the feature map a extracted by Backbone, a 3×3 convolution is performed with Pading set to (1, 1). At each position, 9 anchors (20k) are generated. The out-of-bounds anchors are then removed. The remaining anchors (6k) are adjusted using the bounding box regression parameters (Formula 1). After the adjustment, non-maximum suppression is used to remove candidate boxes with high repetition rates. Finally, 2k candidate boxes are left. These 2k candidate boxes are then projected onto the feature map a to obtain the corresponding feature matrix.

[0069] (3) The feature matrix obtained in (2) is scaled to a feature map b of (7×7, 256) through the ROI pooling layer. Through full connection, classification prediction, and bounding box regression prediction, positive samples are defined as those whose IOU with GT is greater than 0.7, and negative samples are defined as those whose IOU with all GT is less than 0.3. Positive and negative samples are used to calculate the loss. The IOU calculation formula is: The classification loss function and bounding box regression loss function are as follows:

[0070]

[0071] The specific meanings of the parameters in the function are as follows:

[0072] p i Represents the probability that the i-th anchor is predicted to be the true label.

[0073] It is 1 when it is a positive sample and 0 when it is a negative sample.

[0074] t i Represents the bounding box regression parameters for predicting the i-th anchor.

[0075] Indicates the GTBox corresponding to the i-th anchor.

[0076] N cls Indicates the number of samples in a mini-batch.

[0077] N reg Indicates the number of anchor positions.

[0078] in For classification loss, binary cross entropy loss is used for calculation, and the formula is as follows:

[0079]

[0080] in is the bounding box regression loss, and the specific parameters of the loss function are as follows:

[0081]

[0082]

[0083]

[0084] t x =(xx a ) / w a , t y =(yya ) / h a t w =log(w / w a ), t h =log(h / h a )(7)

[0085]

[0086] Step 4: Call the draw prediction box function to draw the recognized object and the recognition box on the frame, using the OpenCV image processing toolkit.

[0087] Step 5, call the write function to write the processed frame to the output video stream.

[0088] Step 6: When a pipe crack is detected, the vehicle stops moving forward and returns the data to the human-machine PAD exchange interface.

[0089] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0090] Embodiment 1

[0091] like Figure 6 As shown in the figure, the detection device obtains a picture of a pipeline crack through the sensor. After edge detection, filtering and denoising, it is converted into a grayscale image and binarized. Then, through morphological operations, corrosion and expansion, the result is as follows Figure 7 As shown in the figure, the cracks are clearly visible. Then, after algorithm recognition, the prediction box is obtained in the binary image, as shown in Figure 8 The final output is to determine the pipeline cracks, such as Fig. 9 Due to the light collected, there are darker areas around the image, so the algorithm determines that there are cracks.

[0092] Embodiment 2

[0093] like Fig.10 As shown in the figure, the detection device collects an image of a small crack through the sensor. After edge detection, filtering and denoising, it is converted into a grayscale image and binarized. Then, through morphological operations, corrosion and expansion, the result is as follows Fig.11 As shown in the figure, the cracks are clearly visible. Then, after algorithm recognition, the prediction box is obtained in the binary image, as shown in Fig.12 The final output is to determine the pipeline cracks, such as Fig.13 Also due to the light collected, there are darker areas around the image, so the algorithm determines that there are cracks.

[0094] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A recognition method for an underground pipeline crack detection device based on deep learning, characterized in that: The detection device comprises a main shaft (1), a stepper motor (2), a ball screw (3), a spring support crank (4) and a track wheel (5), wherein the stepper motor (2) is mounted on one end of the main shaft (1), the output end of the stepper motor (2) is connected to the spring support crank (4) via the ball screw (3), the track wheel (5) is mounted on the spring support crank (4), a sensor module (6) is also mounted on the main shaft (1), and a battery pack is built into the track wheel (5); The sensor module (6) further comprises an IMU module, a camera module (7), an AI camera (8), a binocular camera and an MCU core control board, wherein the AI ​​camera is used for visual AI perception, the binocular camera is used for 3D imaging and depth information acquisition, the IMU module is used for detecting motion angle and acceleration, and the MCU core board is used for video algorithm and logic processing; The MCU core control board of the detection device realizes the recognition of cracks. The recognition method is constructed as follows: Step 1: Create a video stream input and read the video stream from the AI ​​camera through the VideoCapture() function provided by Opencv; Step 2: Load the detection model and load the trained crack key point marking model into the MCU. The specific steps for building the detection model are as follows: (1) Convert the input image into Tensor format, use HyperNet as the backbone, and obtain the feature map; (2) The lightweight network realizes candidate region generation, defines the loss function, calculates the classification loss and bounding box regression loss and saves them. The loss function is as follows: p i Indicates the probability that the i-th anchor is predicted to be the true label; When it is a positive sample, it is 1, and when it is a negative sample, it is 0; t i Represents the bounding box regression parameters for predicting the i-th anchor; Indicates the GTBox corresponding to the i-th anchor; N cls Indicates the number of all samples in a mini-batch; N reg Indicates the number of anchor positions; (3) Access the fully connected layer to complete the final classification and regression; Step 3, score prediction is performed for each frame to obtain labels and coordinates; Step 4: Draw a prediction box on the detected object; Step 5, write the processed frames into the output video stream; Step 6: When the prediction box is detected, the car stops moving and transmits the data to the human-computer interaction PAD for manual confirmation.

2. The identification method of the underground pipeline crack detection device based on deep learning according to claim 1 is characterized in that: The following steps are included: For classification loss, binary cross entropy loss is used for calculation, and the formula is as follows: in is the bounding box regression loss, and the specific parameters of the loss function are as follows: 。 3. The identification method of the underground pipeline crack detection device based on deep learning according to claim 1 is characterized in that: There are three track wheels (5), each track wheel (5) is driven by an independent motor, the track adopts a double-sided tooth track, and the track wheels (5) are evenly distributed radially around the main shaft (1).

4. The identification method of the underground pipeline crack detection device based on deep learning according to claim 1 is characterized in that: A 6-axis IMU is installed in the main shaft (1), specifically comprising a 3-axis accelerometer and a 3-axis gyroscope.

Images