Pipeline defect detection method based on unmanned aerial vehicle spectrum and unmanned aerial vehicle

Abstract

The application provides a pipeline defect detection method based on unmanned aerial vehicle (UAV) spectrum and an unmanned aerial vehicle, belongs to the technical field of pipeline flaw detection, and is applied to the unmanned aerial vehicle. A spectrum camera capable of relative rotation is arranged on the unmanned aerial vehicle. The method comprises the following steps: flying the unmanned aerial vehicle along the pipeline to determine a pipeline section with defects, and controlling the unmanned aerial vehicle to fly above the pipeline section; extracting temperature values T1 and T2 in the edge regions on both sides of the pipeline section, adjusting the unmanned aerial vehicle to move to the side of the pipeline section with the larger value of T1 and T2, and determining a temperature value ΔT2 in the edge regions on both sides of the pipeline section at each adjustment position in the adjustment process; when ΔT2 is not greater than a preset temperature difference, collecting a target real image and a target hyperspectral image of the current pipeline section at the target adjustment position. The method can enable the staff to collect complete defect morphologies, facilitate the staff to find the specific positions of the defects, and make a repair plan for the pipeline in advance, thereby improving the efficiency of repairing the pipeline.

Description

Technical Field

[0001] This invention relates to the field of pipeline flaw detection technology, and in particular to a pipeline defect detection method based on UAV spectroscopy and the UAV. Background Technology

[0002] In projects involving the inspection of oil pipelines, especially long-distance oil and gas pipelines (within 400 km in length) and regional oil and gas field fault-block systems (with an area of ​​100 km²), 2 In terms of (left and right). These projects usually require "short, quick, and efficient" operation, with short inspection cycles, heavy workloads, and high quality requirements. Moreover, the terrain and landforms in the area are complex and varied, and pipelines often cross uninhabited areas such as deserts, Gobi, forests, and mountains. Surveyors and instruments cannot quickly reach the destination. Therefore, using drones for remote sensing monitoring of pipelines has become the fastest and most efficient inspection method.

[0003] Patent CN112116566A discloses a method for diagnosing pipeline defects, particularly on-road pipelines, based on hyperspectral remote sensing technology. This method utilizes hyperspectral remote sensing to input hyperspectral images of oil and gas pipelines generated by a spectral imager into a computer. Backpropagation (BP) neural networks and machine learning methods are used to label and train the experimentally obtained spectral data of the oil and gas pipelines, extracting hyperspectral images and spectral features of the pipelines to identify leaks and defects, thus determining the location of leaks and defects. Unsupervised machine learning entropy methods are used to classify the surface data of the oil and gas pipelines, and then BP neural networks are used to automatically classify the defects according to their severity, displaying warnings as "normal," "fault," or "alarm." This technology enables remote sensing monitoring of oil and gas pipeline leak and defect areas using unmanned aerial vehicles (UAVs).

[0004] However, pipeline defects vary in type and severity. Drones typically only take remote photos from top to bottom, and cannot accurately capture and photograph defects distributed in different locations on the pipeline. Therefore, ground personnel can only predict the severity of pipeline defects based on long-term work experience, and cannot accurately formulate a maintenance plan for the pipeline before personnel set out to carry out pipeline maintenance. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, this invention provides a method for detecting pipeline defects based on UAV spectroscopy and a UAV.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0007] The first aspect of this invention provides a method for detecting pipe defects based on the spectrum of a drone, applied to a drone, wherein the drone is equipped with at least one spectral camera for outputting both real and hyperspectral images, and the spectral camera is rotatable relative to the drone; the method includes:

[0008] The drone is used to follow the pipeline and obtain the first hyperspectral image of the pipeline surface through a spectral camera. Based on the analysis of the first hyperspectral image of the pipeline surface, the defective pipeline section is identified, and the drone is controlled to fly directly above the pipeline section.

[0009] A second hyperspectral image of the pipe section surface is acquired. Based on the second hyperspectral image, temperature values ​​T1 and T2 are extracted from the edge regions on both sides of the pipe section. The values ​​of T1 and T2 are compared. When the difference between T1 and T2, ΔT1, is greater than a preset temperature difference, the drone is moved towards the side of the pipe section with the larger value of T1 and T2. At each adjustment position during the adjustment process, the temperature value ΔT2 in the edge regions on both sides of the pipe section in the spectral image acquired by the spectral camera at that adjustment position is determined.

[0010] When ΔT2 is not greater than the preset temperature difference, determine the target adjustment position of the UAV.

[0011] Based on the target adjustment position of the UAV, the defect on the pipeline is located in the middle of the image range captured by the spectral camera. The target real image and target hyperspectral image of the current pipeline segment are then acquired at the target adjustment position.

[0012] Furthermore, the spectral camera is equipped with a laser rangefinder sensor, which measures the distance from the lens of the spectral camera to the pipe segment located in the middle of the image range captured by the spectral camera; adjusting the drone to move towards the pipe segment with the larger value between T1 and T2 includes:

[0013] The drone's spectral camera is positioned vertically downwards and acquires a second hyperspectral image of the pipe section surface. The distance D1 between the lens of the spectral camera on the drone and the middle of the pipe section in the image range acquired by the spectral camera is determined. The drone is then adjusted to fly parallel to the pipe section with radius D1 and center. The rotation angle of the spectral camera on the drone is adjusted so that the pipe section is within the image range captured by the spectral camera.

[0014] Furthermore, the rotation angle of the spectral camera on the drone is adjusted to keep the pipe section within the image range captured by the spectral camera, including:

[0015] Based on the third hyperspectral image acquired by the spectral camera, the region of the pipe segment in the third hyperspectral image is identified as the first active region, and the regions located on both sides of the pipe segment in the third hyperspectral image are determined as adjustment regions. When the first active region moves to overlap with one of the two adjustment regions, the spectral camera is adjusted to rotate towards the overlapping adjustment region.

[0016] Furthermore, a first preset correspondence is established between multiple overlapping areas and the rotation rate of the spectral camera; when the first active area moves to overlap with one of the two adjustment areas, the spectral camera is adjusted to rotate towards the overlapping adjustment area, including:

[0017] When the first active area moves to overlap with one of the two adjustment areas, the overlapping area S1 of the overlapping part is identified;

[0018] The target rotation speed is determined based on the correspondence between the overlapping area S1 and the first preset value.

[0019] Control the spectral camera to rotate at the target's rotation rate.

[0020] Furthermore, the methods also include:

[0021] When the rotation angle of the spectral camera relative to the drone reaches a preset angle threshold, the spectral camera is controlled to stop moving, and the real image of the pipeline section collected by the current spectral camera is determined as the target real image, and the hyperspectral image is determined as the target hyperspectral image.

[0022] Furthermore, the region of the pipe segment in the third hyperspectral image is identified as the first active region, including:

[0023] A first real image containing a pipe segment is acquired using a spectral camera. The acquired first real image is then analyzed using a pre-trained pipe image network to determine the region of the pipe image in the first real image.

[0024] By overlaying the first real image and the third hyperspectral image, the region of the pipe image in the first real image is projected into the third hyperspectral image as the first active region.

[0025] Furthermore, the methods also include:

[0026] Based on the target hyperspectral image, the temperature values ​​T3 and T4 in the edge regions on both sides of the pipe segment, and the temperature value T5 in the middle of the pipe segment are extracted. The values ​​of T5 and T3, and T5 and T4 are compared respectively. When the value of T5 is less than the values ​​of T3 and T4 at the same time, it is determined that the defect on the pipe surface is located on the back side of the pipe segment image acquired by the spectral camera.

[0027] A second aspect of the present invention provides a drone, which includes at least a drone body and a spectral camera. The drone body is provided with a control device, which includes at least a processor and a controller. The spectral camera is rotatably mounted on the bottom of the drone body. The spectral camera is electrically connected to the processor, and the controller is electrically connected to the processor. The output terminal of the controller is connected to the spectral camera, and the controller is used to drive the spectral camera to rotate.

[0028] The processor is used to execute a detection method as proposed in the first aspect of the embodiments of the present invention.

[0029] Furthermore, the controller includes a micro motor, a lead screw assembly, and a connecting rod. The lead screw assembly is horizontally mounted on the bottom wall of the drone. The micro motor is used to drive the lead screw in the lead screw assembly to rotate. One end of the connecting rod is hinged to the moving block in the lead screw assembly, and the other end is connected to the outer wall of the spectral camera. The outer wall of the spectral camera has a slide rail perpendicular to the hinge axis of the spectral camera. The end of the connecting rod away from the moving block slides along the length of the slide rail and can be rotatably connected within the slide rail.

[0030] The beneficial effects of this invention are as follows: after the drone detects a defect on the pipeline, it deflects towards the side of the pipeline with the defect. Once the spectral camera centers the defect on the pipeline segment within the acquisition area, the staff can collect the complete defect morphology, making it easier for them to find the specific location of the defect and formulate a repair plan for the pipeline in advance, thereby improving the efficiency of pipeline repair. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the structure of the spectral camera and controller according to Embodiment 1 of the present invention;

[0032] Figure 2 This is a schematic diagram of the control logic of the drone according to Embodiment 2 of the present invention;

[0033] Figure 3 This is a schematic flowchart of the detection method according to Embodiment 3 of the present invention.

[0034] The components include: 1. Spectral camera; 2. Miniature motor; 3. Lead screw assembly; 4. Connecting rod; 5. Laser rangefinder sensor; 6. Drone; 7. Processor; and 8. Controller. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Example 1

[0037] Reference Figure 1This application discloses a drone, including a drone body 6 and a spectral camera 1. The spectral camera 1 is rotatably mounted on the bottom of the drone body 6, with its rotation axis horizontal. A control device is provided within the drone body 6, including at least a processor 7 and a controller 8. Both the controller 8 and the spectral camera 1 are electrically connected to the processor 7. The processor 7 is used to execute a pipeline defect detection method based on the drone's spectrum. The controller 8 is used to control the rotation of the spectral camera 1 relative to the drone body 6.

[0038] The spectral camera 1 includes at least the functions of far-infrared spectral imaging and actual image capture. The spectral camera 1 outputs hyperspectral images through infrared spectral imaging and outputs real images through actual image capture. The controller 8 includes a micro motor 2, a lead screw assembly 3, and a connecting rod 4. The lead screw assembly 3 includes a lead screw and a moving block. The lead screw is horizontally positioned, with both ends rotatably connected to the drone body 6. The axis of the lead screw is perpendicular to the rotation axis of the spectral camera 1. The moving block is slidably connected to the bottom of the drone body 6 along the axis of the lead screw and is threadedly connected to the lead screw. A slide rail is formed on the side wall of the spectral camera 1 near the lead screw, perpendicular to its hinge axis. One end of the connecting rod 4 is hinged to the moving block, and the other end is connected to a movable head. The connecting rod 4 is slidably and rotatably connected to the slide rail on the spectral camera 1 through the movable head. The output shaft of the micro motor 2 is connected to one end of the lead screw and is used to drive the lead screw to rotate.

[0039] Furthermore, a gyroscope is also installed on the spectral camera 1, which can detect the rotation angle of the spectral camera 1 relative to the drone 6 while the drone 6 is flying in a horizontal attitude. A laser rangefinder 5 is also installed on the spectral camera 1, which can detect the distance between the spectral camera 1 and the object being photographed.

[0040] Example 2

[0041] Reference Figure 2 , 3 Based on the same inventive concept, another embodiment of this application provides a method for detecting pipe defects based on the spectrum of a drone, applied to a drone. The drone is equipped with at least one spectral camera for outputting both real and hyperspectral images, wherein the spectral camera is rotatable relative to the drone. The detection method mainly includes the following steps:

[0042] S01. Use a drone to follow the pipeline and collect a first hyperspectral image of the pipeline surface using a spectral camera. Analyze the collected first hyperspectral image of the pipeline surface to determine the pipeline segment with defects, and control the drone to fly directly above the defective pipeline segment.

[0043] In this scenario, the drone follows the pipeline while the spectral camera captures images vertically downwards. A pre-trained pipeline imagery network analyzes various pipeline images under different environments and terrains, identifying the target pipeline from real images captured by the spectral camera. This pipeline imagery network, in its experimental phase, utilizes neural network algorithms to train for pipeline detection under different environmental conditions captured in the images. By overlaying real images captured at the same time with the hyperspectral image, the location of the pipeline within the hyperspectral image can be determined.

[0044] By pre-setting temperature thresholds for the normal pipe and its surrounding area, when the processor or computer analyzes the hyperspectral image and finds a region where the temperature exceeds the pre-set temperature threshold, it determines that the pipe has a defect. Then, the aircraft is controlled to move horizontally, causing the highest temperature point in the detected defect patch to be positioned at the horizontal midline of the acquired image range.

[0045] S02. Acquire a second hyperspectral image of the pipe section surface. Based on the second hyperspectral image, extract the temperature values ​​T1 and T2 in the edge regions on both sides of the pipe section. Compare the values ​​of T1 and T2. When the difference between T1 and T2, ΔT1, is greater than a preset temperature difference, adjust the drone to move towards the side of the pipe section with the larger value of T1 and T2. At each adjustment position during the adjustment process, determine the temperature value ΔT2 in the edge regions on both sides of the pipe section in the spectral image acquired by the spectral camera at that adjustment position.

[0046] The difference between T1 and T2 is the absolute value. Adjusting the drone to move towards the pipe segment with the larger of T1 and T2 includes:

[0047] The drone's spectral camera points vertically downwards and captures a second hyperspectral image of the pipe section's surface. The distance D1 between the drone's spectral camera lens and the center of the image captured by the camera is determined. The drone is then adjusted to fly parallel to the pipe section with radius D1 and the pipe section as the center. The rotation angle of the spectral camera is adjusted to ensure the pipe section is within the camera's image range. By rotating the drone around the pipe section, the spectral camera maintains a consistent focal length, allowing for accurate and clear acquisition of images of the pipe section's side.

[0048] Each adjustment position during the adjustment process can be the location of the drone after it has flown a fixed distance to one side of the pipe section; the location of the drone after the spectral camera has rotated a fixed angle relative to the drone during flight; or the location of the drone after it has flown a fixed time to one side of the pipe section.

[0049] As the drone flies towards one side of the pipe section, it needs to remain within the effective acquisition area of ​​the spectral camera. Therefore, the spectral camera needs to rotate relative to the drone in real time. Adjusting the rotation angle of the spectral camera on the drone to keep the pipe section within the image range captured by the spectral camera includes:

[0050] Based on the third hyperspectral image acquired by the spectral camera, the region containing the pipe segment in the third hyperspectral image is identified as the first active region. Specifically, a first real image containing the pipe segment is acquired using the spectral camera. This first real image is analyzed using a pre-trained pipe image network to determine the region containing the pipe image within the first real image. By overlaying the first real image with the third hyperspectral image, the projection of the region containing the pipe image from the first real image onto the third hyperspectral image is determined as the first active region.

[0051] Next, the regions located on both sides of the pipe segment in the third hyperspectral image are identified as adjustment regions. These regions can be the areas in the image acquired by the spectrophotometer camera that are spaced apart from both sides of the pipe segment and parallel to it. When the first active region moves to overlap with one of the two adjustment regions, the spectrophotometer camera is adjusted to rotate towards the overlapping adjustment region. The spectrophotometer camera can be automatically deflected so that it can rotate to place the pipe segment within the effective area of ​​the image.

[0052] Furthermore, in order to adapt the rotation speed of the spectral camera to the flight speed of the drone and enable the spectral camera to align with the pipe section more quickly, a first preset correspondence between multiple overlapping areas and the rotation speed of the spectral camera is established in this embodiment. The overlapping area is the area of ​​the overlapping part when the first active area and the adjustment area overlap.

[0053] Specifically, when the first active area moves to overlap with one of the two adjustment areas, the overlapping area S1 is identified. Based on the overlapping area S1 and a first preset correspondence, the target rotation rate is determined, and the spectral camera is controlled to rotate at the target rotation rate. In the first preset correspondence, the overlapping area and the rotation rate can be inversely proportional. That is, when the drone flies at high speed and causes the first active area and the adjustment area to overlap rapidly, the spectral camera increases its rotation rate to quickly align the image back to normal.

[0054] Due to the influence of the actual mechanical structure and the actual environment, neither the drone nor the spectral camera can rotate indefinitely around the pipe segment. In this embodiment, when the rotation angle of the spectral camera relative to the drone reaches a preset angle threshold, the spectral camera is controlled to stop moving, and the real image of the pipe segment currently captured by the spectral camera is determined as the target real image, and the hyperspectral image is determined as the target hyperspectral image.

[0055] The default angle threshold is 45°, but in forest environments, the threshold can be manually set to 30° or below to ensure the drone can fly normally.

[0056] S03. When ΔT2 is not greater than the preset temperature difference, determine the target adjustment position of the UAV.

[0057] S04. Based on the target adjustment position of the UAV, determine that the defect on the pipeline is located in the middle of the image range acquired by the spectral camera, and acquire the target real image and target hyperspectral image of the current pipeline segment at the target adjustment position.

[0058] The collected real and highlight images of the target can be stored in the drone's internal storage device or remotely transmitted to staff computers or other platforms for remote viewing via wireless communication methods such as Bluetooth.

[0059] Based on the target hyperspectral image, the temperature values ​​T3 and T4 in the edge regions on both sides of the pipe segment, and the temperature value T5 in the middle of the pipe segment are extracted. The values ​​of T5 and T3, and T5 and T4 are compared respectively. When the value of T5 is less than the values ​​of T3 and T4 at the same time, it is determined that the defect on the pipe surface is located on the back side of the pipe segment image acquired by the spectral camera.

[0060] By having a drone fly directly to a point on a pipeline where a spectral camera is positioned to capture both real and hyperspectral images of the pipeline, remote workers can directly observe the shape and location of the defects on the pipeline from monitors, computers, and other devices. This allows workers to develop appropriate pipeline repair plans in advance based on the defect shape, improving the efficiency of pipeline repair.

[0061] Those skilled in the art will understand that although preferred embodiments of the invention have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims of the invention and their equivalents, the invention also intends to include these modifications and modifications.

Claims

1. A method for detecting pipeline defects based on UAV spectral analysis, characterized in that: This method is applied to drones, where the drone is equipped with at least one spectral camera for outputting both real-world and hyperspectral images. The spectral camera is equipped with a laser rangefinder sensor, which measures the distance from the lens of the spectral camera to a pipe segment located in the center of the image range captured by the spectral camera. The spectral camera can rotate relative to the drone. The method includes: The drone is used to follow the pipeline and obtain the first hyperspectral image of the pipeline surface through a spectral camera. Based on the analysis of the first hyperspectral image of the pipeline surface, the defective pipeline section is identified, and the drone is controlled to fly directly above the pipeline section. The second hyperspectral image of the pipe section surface is acquired. Based on the second hyperspectral image, the temperature values ​​T1 and T2 in the edge regions on both sides of the pipe section are extracted. The values ​​of T1 and T2 are compared. When the difference between T1 and T2 ΔT1 is greater than the preset temperature difference, the UAV is adjusted to move towards the side of the pipe section where the larger value of T1 and T2 is located. The drone's spectral camera points vertically downwards and captures a second hyperspectral image of the pipe section's surface. The distance D1 from the drone's spectral camera lens to the middle of the pipe section within the captured image range is determined. The drone is then adjusted to fly parallel to the pipe section with radius D1. A first real image containing the pipe section is captured using the spectral camera. This first real image is analyzed using a pre-trained pipe image network to determine the region of the pipe image within it. The first real image is then overlaid with a third hyperspectral image to determine the first active region projected onto the third hyperspectral image. A portion of the third hyperspectral image located on either side of the pipe section is identified as an adjustment region. Multiple overlapping areas are established in a first preset correspondence with the spectral camera's rotation rate. When the first active region overlaps with one of the two adjustment regions, the overlapping area S1 is identified. Based on the overlapping area S1 and the first preset correspondence, a target rotation rate is determined. The spectral camera is then controlled to rotate at the target rotation rate towards the overlapping adjustment region, ensuring the pipe section is within the image range captured by the spectral camera. At each adjustment position during the adjustment process, the temperature value ΔT2 is determined in the spectral image of the pipe segment acquired by the spectral camera at that adjustment position, within the edge region on both sides of the pipe segment. When ΔT2 is not greater than the preset temperature difference, determine the target adjustment position of the UAV. Based on the target adjustment position of the UAV, it is determined that the defect on the pipeline is located in the middle of the image range captured by the spectral camera. The target real image and target hyperspectral image of the current pipeline segment are collected at the target adjustment position. Based on the target hyperspectral image, the temperature values ​​T3 and T4 in the edge regions on both sides of the pipe segment, and the temperature value T5 in the middle of the pipe segment are extracted. The values ​​of T5 and T3, and T5 and T4 are compared respectively. When the value of T5 is less than the values ​​of T3 and T4 at the same time, it is determined that the defect on the pipe surface is located on the back side of the pipe segment image acquired by the spectral camera.

2. The method for detecting pipeline defects based on UAV spectral analysis according to claim 1, characterized in that, The method further includes: When the rotation angle of the spectral camera relative to the drone reaches a preset angle threshold, the spectral camera is controlled to stop moving, and the real image of the pipeline section collected by the current spectral camera is determined as the target real image and the hyperspectral image is determined as the target hyperspectral image.

3. A drone, characterized in that, The system includes at least a drone (6) body and a spectral camera (1). The drone (6) body is equipped with a control device, which includes at least a processor (7) and a controller (8). The spectral camera (1) is rotatably mounted on the bottom of the drone (6) body. The spectral camera (1) is electrically connected to the processor (7). The controller (8) is electrically connected to the processor (7). The output of the controller (8) is connected to the spectral camera (1). The controller (8) is used to drive the spectral camera (1) to rotate. The processor (7) is used to execute a detection method according to claim 1 or 2.

4. The unmanned aerial vehicle (UAV) according to claim 3, characterized in that: The controller (8) includes a micro motor (2), a lead screw assembly (3) and a connecting rod (4). The lead screw assembly (3) is horizontally mounted on the bottom wall of the drone (6). The micro motor (2) is used to drive the lead screw in the lead screw assembly (3) to rotate. One end of the connecting rod (4) is hinged to the moving block in the lead screw assembly (3), and the other end is connected to the outer wall of the spectral camera (1). The outer wall of the spectral camera (1) is provided with a slide rail perpendicular to the hinge axis of the spectral camera (1). The end of the connecting rod (4) away from the moving block slides along the length of the slide rail and can be rotatably connected in the slide rail.

Images