A resident community gas pipe network leakage inspection system based on unmanned aerial vehicle group cooperation

Abstract

The application discloses a kind of based on unmanned aerial vehicle group cooperation's residential gas pipe network leakage inspection system, belong to unmanned aerial vehicle inspection field, including unmanned aerial vehicle group inspection unit, loading car processing unit, unmanned aerial vehicle take-off and landing platform;Unmanned aerial vehicle take-off and landing platform is used to realize the release and recovery of unmanned aerial vehicle;Loading car processing unit can plan the flight path of unmanned aerial vehicle group to generate flight trajectory, and output flight path signal to unmanned aerial vehicle inspection unit;Unmanned aerial vehicle inspection unit realizes accurate navigation, obstacle real-time avoidance, track correction after obstacle avoidance and monitoring of gas pipeline leakage by flight path;Loading car processing unit can also process received gas leakage data, realize the accurate positioning of gas leakage position, the unmanned aerial vehicle group cooperation's residential gas pipe network leakage inspection system provided by the application can realize the rapid and accurate inspection of residential gas pipe network, and accurately determine the specific position of gas leakage.

Description

Technical Field

[0001] This invention relates to the field of drone inspection, specifically a residential gas pipeline leak inspection system based on drone swarm collaboration. Background Technology

[0002] In recent years, with the rapid development and maturity of sensor technology, signal transmission, information interaction, and binocular vision, drones, equipped with various sensors and cameras, can accurately and reliably collect data and exchange information on inspection targets regardless of complex terrain. The application of drones in various inspection fields is becoming increasingly widespread, such as power grid inspection, gas pipeline inspection, oil and petrochemical inspection and patrol, railway inspection and patrol, and bridge patrol. Drone inspection is typically used in geographical areas where it is inconvenient for people or vehicles to inspect, thus overcoming terrain difficulties. Multi-drone collaborative applications refer to a new application method where a group system composed of intelligent individuals with independent control capabilities jointly completes tasks through information sharing, task allocation, and route planning.

[0003] Currently, drones and related technologies are maturing and widely used in various industries. Existing research on drone inspections largely focuses on power line inspections, with limited research on using drone swarms for gas pipeline inspections. Due to the limited flight time of drones, their maximum flight distance is very limited, resulting in low efficiency for inspecting large-scale gas pipeline networks. Furthermore, drone communication range is also limited. Therefore, there is an urgent need for a drone swarm collaborative inspection system, using a loading vehicle as the data processing center for information exchange. Through path planning and autonomous navigation, the system enables the drone swarm to autonomously and quickly inspect residential building gas pipeline networks, while accurately locating gas pipeline leaks. Summary of the Invention

[0004] To address the current problems of low efficiency in gas pipeline network inspections and the insufficient accuracy of drones in monitoring gas pipelines during inspections, this invention aims to provide a gas pipeline network leak inspection system for residential communities based on drone swarm collaboration. By rationally planning the inspection path of the drone swarm and accurately locating the specific location of gas pipeline leaks, the system improves the inspection efficiency of gas pipeline networks in residential communities.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A gas pipeline leak inspection system for residential communities based on drone swarm collaboration includes:

[0007] A drone launch and landing platform is used to launch and recover drones.

[0008] The loading vehicle processing unit is used to plan the flight path of the UAV swarm, generate flight trajectory, and output the flight trajectory signal to the UAV inspection unit. It is also used to process the received gas leak data to achieve accurate location of the gas leak.

[0009] The drone swarm inspection unit is used to achieve precise navigation through flight paths, real-time obstacle avoidance, trajectory correction after obstacle avoidance, and monitoring of gas pipeline leaks.

[0010] The loading vehicle processing unit includes:

[0011] The loading vehicle is used to carry the UAV take-off and landing platform and is placed at the UAV take-off and landing position according to the flight path of the UAV swarm, so as to realize the autonomous take-off and landing of the UAV.

[0012] The flight path planning module is used to generate flight paths for drone swarms.

[0013] The information interaction module is used to accurately locate the gas leak.

[0014] The drone swarm inspection unit includes:

[0015] The precision navigation module is used for precise navigation along the pipeline based on the flight path;

[0016] The swarm cooperative obstacle avoidance module is used to achieve real-time obstacle avoidance;

[0017] The trajectory correction module is used to correct the trajectory of the UAV.

[0018] Laser monitoring module is used for leak detection in pipelines;

[0019] The drone swarm inspection unit also includes a data transmission module, a signal receiving module, and a flight control module;

[0020] The information interaction module includes a data receiving module, a data processing module, and a signal transmission module.

[0021] As a further technical solution of the present invention, the flight path planning module is connected to the signal transmission module, and sends commands to the precision navigation module by outputting signals, so that the UAV swarm can accurately inspect the gas pipeline network along the flight path. The flight path planning module includes:

[0022] The baseline calculation submodule establishes a three-dimensional network model based on the geographic information of the area to be inspected, divides the key inspection grid, plans the path for the drone swarm to complete the inspection in the shortest time, and determines the appropriate number of inspection drones to be deployed in the community.

[0023] The adaptive control submodule plans routes based on endurance and key area grids, and sets recovery points according to the launch sequence and flight path, enabling UAVs to accurately inspect key grids.

[0024] As a further technical solution of the present invention, the precise navigation module includes:

[0025] The GPS navigation module receives commands from the flight path planning module to enable the UAV to perform inspections along the flight path.

[0026] The RTK / INS positioning module, connected to the GPS navigation module, is used to achieve precise positioning of the drone.

[0027] The RTK / INS positioning module includes:

[0028] The RTK positioning submodule is used to acquire precise location data for the drone;

[0029] The INS inertial navigation submodule contains a gyroscope and an accelerometer, which are used to acquire the attitude information of the UAV.

[0030] The RTK / INS filter processor is used to fuse the UAV pose data obtained by the INS inertial navigation submodule and the UAV position data obtained by the RTK positioning submodule to obtain high-precision position and pose information. The RTK / INS filter processor is connected to the flight control module to accurately control the UAV to inspect the gas pipeline without deviation along the flight path.

[0031] As a further technical solution of the present invention, the INS inertial navigation submodule processes the data obtained by the gyroscope and accelerometer to obtain velocity and attitude angle information, thereby obtaining the pose information of the UAV. By combining the UAV position information and the UAV pose information through the RTK / INS filter, the INS navigation parameters can be corrected while obtaining high-precision position information, and high-precision pose information can be obtained. The flight control module continuously adjusts the position and attitude of the UAV to achieve precise inspection of the gas pipeline by the UAV.

[0032] As a further technical solution of the present invention, the UAV is embedded with a swarm cooperative obstacle avoidance module, the swarm cooperative obstacle avoidance module comprising:

[0033] The ultrasonic obstacle detection module is used to emit ultrasonic waves during the flight of the drone and receive the reflected ultrasonic signals when an obstacle is detected.

[0034] A binocular vision precision measurement module is used to acquire image data through a binocular camera;

[0035] The obstacle avoidance processing module receives ultrasonic reflection signals collected by the ultrasonic obstacle perception module and image data collected by the binocular vision precision measurement module. It then processes the two types of data through data fusion and provides feedback control information to the flight obstacle avoidance controller through three-dimensional obstacle reconstruction.

[0036] As a further technical solution of the present invention, the swarm cooperative obstacle avoidance module uses a binocular vision precision measurement module as the main perception module and is supplemented by an ultrasonic obstacle perception module. The binocular vision precision measurement module includes a binocular camera and a binocular vision processor. The ultrasonic obstacle perception module includes an ultrasonic sensor and an ultrasonic processor. The binocular camera is directly connected to the binocular vision processor, and the ultrasonic sensor is directly connected to the ultrasonic processor. The binocular vision processor and the ultrasonic processor are connected to the fusion obstacle avoidance processing module. Automatic obstacle avoidance of the UAV is realized through the decision-making and data conversion of the obstacle avoidance fusion processing module.

[0037] As a further technical solution of the present invention, the UAV acquires image information through a binocular camera during flight and transmits the acquired information to a binocular vision processor in real time; the ultrasonic sensor receives the reflected ultrasonic signals and transmits the information to the ultrasonic processor to obtain ultrasonic data information.

[0038] As a further technical solution of the present invention, the obstacle avoidance processing module based on data fusion processes ultrasonic reflection signals and image data through an obstacle perception algorithm.

[0039] As a further technical solution of the present invention, the fusion obstacle avoidance processing module extracts obstacle feature information by fusing two types of data to achieve accurate matching of feature points. Based on the feature point information, it obtains the center position and external dimensions of the obstacle, and establishes a UAV and a geodetic coordinate system. Based on the geographical location information of the UAV itself, it transforms the coordinate position of the obstacle in the UAV to the position in the geodetic coordinate system through the transformation between the two coordinate systems.

[0040] As a further technical solution of the present invention, the obstacle is determined to be a movable obstacle by the flight speed of the UAV and the range of the position vector change between the obstacle and the UAV. For movable obstacles, a motion model of the movable obstacle is established by the position change of the obstacle at continuous time to predict the position change of the obstacle.

[0041] As a further technical solution of the present invention, a three-dimensional obstacle model is established using obstacle information obtained by binocular vision and ultrasound. When the UAV's flight path conflicts with the obstacle, control information is output to avoid the obstacle in advance. After avoiding the obstacle, the UAV can continue to inspect along the preset route by correcting its flight path.

[0042] As a further technical solution of the present invention, the flight obstacle avoidance controller receives feedback control information output by the fusion obstacle avoidance processing module, and then controls the UAV to fly around the obstacle through the flight control module, so as to realize the UAV's early perception and real-time avoidance of the obstacle.

[0043] As a further technical solution of the present invention, the trajectory correction module integrates the UAV's pose information obtained by the INS inertial navigation submodule and the image data collected by the binocular vision precision measurement module. It corrects the UAV's trajectory by the deviation of the obstacle avoidance position change, so that the UAV can continue to inspect the gas pipeline along the original path after obstacle avoidance.

[0044] As a further technical solution of the present invention, the UAV integrates the image data collected by the binocular vision precision measurement module into the INS inertial navigation submodule when correcting its flight path. If the UAV deviates in its flight attitude and position during navigation in order to avoid obstacles, the binocular vision precision measurement module uses the principle of left and right parallax of the human eye through the binocular camera to obtain the position change deviation based on the change of the coordinates of the object in the two lens images, and integrates it into the INS inertial navigation submodule to correct the UAV's motion attitude in a timely manner.

[0045] As a further technical solution of the present invention, the laser monitoring module includes:

[0046] Laser remote sensing detectors are used to collect emitted and reflected laser signals when drones are inspecting pipelines.

[0047] An image acquisition device, connected to a camera, acquires image information of the gas pipeline through the camera;

[0048] The gimbal is equipped with the laser remote sensing detector, camera, and image acquisition device. The gimbal is embedded in the side of the UAV and can rotate 180 degrees. The gimbal transmits the laser signal data collected by the laser remote sensing detector and the image information data collected by the image acquisition device to the data receiving module through the data transmission module.

[0049] As a further technical solution of the present invention, when the UAV inspects along the pipeline, the information interaction module receives laser signal data from the data transmission module to deduce the change in methane concentration. When a gas pipeline leak is detected, the camera and image acquisition device are activated and commands are issued to collect image information data of the gas pipeline and transmit the data to the data processing module. The leak location of the gas pipeline is marked using machine vision technology. The data processing module also has the functions of recording the concentration of leaked methane gas and capturing the leak scene.

[0050] As a further technical solution of the present invention, when the gas pipeline leaks, the laser remote sensing detector on the UAV will absorb some of the laser energy when it scans the CH4 gas cloud formed around the leak point. The concentration of CH4 can be determined based on the initial power and echo power of the laser. At the same time, the leak range of the gas pipeline is marked by the camera and machine vision. Based on the geographical location information of the UAV itself, the precise location of the gas leak can be accurately determined.

[0051] As a further technical solution of the present invention, the laser remote sensing detector and the camera are installed side by side on the UAV. The laser remote sensing detector is used to transmit and receive laser signals, and the camera can collect image information during gas pipeline inspection and observe the real-time status of the gas pipeline. The same gimbal is used to ensure that the laser light path and the camera light path are parallel. When the UAV performs automatic obstacle avoidance and turning actions, the gimbal can be adjusted to change the laser direction and inspect the gas pipeline.

[0052] As a further technical solution of the present invention, the flight control module receives control commands from the precision navigation module, the swarm cooperative obstacle avoidance module, and the trajectory correction module, and continuously adjusts the flight attitude to enable the UAV to perform inspections along the pipeline according to the planned route.

[0053] As a further technical solution of the present invention, the drone is used for inspection by taking off and landing in different locations. At the same time, based on the results of path planning and to ensure the orderly recovery of the drone, the number of drones to be launched, as well as the location of the take-off and landing point and the flight path of each drone are reasonably planned.

[0054] As a further technical solution of the present invention, redundancy is set on the basis of reasonably configuring the number of drones in the flight path planning module. When a drone fails to perform a task, the redundant drone is used to replace the original drone to continue to perform the inspection task, and the inspection is "relayed".

[0055] As a further technical solution of the present invention, the loading vehicle serves as a mobile base station to communicate with the drone. The loading vehicle receives and processes the data collected by the drone. During the drone inspection process, the drone landing platform is recovered. According to the landing time of the drone swarm, the landing platform is pre-positioned at the drone landing location to recover the drones in sequence.

[0056] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention designs a gas pipeline leak inspection system for residential communities based on UAV swarm collaboration. It integrates data information from binocular vision and ultrasound, and controls UAV obstacle avoidance flight through obstacle model reconstruction. It determines the inspection routes of each UAV swarm through flight path planning, and sets redundancy according to the maximum endurance of the UAVs. When a UAV cannot continue to perform the inspection task, a new UAV is dispatched to "relay" the inspection, ensuring the inspection progress. It uses RTK / INS technology for navigation and uses the results of binocular vision measurement to compensate for the error of INS navigation and correct the trajectory, realizing unbiased inspection of the pipeline by UAVs. It monitors changes in gas through laser remote sensing to determine whether a leak has occurred and marks the location of the gas leak through machine vision, realizing precise location of the leak. Attached Figure Description

[0057] Figure 1 This is an overall framework diagram of the collaborative inspection of unmanned aerial vehicle (UAV) swarms provided in an embodiment of the present invention.

[0058] Figure 2 A flowchart illustrating the precise flight process of a drone swarm provided in an embodiment of the present invention.

[0059] Figure 3 A flowchart for gas leak identification and location provided in an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0061] As one embodiment of the present invention, please refer to Figure 1 A residential gas pipeline leak inspection system based on drone swarm collaboration includes:

[0062] A drone launch and landing platform is used to launch and recover drones.

[0063] The loading vehicle processing unit is used to plan the flight path of the UAV swarm, generate flight trajectory, and output the flight trajectory signal to the UAV inspection unit. It is also used to process the received gas leak data to achieve accurate location of the gas leak.

[0064] The drone swarm inspection unit is used to achieve precise navigation through flight paths, real-time obstacle avoidance, trajectory correction after obstacle avoidance, and monitoring of gas pipeline leaks.

[0065] The loading vehicle processing unit includes:

[0066] The loading vehicle is used to carry the UAV take-off and landing platform and is placed at the UAV take-off and landing position according to the flight path of the UAV swarm, so as to realize the autonomous take-off and landing of the UAV.

[0067] The flight path planning module is used to generate flight paths for drone swarms.

[0068] The information interaction module is used to accurately locate the gas leak.

[0069] The drone swarm inspection unit includes:

[0070] The precision navigation module is used for precise navigation along the pipeline based on the flight path;

[0071] The swarm cooperative obstacle avoidance module is used to achieve real-time obstacle avoidance;

[0072] The trajectory correction module is used to correct the trajectory of the UAV.

[0073] Laser monitoring module is used for leak detection in pipelines;

[0074] The drone swarm inspection unit also includes a data transmission module, a signal receiving module, and a flight control module;

[0075] The information interaction module includes a data receiving module, a data processing module, and a signal transmission module.

[0076] As one embodiment of the present invention, please refer to Figure 2 The flight path planning module includes:

[0077] The baseline calculation submodule establishes a three-dimensional network model based on the geographic information of the area to be inspected, divides the key inspection grid, plans the path for the drone swarm to complete the inspection in the shortest time, and determines the appropriate number of inspection drones to be deployed in the community.

[0078] The adaptive control submodule plans routes based on endurance and key area grids, and sets recovery points according to the launch sequence and flight path, enabling UAVs to accurately inspect key grids. The flight path planning module is connected to the signal transmission module, and sends commands to the precision navigation module through output signals, enabling the UAV swarm to accurately inspect the gas pipeline network along the flight path.

[0079] The steps of the flight path planning module to achieve complete path inspection of the UAV swarm are as follows:

[0080] Step 1: Establish a coordinate system in the 3D network modeling layer using the baseline solution submodule. Input the x, y, and z coordinates of each node of the gas pipeline network in the residential area to be inspected into the modeling layer to establish nodes in sequence. Then connect the nodes according to the length and diameter of each pipeline segment to realize the 3D network modeling of the gas pipeline in the entire inspection area. Take the gas pipeline section entering the kitchen as the key area for drone inspection and monitoring, and input the coordinates of the four corners of the windowsill into the modeling layer to form a key inspection grid.

[0081] Step 2: Designate the ground-level end of the gas pipeline entering the residential building as both the launch and recovery point, ensuring the drones can be launched and recovered. That is, the bottom of the gas pipeline in each building can serve as both a launch and recovery point. Let the launch point be m. i Recycling point n j The first launch point is m1, and the first recovery point is n1. The number of drones is determined by matching the total length of the gas pipeline and the time required to inspect the key grid.

[0082] Step 3: Let v be the average speed of the drone during inspection, L be the maximum range of the drone, T be the inspection time required for each building, and nT be the inspection time required for inspecting n buildings. Each drone should ensure... To ensure the smooth recovery of the drone, after the drone has inspected multiple buildings, it needs to be retrieved at a designated recovery point (nk). This means the drone's endurance is insufficient to complete the inspection of the next building and return to the next recovery point (n). k+1 At that time, it is necessary to m k+1 Another drone was launched to carry out an inspection mission.

[0083] Step 4: When the drone inspects the kitchen inlet pipe section, it sends a command to the precision navigation module based on the adaptive control submodule to scan and inspect the kitchen inlet pipe section outdoors. After the inspection is completed, it continues to inspect along the pipe until the next household, repeating the previous operation until the inspection task is completed and it returns to the recovery point.

[0084] Step 5: By expanding upon Steps 3 and 4, the number of drones required for the inspection area can be determined. Drones are launched sequentially from the launch points in the inspection area using the drone landing platform, and then retrieved sequentially from the pre-set landing platform at the recovery point according to the launch order. This ensures that all pipelines and important target points can be inspected in the shortest possible time, while maintaining the maximum endurance of the drones.

[0085] In this embodiment of the invention, the precise navigation module includes:

[0086] The GPS navigation module receives commands from the flight path planning module to enable the UAV to perform inspections along the flight path.

[0087] The RTK / INS positioning module, connected to the GPS navigation module, is used to achieve precise positioning of the drone.

[0088] The RTK / INS positioning module includes:

[0089] The RTK positioning submodule is used to acquire precise location data for the drone;

[0090] The INS inertial navigation submodule contains a gyroscope and an accelerometer, which are used to acquire the attitude information of the UAV.

[0091] The RTK / INS filter processor is used to fuse the UAV pose data obtained by the INS inertial navigation submodule and the UAV position data obtained by the RTK positioning submodule to obtain high-precision position and pose information. The RTK / INS filter processor is connected to the flight control module to accurately control the UAV to inspect the gas pipeline without deviation along the flight path.

[0092] In this embodiment of the invention, the INS inertial navigation submodule processes the data acquired by the gyroscope and accelerometer to obtain velocity and attitude angle information, thereby obtaining the UAV's pose information. By combining the UAV's position information and pose information with the UAV's pose information through an RTK / INS filter, the INS navigation parameters can be corrected while obtaining high-precision position information, thus obtaining high-precision pose information. The flight control module continuously adjusts the UAV's position and attitude to achieve precise inspection of the gas pipeline by the UAV.

[0093] In this embodiment of the invention, the UAV is embedded with a swarm cooperative obstacle avoidance module, the swarm cooperative obstacle avoidance module comprising:

[0094] The ultrasonic obstacle detection module is used to emit ultrasonic waves during the flight of the drone and receive the reflected ultrasonic signals when an obstacle is detected.

[0095] A binocular vision precision measurement module is used to acquire image data through a binocular camera;

[0096] The obstacle avoidance processing module receives ultrasonic reflection signals collected by the ultrasonic obstacle perception module and image data collected by the binocular vision precision measurement module. It then processes the two types of data through data fusion and provides feedback control information to the flight obstacle avoidance controller through three-dimensional obstacle reconstruction.

[0097] In this embodiment of the invention, the swarm-cooperative obstacle avoidance module uses a binocular vision precision measurement module as the main perception module, supplemented by an ultrasonic obstacle perception module. The binocular vision precision measurement module includes a binocular camera and a binocular vision processor. The ultrasonic obstacle perception module includes an ultrasonic sensor and an ultrasonic processor. The binocular camera and the binocular vision processor are directly connected, and the ultrasonic sensor and the ultrasonic processor are also directly connected. The binocular vision processor and the ultrasonic processor are connected to a fusion obstacle avoidance processing module. Automatic obstacle avoidance by the UAV is achieved through decision-making and data conversion by the obstacle avoidance fusion processing module. The binocular camera is composed of two USB cameras and a CMOS image sensor. The binocular vision processor uses an Intel Computer Stick BOXSTK1AW32SCL integrated processor, supporting dual USB interfaces and capable of direct connection to the cameras. The ultrasonic sensor uses a KS103 ranging sensor, enabling high-precision ranging. The ultrasonic processor uses an STM32F407VGT chip for processing. Obstacle avoidance is achieved through the following steps:

[0098] Step 1: During flight, the UAV acquires image information through binocular vision and transmits the acquired information to the binocular vision processor in real time; the ultrasonic sensor receives the reflected ultrasonic signals and transmits the information to the ultrasonic processor to obtain ultrasonic data information.

[0099] Step 2: Based on the data fusion obstacle avoidance processing module, two types of data are processed through obstacle perception algorithms:

[0100] Perform data filtering:

[0101] Assume the system is linear, the measurement noise is Gaussian white noise, and the state equation is:

[0102] T(k+l)=F(k)T(k)+D(k)U(k)+C(k)V(k)

[0103] Where T(k+l) represents the state vector at time k+l, and F(k) represents the state vector at time km. The state transition matrix of m, D(k) represents m Let U(k) be an n-order input matrix, U(k) represent an n-dimensional input vector, and C(k) represent an m-dimensional input matrix. The n-order motivation matrix, V(k) represents n-dimensional perturbation Gaussian white noise.

[0104] The measurement equation is a linear function, which satisfies...

[0105] O(k) = H(k)T(k) + W(k)

[0106] Where O(k) represents the q-dimensional vector at time k, and H(k) represents q Let W(k) be an n-order measurement matrix, representing q-dimensional measurement noise, and satisfy the condition that Gaussian white noise has...

[0107] EW(k)=0;EW(k)W T (j)=R(k) kj

[0108] in kj It is the Dirichlet function.

[0109] Perform data fusion:

[0110] Assume V(k) and W(k) are independent and satisfy EW(k)W T (k)=0

[0111] Assume the filtered value of state T(k) at time k is known. From (k|k) and the covariance matrix P(k|k), we obtain the predicted state values:

[0112] (k+1|k)=F(k) (k|k)+D(k)U(k)

[0113] The fusion formula is:

[0114] (k)= (k-1)+Ev(k-1)

[0115] Y(k)=C (k)+Z(k)

[0116] Where T is the state vector. Let E be the state transition matrix, v be the target Gaussian white noise, v(k) follows a Gaussian distribution with variance w, Y be the observation vector, C be the observation matrix, and Z be the observation Gaussian white noise, Z(k) follows a Gaussian distribution with variance w.

[0117] Step 3: The obstacle avoidance processing module integrates two types of data by extracting obstacle feature information to achieve accurate matching of feature points. Based on the feature point information, it obtains the center position and external dimensions of the obstacle and establishes the UAV and the geodetic coordinate system. Based on the geographical location information of the UAV itself, it transforms the coordinate position of the obstacle in the UAV to the position in the geodetic coordinate system through the transformation between the two coordinate systems.

[0118] Step 4: Determine whether the obstacle is a movable obstacle by the drone's flight speed and the range of changes in the obstacle's position vector relative to the drone. For movable obstacles, establish a motion model of the movable obstacle by continuously changing the obstacle's position, and predict the changes in the obstacle's position.

[0119] Step 5: Build a 3D obstacle model using obstacle information obtained from binocular vision and ultrasound, and output control information when the UAV's flight path conflicts with the obstacle to avoid the obstacle in advance. After avoiding the obstacle, the UAV can continue to inspect along the preset route by correcting its flight path.

[0120] Step 6: Finally, the flight obstacle avoidance controller receives the feedback control information output by the fusion obstacle avoidance processing module, and then controls the UAV to fly around the obstacle through the flight control module, so as to realize the UAV's early perception and real-time avoidance of obstacles.

[0121] In this embodiment of the invention, the trajectory correction module simultaneously integrates the UAV's pose information obtained by the INS inertial navigation submodule and the image data collected by the binocular vision precision measurement module. It corrects the UAV's trajectory by adjusting the deviation of the obstacle avoidance position, enabling the UAV to continue inspecting the gas pipeline along the original path after obstacle avoidance.

[0122] In this embodiment of the invention, when the UAV corrects its flight path, it integrates the image data collected by the binocular vision precision measurement module into the INS inertial navigation submodule. If the UAV's flight attitude and position deviate during navigation in order to avoid obstacles, the binocular vision precision measurement module uses the principle of left and right parallax of the human eye through the binocular camera to obtain the position change deviation based on the change of the object's coordinates in the two lens images, and integrates it into the INS inertial navigation submodule to correct the UAV's motion attitude in a timely manner.

[0123] In this embodiment of the invention, the laser monitoring module includes:

[0124] Laser remote sensing detectors are used to collect emitted and reflected laser signals when drones are inspecting pipelines.

[0125] An image acquisition device, connected to a camera, acquires image information of the gas pipeline through the camera;

[0126] The gimbal is equipped with the laser remote sensing detector, camera, and image acquisition device. The gimbal is embedded in the side of the UAV and can rotate 180 degrees. The gimbal transmits the laser signal data collected by the laser remote sensing detector and the image information data collected by the image acquisition device to the data receiving module through the data transmission module.

[0127] The laser remote sensing detector is composed of a DFB laser and a PIN photodetector; the image acquisition unit uses a PCIE1181 / 1182 network card, which can achieve efficient acquisition of image data.

[0128] As one embodiment of the present invention, please refer to Figure 3When the drone inspects the pipeline, the information interaction module receives laser signal data from the data transmission module and reflects the changes in methane concentration. When a gas pipeline leak is detected, the camera and image acquisition device are activated and commands are issued to collect image information data of the gas pipeline and transmit the data to the data processing module. The leak location of the gas pipeline is marked using machine vision technology. The data processing module also has the functions of recording the concentration of leaked methane gas and capturing the leak scene.

[0129] In this embodiment of the invention, when the gas pipeline leaks, the laser remote sensing detector on the drone absorbs some of the laser energy as it scans the CH4 gas cloud formed around the leak point. The concentration of CH4 can be determined based on the initial and echo power of the laser. Simultaneously, a camera, aided by machine vision, marks the leak area of ​​the gas pipeline. Based on the drone's own geographical location information, the precise location of the gas leak can be accurately determined. The specific steps are as follows:

[0130] Step 1: When a gas pipeline or key inspection grid leaks, as the laser remote sensing detector scans the inspection area, align the center frequency of the laser remote sensing detector's output light with the spectral absorption peak of CH4. Express the absorption spectrum of CH4 gas using the Lorentz function, and expand the gas's absorption coefficient L(v) to a Fourier series:

[0131] L(v)=L0[W0-W2cos(2ωt)+……]

[0132]

[0133]

[0134]

[0135] In the formula, (v) is the absorption coefficient of CH4 for light at frequency v; is the peak absorption coefficient of the gas absorption spectrum; γ is the frequency modulation amplitude; γ is the half-width of the absorption line.

[0136] Since the absorption coefficient of gas in the near-infrared band is very small, the relationship between the output light and the input light can be expressed as:

[0137]

[0138] In the formula, is the optical power received by the photodetector; s is the light collection efficiency. The optical power output of a semiconductor laser; is the ratio of the optical power received by the laser remote sensing detector to the optical power output when there is no gas absorption; C is the volume concentration of the gas being measured; R is the length of the gas absorption optical path.

[0139] Based on the above formulas, the first harmonic and the second harmonic can be derived as follows:

[0140]

[0141]

[0142] It can be seen that the second harmonic contains CH4 concentration information, while the first harmonic is unrelated to CH4 concentration.

[0143] The monitored CH4 gas concentration can be obtained by combining the above formula:

[0144]

[0145] Step 2: The data processing module can obtain the gas concentration change through the above calculations, and the point of highest concentration is the location of the gas leak;

[0146] Step 3: To eliminate the influence of factors such as wind speed on the monitoring of gas concentration, machine vision is used to mark the location of gas leaks, thereby improving the accuracy of leak location.

[0147] First, the spatial information of the image acquired by the image acquisition card is described using the frequency domain. Then:

[0148] F{ }∝

[0149] In the formula: F is the Fourier transform; The average Fourier amplitude of the image set acquired by the image acquisition card; The rules governing the statistical properties of images.

[0150] According to the image transformation rules shown in the above formula, the log spectrum of the image can be obtained. Then, after performing mean filtering on it, we have:

[0151] W=V·h m

[0152] Where: W is the log spectrum of V after mean filtering; V is the original log spectrum without mean filtering; h m This is a filtering operator.

[0153] Based on the calculated V and W, the spectral residual R = WV can be obtained. Under this condition, the obtained R contains salient information of the gas pipeline image. Therefore, let the salient plot in the spatial domain be S; the phase after the Fourier transform operation be φ; and the salient plot of the gas pipeline image be S.

[0154] S=|F -1 [exp(R+iφ)]| 2

[0155] In the formula: F -1 This represents the inverse Fourier transform; i is the imaginary unit. At this point, the saliency map of the gas pipeline image is obtained from the above formula, thus determining the precise location of the gas pipeline leak.

[0156] In this embodiment of the invention, the laser remote sensing detector and the camera are installed side by side on the UAV. The laser remote sensing detector is used to transmit and receive laser signals, and the camera can collect image information during gas pipeline inspection and observe the real-time status of the gas pipeline. The same gimbal is used to ensure that the laser light path and the camera light path are parallel. When the UAV performs automatic obstacle avoidance and turning actions, the gimbal can be adjusted to change the laser direction and inspect the gas pipeline.

[0157] In this embodiment of the invention, the flight control module receives control commands from the precision navigation module, the swarm cooperative obstacle avoidance module, and the trajectory correction module, and continuously adjusts the flight attitude to enable the UAV to perform inspections along the pipeline according to the planned route.

[0158] In this embodiment of the invention, the drones are inspected using a remote take-off and landing method. At the same time, based on the path planning results and to ensure the orderly recovery of the drones, the number of drones to be launched, as well as the location of the take-off and landing point and the flight path of each drone, are reasonably planned.

[0159] In this embodiment of the invention, redundancy is set based on the reasonable configuration of the number of drones in the flight path planning module. When a drone fails to perform a task, a redundant drone is used to replace the original drone to continue performing the inspection task, thus realizing the inspection "relay".

[0160] In this embodiment of the invention, the loading vehicle acts as a mobile base station to communicate with the drone. The loading vehicle receives and processes the data collected by the drone. During the drone inspection process, the drone landing platform is retrieved. According to the landing time of the drone swarm, the landing platform is pre-positioned at the drone landing location to retrieve the drones.

[0161] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0162] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0163] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0164] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

[0165] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A gas pipeline leak inspection system for residential communities based on UAV swarm collaboration, characterized in that, include: A drone launch and landing platform is used to launch and recover drones. The loading vehicle processing unit is used to plan the flight path of the UAV swarm, generate flight trajectory, and output the flight trajectory signal to the UAV inspection unit. It is also used to process the received gas leak data to achieve accurate location of the gas leak. The drone swarm inspection unit is used to achieve precise navigation through flight paths, real-time obstacle avoidance, trajectory correction after obstacle avoidance, and monitoring of gas pipeline leaks. The loading vehicle processing unit includes: The loading vehicle is used to carry the UAV take-off and landing platform and is placed at the UAV take-off and landing position according to the flight path of the UAV swarm, so as to realize the autonomous take-off and landing of the UAV. The flight path planning module is used to generate flight paths for drone swarms. The information interaction module is used to accurately locate the gas leak. The information interaction module includes a data receiving module, a data processing module, and a signal transmission module. The drone swarm inspection unit includes: The precision navigation module is used for precise navigation along the pipeline based on the flight path; The swarm cooperative obstacle avoidance module is used to achieve real-time obstacle avoidance; The trajectory correction module is used to correct the trajectory of the UAV. Laser monitoring module is used for leak detection in pipelines; The drone swarm inspection unit also includes a data transmission module, a signal receiving module, and a flight control module.

2. The residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 1, characterized in that, The flight path planning module is connected to the signal transmission module and sends commands to the precision navigation module through output signals, enabling the UAV swarm to accurately inspect the gas pipeline network along the flight path. The flight path planning module includes: The baseline calculation submodule establishes a three-dimensional network model based on the geographic information of the area to be inspected, divides the key inspection grid, plans the path for the drone swarm to complete the inspection in the shortest time, and determines the appropriate number of inspection drones to be deployed in the community. The adaptive control submodule plans routes based on endurance and key area grids, and sets recovery points according to the launch sequence and flight path, enabling UAVs to accurately inspect key grids.

3. The residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 1, characterized in that, The precise navigation module includes: The GPS navigation module receives commands from the flight path planning module to enable the UAV to perform inspections along the flight path. The RTK / INS positioning module, connected to the GPS navigation module, is used to achieve precise positioning of the drone. The RTK / INS positioning module includes: The RTK positioning submodule is used to acquire precise location data for the drone; The INS inertial navigation submodule contains a gyroscope and an accelerometer, which are used to acquire the attitude information of the UAV. The RTK / INS filter processor is used to fuse the UAV pose data obtained by the INS inertial navigation submodule and the UAV position data obtained by the RTK positioning submodule to obtain high-precision position and pose information. The RTK / INS filter processor is connected to the flight control module to accurately control the UAV to inspect the gas pipeline without deviation along the flight path.

4. A residential gas pipeline leak inspection system based on UAV swarm collaboration as described in claim 3, characterized in that, The INS inertial navigation submodule processes the data acquired by the gyroscope and accelerometer to obtain velocity and attitude angle information, thereby obtaining the UAV's pose information. The UAV's position information and pose information are combined through an RTK / INS filter to obtain high-precision position information while correcting the INS navigation parameters, thus obtaining high-precision pose information. The flight control module continuously adjusts the UAV's position and attitude to achieve precise inspection of the gas pipeline by the UAV.

5. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 3, characterized in that, The UAV is embedded with a swarm collaborative obstacle avoidance module, which includes: The ultrasonic obstacle detection module is used to emit ultrasonic waves during the flight of the drone and receive the reflected ultrasonic signals when an obstacle is detected. A binocular vision precision measurement module is used to acquire image data through a binocular camera; The obstacle avoidance processing module receives ultrasonic reflection signals collected by the ultrasonic obstacle perception module and image data collected by the binocular vision precision measurement module. It then processes the two types of data through data fusion and provides feedback control information to the flight obstacle avoidance controller through three-dimensional obstacle reconstruction.

6. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 5, characterized in that, The swarm collaborative obstacle avoidance module uses a binocular vision precision measurement module as the main perception module, supplemented by an ultrasonic obstacle perception module. The binocular vision precision measurement module includes a binocular camera and a binocular vision processor. The ultrasonic obstacle perception module includes an ultrasonic sensor and an ultrasonic processor. The binocular camera and the binocular vision processor are directly connected, and the ultrasonic sensor and the ultrasonic processor are directly connected. The binocular vision processor and the ultrasonic processor are connected to the fusion obstacle avoidance processing module. The automatic obstacle avoidance of the UAV is achieved through the decision-making and data conversion of the obstacle avoidance fusion processing module.

7. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 5, characterized in that, The trajectory correction module integrates the UAV's pose information obtained by the INS inertial navigation submodule and the image data collected by the binocular vision precision measurement module. It corrects the UAV's trajectory based on the deviation of the obstacle avoidance position change, enabling the UAV to continue inspecting the gas pipeline along the original path after obstacle avoidance.

8. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 7, characterized in that, When correcting its flight path, the UAV integrates the image data collected by the binocular vision precision measurement module into the INS inertial navigation submodule. If the UAV's flight attitude and position deviate during navigation to avoid obstacles, the binocular vision precision measurement module uses the principle of left and right parallax of the human eye through the binocular camera to obtain the position change deviation based on the change of the object's coordinates in the two lens images, and integrates it into the INS inertial navigation submodule to correct the UAV's motion attitude in a timely manner.

9. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 1, characterized in that, The laser monitoring module includes: Laser remote sensing detectors are used to collect emitted and reflected laser signals when drones are inspecting pipelines. An image acquisition device, connected to a camera, acquires image information of the gas pipeline through the camera; The gimbal is mounted on the side of the UAV. The gimbal transmits the laser signal data collected by the laser remote sensing detector and the image information data collected by the image acquisition device to the data receiving module through the data transmission module.

10. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 9, characterized in that, When the drone inspects the pipeline, the information interaction module receives laser signal data from the data transmission module and reflects the changes in methane concentration. When a gas pipeline leak is detected, the camera and image acquisition device are activated and commands are issued to collect image information data of the gas pipeline and transmit the data to the data processing module. The leak location of the gas pipeline is marked using machine vision technology. The data processing module also has the functions of recording the concentration of leaked methane gas and capturing the leak scene.

11. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 9, characterized in that, When a gas pipeline leaks, the laser remote sensing detector on the drone will absorb some of the laser energy as it scans the CH4 gas cloud formed around the leak point. The concentration of CH4 can be determined based on the initial power and echo power of the laser. At the same time, the leak range of the gas pipeline is marked by a camera and machine vision. Based on the geographical location information of the drone itself, the precise location of the gas leak can be accurately determined.

12. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 9, characterized in that, The laser remote sensing detector and camera are installed side by side on the UAV. The laser remote sensing detector is used to transmit and receive laser signals, and the camera collects image information during gas pipeline inspection to observe the real-time status of the gas pipeline. The same gimbal is used to ensure that the laser beam path is parallel to the camera beam path. When the UAV performs automatic obstacle avoidance and turning actions, the gimbal can be adjusted to change the laser direction to inspect the gas pipeline.

13. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 1, characterized in that, The flight control module receives control commands from the precision navigation module, the swarm cooperative obstacle avoidance module, and the trajectory correction module, and continuously adjusts the flight attitude to enable the UAV to perform inspections along the pipeline according to the planned route.

14. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 1, characterized in that, The drones are used for inspections by taking off and landing at different locations. At the same time, based on the results of path planning and to ensure the orderly recovery of the drones, the number of drones to be launched, as well as the location of the take-off and landing point and the flight path of each drone are reasonably planned.

15. A residential gas pipeline leak inspection system based on UAV swarm collaboration as described in claim 14, characterized in that, Based on the reasonable configuration of the number of drones in the flight path planning module, redundancy is set. When a drone fails to perform a task, a redundant drone is used to replace the original drone to continue the inspection task, thus realizing the inspection "relay".

16. A residential gas pipeline leak inspection system based on UAV swarm collaboration according to claim 1, characterized in that, The loading vehicle acts as a mobile base station, communicating with the drones. The loading vehicle receives and processes the data collected by the drones. During the drone inspection process, the drone landing platform is retrieved. Based on the landing time of the drone swarm, the landing platform is pre-positioned at the drone landing location to retrieve the drones.

Images