Multi-unmanned aerial vehicle cooperative path planning method for mountain monitoring

Abstract

The invention provides a multi-unmanned aerial vehicle cooperative path planning method for mountain monitoring, and the method comprises the steps: collecting an image of a to-be-detected mountain through a set unmanned aerial vehicle cluster formation, carrying out the feature point extraction of the to-be-detected mountain through a Harris corner detection algorithm, marking the feature points of a mountain top, the feature points of a mountain depression and the feature points of a mountain foot, and carrying out the positioning of the to-be-detected mountain. The method comprises the steps that feature points of a to-be-detected mountain are obtained, coordinates of the feature points in a world coordinate system are obtained through calculation, then a three-dimensional model of the to-be-detected mountain is obtained, then the feature points serve as task detection points to be distributed to an unmanned aerial vehicle cluster through a greedy algorithm, and three-dimensional path planning is conducted on all unmanned aerial vehicles distributed with detection tasks according to an A-Star algorithm through the three-dimensional model of the to-be-detected mountain. According to the invention, the unmanned aerial vehicles are used for automatic formation and cooperative acquisition of the to-be-detected mountain image, the labor investment is reduced, the system can adapt to different terrains and environment conditions, the data acquisition coverage range is wider, the working efficiency is improved, and powerful support is provided for monitoring of natural disasters in mountainous areas.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) path planning, and in particular to a multi-UAV collaborative path planning method for mountain monitoring. Background Art

[0002] Geological disasters refer to geological actions or phenomena that are caused by natural or human factors and cause loss of human life and property and damage to the environment. In the prior art, simple manual detection methods or instrument detection methods are usually used to detect and warn geological disasters. For example, personnel are dispatched to conduct regular screening and exploration at geological disaster risk points or multiple types of sensors are set up at geological disaster risk points to collect and process data according to the set cycle. Landslides are a typical type of geological disaster, which are characterized by strong suddenness, great destructiveness, and high difficulty in prediction. It is difficult to make timely and accurate predictions through conventional manual means.

[0003] In order to reduce the potential threat of landslides, professionals need to conduct regular inspections and construction protection of the mountains. At present, the detection of landslides mainly relies on human screening and ground exploration technology. However, human screening is not only labor-intensive and inefficient, but also covers a limited area of ​​the mountain area each time. In addition, ground exploration technology is expensive and easily restricted by external factors such as weather and environment, making it difficult to achieve long-term and continuous monitoring of the mountains. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a multi-UAV collaborative path planning method for mountain monitoring that can automatically obtain mountain models.

[0005] Technical solution: A multi-UAV collaborative path planning method for mountain monitoring, comprising the following steps: S1. Select several drones with the same structure to form a drone cluster. Each drone is equipped with a camera and a radar. The control signal of each drone in the drone cluster is generated based on the set drone kinematic model. S2. Construct a drone cluster formation based on the consistency control protocol, set a drone in the drone cluster as the pilot drone, and set the remaining drones as follower drones. When flying, the follower drones are arranged in a fixed order on the same side of the pilot drone; S3, the drone cluster formation set in control step S2 starts from the starting point, flies at a set speed and approaches the mountain to be measured, uses the camera carried by each drone to collect images of the mountain to be measured at preset time intervals, and marks the corresponding position information on each image of the mountain to be measured, uses the radar carried by each drone to measure the distance between each drone and the mountain to be measured in real time, and obtains the inclination angle and inclination direction of the mountain to be measured; by setting a distance threshold between the drone cluster and the mountain to be measured, when the distance between any drone in the drone cluster and the mountain to be measured is less than or equal to the distance threshold, the drone cluster stops collecting images of the mountain to be measured and returns to the starting point; S4, converting the image of the mountain to be measured obtained in step S3 into a grayscale image, extracting feature points of the mountain to be measured using the Harris corner detection algorithm, and marking the extracted feature points as mountain top feature points, valley feature points and mountain foot feature points according to the direction of gradient change, calculating the depth of each feature point to the corresponding drone according to the camera imaging principle, and obtaining the average coordinates of each feature point in the world coordinate system using the conversion matrix; S5, performing multivariate polynomial fitting according to the coordinates of each feature point obtained in step S4 in the world coordinate system to obtain a ridge curve function of the mountain to be measured, setting a ridge curve generation range, extracting ridge curve segments and connecting the endpoints of each ridge curve segment to form a closed profile of the mountain to be measured, and then performing coordinate transformation on the feature points on the profile of the mountain to be measured according to the inclination angle and inclination direction of the mountain to be measured obtained by the radar, and scanning the transformed feature points along the inclination direction to obtain a three-dimensional model of the mountain to be measured; S6. Take the mountain top feature points and valley feature points as task detection points, calculate the Manhattan distance between each UAV and each task detection point, take the minimum total detection distance as the goal, use the greedy algorithm to allocate detection tasks, so that each task detection point is assigned to a UAV, and the UAVs that are not assigned to the detection task stay at the starting point; S7. Using the three-dimensional model of the mountain to be tested obtained in step S5, selecting Manhattan distance as the evaluation function, and using the A-Star algorithm to perform three-dimensional path planning for each UAV assigned to the detection task in step S6.

[0006] Specifically, step S1 includes: The second-order consistent control input of UAV i is obtained by using the second-order motion equation of UAV i. : , Where: , , They are respectively UAV i in the world coordinate system The position coordinates in the x, y, and z directions, , , are the velocity vectors of drone i in the x, y, and z directions, respectively. , , are the second-order consistency control inputs of UAV i Components in the x, y, and z directions.

[0007] Specifically, step S2 includes the following sub-steps: S21, calculating the relative distance between each follower drone and the pilot drone; S22, the pilot drone and each follower drone take off from the starting point in sequence and fly to a preset altitude, and then maintain the preset altitude; S23, setting the speed of each follower drone to be consistent with that of the pilot drone, and controlling the pilot drone to fly on the far left of the formation, and the follower drones are arranged in sequence on the right side of the pilot drone at a fixed distance in the order of numbers. According to the consistency control protocol with spacing in three-dimensional space, the control input of the follower drone is: , Where: is the control input of the following drone i at time t; is a fixed gain; is the damping gain; is the connection strength between following drone i and following drone j; i and j are drone numbers; is the relative distance between the following UAV i and the pilot UAV at time t; is the relative distance between the following UAV j and the pilot UAV at time t; is the expected relative distance between any two UAVs; The world coordinate system of the following drone i and the pilot drone t at time The position distance on the lower z axis; The world coordinate system of the following drone j and the pilot drone t at all times The position distance on the lower z axis; , are the velocity amplitudes of following UAV i and following UAV j at time t respectively; is the total number of drones in the drone swarm; S24. Under the control input of the following drone, the following drone tracks the desired position by tracking the desired relative distance until the speed and position state of the drone cluster meets the consistency control protocol with spacing in three-dimensional space, thus completing the drone cluster formation.

[0008] Specifically, step S4 includes the following sub-steps: S41, converting the image of the mountain to be measured into a grayscale image using a maximum value method; S42, applying the Harris corner detection algorithm to perform image segmentation on the grayscale image converted in step S41, and performing feature point detection on the segmented grayscale image to obtain all feature points in the grayscale image, and then marking the identified feature points as mountain top feature points, valley feature points and foot feature points according to the gradient change direction; S43, randomly selecting two grayscale images captured by two different drone cameras at the same time, using a stereo matching algorithm based on Census transformation to perform stereo matching on the same feature point in the two grayscale images, and obtaining the disparity of the same feature point in the two grayscale images; S44, using the disparity of the same feature point in the two grayscale images obtained in step S43, according to the camera imaging principle and the triangle similarity law, calculate the depth of the feature point from the camera carried by one of the two drones in step S43, and then obtain the three-dimensional coordinates of the feature point in the camera coordinate system, and then use the camera parameters to convert the three-dimensional coordinates of the feature point in the camera coordinate system into coordinates in the world coordinate system; S45, repeating steps S43 and S44, obtaining a coordinate set of the same feature point in the world coordinate system obtained by grayscale images collected by any two drones at each collection time, and calculating the average coordinate of the feature point in the world coordinate system; S46. Repeat steps S43 to S45 to obtain the average coordinates of all feature points in the world coordinate system.

[0009] Specifically, step S43 includes: using a stereo matching algorithm based on Census transformation to match the grayscale images captured by the two cameras, selecting a pixel corresponding to a feature point in a grayscale image , calculate the Census value and pixel value of each feature point in another grayscale image The Hamming distance of the Census value is obtained with the pixel The feature point pixel with the smallest Hamming distance , realize stereo matching of the same feature point in the grayscale images collected by two cameras, and then obtain the pixel point of the feature point on the two grayscale images , The parallax in the horizontal direction is given by the following formula: , Where: To explore the feature points of the camera carried by drone i in the kth acquisition The corresponding pixel Census value; It is a bit-by-bit connection operation; For feature points Corresponding pixel Neighborhood window; q is the pixel in the neighborhood window; , The feature points Corresponding pixels And the gray value of the neighborhood pixel q; For comparison operations; , Where: Pixel and The Hamming distance of To explore the feature points of the camera carried by UAV j in the kth acquisition The corresponding pixel Census value; E is the number of digits of Census transformation; is an XOR operation; It is stipulated that in the two grayscale images to be matched, the exploration drone i should be located on the left side of the exploration drone j; For comparison operations; , Where: To explore the feature points in the image acquired by the camera carried by UAV i and j for the kth time The matching pixels and parallax; , The image pixel coordinate system matching point under , The horizontal pixel coordinate of .

[0010] Specifically, step S44 includes: According to the camera imaging principle and the triangle similarity law, the depth of the feature point from the camera carried by the exploration drone i and its three-dimensional coordinates in the camera coordinate system are calculated. The formula is: , Where: , , The feature points The grayscale image acquired by cameras i and j in the kth acquisition is obtained in the camera coordinate system The x, y, and z coordinates of the drone. i and j are the numbers of the two drones respectively. f is the camera focal length; is the expected relative distance between each UAV; , The feature points At camera i in the image pixel coordinate system The x and y coordinates of the following: To explore the feature points in the image acquired by the camera carried by UAV i and j for the kth time according to the camera imaging principle and the triangle similarity law The depth of the camera carried by the exploration drone i; , is the principal point coordinate of camera i; The camera's rotation matrix and translation matrix are used to construct a transformation matrix to transform the three-dimensional coordinates of the feature points in the camera coordinate system into coordinates in the world coordinate system: , Where: For feature points In the kth acquisition, the grayscale image acquired by cameras i and j is obtained in the world coordinate system The coordinates below; For feature points The image acquired by cameras i and j in the kth acquisition is obtained in the camera coordinate system The coordinates below; is the rotation matrix of camera i, is the translation matrix of camera i, obtained from the camera parameters.

[0011] Specifically, step S5 includes the following sub-steps: S51, performing multivariate polynomial fitting using the average coordinates of each feature point in the world coordinate system, obtaining optimal parameters of the multivariate polynomial by minimizing the error, and constructing a ridge curve function; S52, setting the range of generating the ridge curve according to the average coordinates of the feature points at the top of the mountain and the feature points at the foot of the mountain in the world coordinate system: , Where: , They are the x-axis and y-axis coordinates of the mountain to be measured in the world coordinate system; is the average coordinate of the feature point 1 at the foot of the mountain in the x-axis direction in the world coordinate system, is the average coordinate of the feature point 2 at the foot of the mountain in the x-axis direction in the world coordinate system, is the average coordinate of the feature point i on the top of the mountain in the y-axis direction in the world coordinate system; S is the number of feature points on the top of the mountain; S53, extracting ridge curve segments according to the ridge curve function and the ridge curve generation range, and connecting the end points of the ridge curve segments to obtain a closed profile of the mountain to be measured; S54, according to the inclination angle and inclination direction of the mountain to be measured obtained by the radar, a reduction matrix and a rotation matrix about the y-axis are established to transform the feature points on the profile of the mountain to be measured. The rotation matrix and the reduction matrix are as follows: , , Where: is the rotation matrix about the y-axis; The inclination angle of the mountain to be measured is obtained by the radar; To reduce the matrix; , and They are the reduction ratios in the x, y, and z axis directions respectively; S55, scanning the characteristic points on the transformed profile of the mountain to be measured along the inclined direction to obtain a three-dimensional model of the mountain to be measured.

[0012] Preferably, step S5 further includes: S56. Use Laplace smoothing to reduce the unnatural transition of the edge of the three-dimensional model of the mountain to be measured. The formula is: , Where: is the point position that needs to be smoothed; is the point position after smoothing; For point The set of adjacent points of ; For collection size.

[0013] Specifically, step S7 includes the following sub-steps: S71, rasterizing the three-dimensional space of a certain UAV assigned to the detection task, and determining the initial position coordinates and task detection point coordinates of the UAV; S72, using Manhattan distance as the evaluation function of the A-Star algorithm, starting from the initial position coordinates of the UAV, selecting the node with the minimum evaluation function among its adjacent nodes as the next central node, and repeating this selection method until the task detection point coordinates are reached; S73, tracing back from the coordinates of the mission detection point to construct a track path from the initial position coordinates of the UAV to the coordinates of the mission detection point; S74. Repeat steps S71 to S73 until all drones assigned to the detection task have completed the corresponding complete path planning.

[0014] Preferably, step S73 further includes: The track path is smoothed using a third-order Bezier curve. Each node on the track path obtained by the A-Star algorithm is used as the control point of the third-order Bezier curve. Four consecutive nodes are used as a group of control points. The formula of the third-order Bezier curve is as follows: , Where: is the output point of the third-order Bezier curve; , , and are 4 consecutive Bezier control points; is the changing parameter.

[0015] Beneficial effects: Compared with the prior art, the present invention has the following significant effects: 1. Using automatic drone formations to collaboratively collect images of the mountain to be measured reduces labor input compared to manual measurement or single-machine measurement. Drone formations can adapt to different terrain and environmental conditions and can perform collection operations in areas that are difficult to reach manually. The data collection coverage is wider, effectively shortening the data collection time and improving work efficiency.

[0016] 2. The cameras carried by the drone formation are used to collect images of the mountain to be measured multiple times, providing more image sets for target point distance measurement for visual ranging. In addition, images obtained from different angles by different drones are conducive to reducing the parallax error caused by a single perspective and improving the accuracy of distance calculation, so as to ensure the reliability of the three-dimensional modeling of the mountain to be measured.

[0017] 3. The mountain top, valley and foot of the mountain are identified and modeled as the characteristic points of the mountain and used as the end points of drone detection. They are then matched one by one with the monitoring drones, thus achieving efficient drone image acquisition.

[0018] 4. This method can be applied to the monitoring of various natural disasters in mountainous areas. For example, it can realize long-term and continuous landslide monitoring of mountains, and combine machine learning models to predict and warn of landslides, thereby improving the level of automation of natural disaster monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0020] A preferred embodiment of the present invention is further described below in conjunction with the accompanying drawings.

[0021] See also Figure 1 As shown, the present invention provides a multi-UAV collaborative path planning method for mountain monitoring, comprising the following steps: S1. Select several drones with the same structure to form a drone cluster. Each drone is equipped with the same camera and radar and satisfies the same kinematic model. Establish a set of N drones: , The control signals of each drone in the drone cluster are generated based on the drone kinematic model. Specifically, the second-order motion equation of drone i is used to obtain the second-order consistent control input of drone i: : , Where: , , They are respectively UAV i in the world coordinate system The position coordinates in the x, y, and z directions, , , are the velocity vectors of drone i in the x, y, and z directions, respectively. , , are the second-order consistency control inputs of UAV i Components in the x, y, and z directions.

[0022] According to the kinematic geometry of the drone, It can be expressed as: , Where: is the velocity amplitude of UAV i; is the speed of drone i; is the heading angle of UAV i; is the pitch angle of UAV i.

[0023] S2. Construct a drone cluster formation based on the consistency control protocol, and adopt the "leader-follower" method framework. During the entire exploration mission, the exploration drone formation needs to maintain a specific flight altitude, with the nose facing the mountain to be explored, and the formation is arranged in a linear "one". A drone in the drone cluster is set as the pilot drone, and the remaining drones are set as follower drones. During flight, the follower drones are arranged in a fixed order on the same side of the pilot drone. In this embodiment, the pilot drone is located on the leftmost side of the entire formation, and the remaining follower drones are arranged in sequence on the right side of the pilot exploration drone in order of numbering and at a fixed distance. This formation can not only effectively alleviate the communication and computing pressure, but also ensure the reliability of subsequent visual ranging of various feature points on the mountain.

[0024] This step specifically includes the following sub-steps: S21. Calculate the relative distance between each follower drone and the pilot drone. Suppose the pilot drone is , in the world coordinate system The position coordinates of , the speed is , follow the drone i , follow the drone i and pilot drone The relative distance is: , Where: is the relative distance between the following UAV i and the pilot UAV 0 (0 is the number) at time t; and They are respectively the pilot drone 0 and the follower drone i in the world coordinate system at time t The location coordinates of the following.

[0025] S22, the pilot drone and each follower drone take off from the starting point in sequence, fly to a preset flight altitude, and then maintain the flight altitude.

[0026] S23, set the speed of each follower drone to be consistent with that of the pilot drone, and control the pilot drone to fly on the far left of the formation. Each follower drone is arranged in sequence on the right side of the pilot drone at a fixed distance in the order of number. The coordinate position description is that each follower drone must have the same z-axis coordinate as the pilot drone, and the x-axis coordinates are equidistantly separated: , Where: i and j are the drone numbers; and are the velocity amplitudes of the following UAVs i and j at time t respectively; and The world coordinate system of the following drones i and j at time t Lower z-axis position; and are the distances of follower UAVs i and j relative to the pilot UAV at time t; is the expected relative distance between each UAV.

[0027] According to the consistency control protocol with spacing in three-dimensional space, the control input of the following drone is: , Where: is the control input of the following drone i at time t; is a fixed gain; is the damping gain; is the connection strength between following drone i and following drone j; i and j are drone numbers; is the relative distance between the following UAV i and the pilot UAV at time t; is the relative distance between the following UAV j and the pilot UAV at time t; is the expected relative distance between any two UAVs; The world coordinate system of the following drone i and the pilot drone t at time The position distance on the lower z axis; The world coordinate system of the following drone j and the pilot drone t at all times The position distance on the lower z axis; , are the velocity amplitudes of following UAV i and following UAV j at time t respectively; is the total number of drones in the drone cluster.

[0028] S24, control input of the following drone Under the action, follow the drone through the expected relative distance The tracking of the desired position is realized until the speed and position state of the UAV cluster meet the consistency control protocol with spacing in three-dimensional space, completing the UAV cluster formation.

[0029] S3: The drone cluster formation set in step S2 starts from the starting point, flies at a set speed and approaches the mountain to be measured, and uses the camera carried by each drone to capture images at a preset time interval. The image of the mountain to be measured is acquired, and the corresponding position information is marked on each image of the mountain to be measured, so as to realize the accurate association between the spatial position and the image data. The distance between each drone and the mountain to be measured is measured in real time by using the radar carried by each drone, and the inclination angle and inclination direction of the mountain to be measured are obtained. In order to prevent the exploration drone from colliding with the mountain during the forward movement, it is necessary to set a distance threshold between the drone cluster and the mountain to be measured. When the distance between any drone in the drone cluster and the mountain to be measured is less than or equal to the distance threshold, the drone cluster stops collecting images of the mountain to be measured and returns to the starting point, and executes step S4.

[0030] S4. Convert the image of the mountain to be measured obtained in step S3 into a grayscale image, use the Harris corner detection algorithm to extract feature points of the mountain to be measured, and mark the extracted feature points as mountain top feature points, valley feature points and mountain foot feature points according to the gradient change direction. Calculate the depth of each feature point to the corresponding drone according to the camera imaging principle, and use the transformation matrix to obtain the average coordinates of each feature point in the world coordinate system.

[0031] This step specifically includes the following sub-steps: S41. Based on the intrinsic parameter matrix and distortion coefficient obtained in the camera calibration, all collected images are subjected to distortion correction to eliminate the image distortion problem caused by the camera lens. The color images after distortion correction need to be further preprocessed. First, the color images are filtered using the median filtering technology to better protect the image edge characteristics while eliminating noise. Second, the color images are grayed using the maximum value method to reduce noise, improve image quality, and realize the effective conversion of color images to grayscale images, thereby reducing the complexity of image processing.

[0032] S42. Define the key feature positions of the mountain to be tested, including the "mountain top", "valley" and "foot of the mountain", as the feature points of the mountain to be tested. In a two-dimensional image, these feature points appear as the intersection of two mountain ranges, and belong to the boundaries of two mountain ranges in different directions in the local area. In the grayscale image, the gradient value and gradient direction change rate of these points are extremely significant. Apply the Harris corner detection algorithm to perform image segmentation on the grayscale image converted in step S41, and perform feature point detection on the segmented grayscale image to obtain all feature points in the grayscale image, and then mark the identified feature points as mountain top feature points according to the gradient change direction. , valley feature points , foot feature point and .

[0033] Among them, the mountain top feature point It is the highest peak point of the local gradient change. The direction of the gradient radiates outward from the characteristic point on the top of the mountain, pointing to the descending direction of the mountain slope; the characteristic point on the valley It is the lowest valley point of local gradient change. The direction of the gradient is concentrated inward, along the descending direction of the mountain slope, pointing to the characteristic point of the valley. The characteristic point at the foot of the mountain is the critical value point of local gradient change. The direction of the gradient is arranged along the direction of the hillside. The gradient vector points to the lower left. , the gradient vector points to the lower right. The number of feature points is determined by the Harris corner detection algorithm on the grayscale image.

[0034] S43. Select any two grayscale images captured by two different drone cameras at the same time, use a stereo matching algorithm based on Census transformation to perform stereo matching on the same feature point in the two grayscale images, and obtain the disparity of the same feature point in the two grayscale images.

[0035] Specifically, in the kth acquisition, any two images are selected, and the stereo matching algorithm based on Census transformation is used to complete the stereo matching of each feature point in the image, and the disparity of each point is obtained. For example, select two cameras i and j from any N cameras (it is stipulated that for the two selected cameras, the exploration UAV equipped with camera i should be located to the left of the exploration UAV equipped with camera j, that is, i < j). Use the stereo matching algorithm based on Census transform to match the grayscale images collected by the two cameras at the kth time, and select the mountaintop feature points in the image obtained by the left camera i The corresponding pixel , by calculating the Hamming distance between the Census value of each feature point pixel p in the image obtained by the right camera j and the Census value of the pixel , the feature point pixel with the smallest Hamming distance from the pixel is obtained , realizing the stereo matching of the mountaintop feature points in the pictures collected by the two cameras i and j at the kth time , and then obtaining the pixel points of the mountaintop feature points on the two images , The parallax in the horizontal direction , the transformation formula function of Census is: , In the formula: is the Census value of the pixel point corresponding to the feature point in the kth collection by the camera carried by the exploration UAV i; ; is the bit-by-bit connection operation of bit positions; is the neighborhood window of the feature point corresponding to the pixel point ; q is the pixel in the neighborhood window; , are the grayscale values of the pixel corresponding to the feature point and the neighborhood pixel q respectively; ; is the comparison operation; The Hamming distance formula is: , In the formula: is the Hamming distance between the pixel and ; is the Census value of the pixel point corresponding to the feature point in the kth collection by the camera carried by the exploration UAV j; E is the number of bits of the Census transform; ; is the exclusive OR operation; It is stipulated that in the two grayscale images to be matched, the exploration UAV i should be located to the left of the exploration UAV j; is the comparison operation; The parallax formula is: , Where: To explore the feature points in the image acquired by the camera carried by UAV i and j for the kth time The matching pixels and parallax; , The image pixel coordinate system matching point under , The horizontal pixel coordinate of .

[0036] S44, using the mountain top feature points in the two grayscale images obtained in step S43 The parallax of the mountain top is calculated based on the camera imaging principle and the triangle similarity law. The depth from the camera carried by drone i in step S43, and then the mountain top feature point is obtained The three-dimensional coordinates in the camera coordinate system are then used to convert the mountain top feature points into The three-dimensional coordinates in the camera coordinate system are converted to coordinates in the world coordinate system.

[0037] The three-dimensional coordinates in the camera coordinate system are: , Where: , , The mountain top feature points The grayscale image acquired by cameras i and j in the kth acquisition is obtained in the camera coordinate system The x, y, and z coordinates of the drone. i and j are the numbers of the two drones respectively. f is the camera focal length; is the expected relative distance between each UAV; , The mountain top feature points At camera i in the image pixel coordinate system The x and y coordinates of the following: To explore the mountaintop feature points inferred from the k-th image acquired by the camera carried by UAV i and j according to the camera imaging principle and the triangle similarity law The depth of the camera carried by the exploration drone i; , is the principal point coordinate of camera i.

[0038] The camera's rotation matrix and translation matrix are used to construct the transformation matrix and transform the mountain top feature points The three-dimensional coordinates in the camera coordinate system are converted to coordinates in the world coordinate system: , Where: The peak feature point In the kth acquisition, the grayscale image acquired by cameras i and j is obtained in the world coordinate system The coordinates below; The peak feature point The image acquired by cameras i and j in the kth acquisition is obtained in the camera coordinate system The coordinates below; is the rotation matrix of camera i, is the translation matrix of camera i, obtained from the camera parameters.

[0039] S45, repeat steps S43 and S44 to obtain the coordinate set of the world coordinate system of the feature point of the mountain top obtained by the grayscale images collected by any two drones in k acquisitions: .

[0040] In order to improve the accuracy of the measurement model, neighborhood filtering is used to screen and eliminate the coordinates of feature points with large errors. Sum and average the coordinate sets to obtain the peak feature points In the world coordinate system The average coordinates of .

[0041] S46, repeat steps S43 to S45 to obtain the peak feature point , valley feature points , foot feature point and The average coordinate in world coordinate system.

[0042] S5. Perform multivariate polynomial fitting on the coordinates of each feature point obtained in step S4 in the world coordinate system to obtain the ridge curve function of the mountain to be measured, set the ridge curve generation range, extract the ridge curve segments and connect the endpoints of each ridge curve segment to form a closed profile of the mountain to be measured, and then perform coordinate transformation on the feature points on the profile of the mountain to be measured according to the inclination angle and inclination direction of the mountain to be measured obtained by the radar, and scan the transformed feature points along the inclination direction to obtain a three-dimensional model of the mountain to be measured.

[0043] This step specifically includes the following sub-steps: S51. Multivariate polynomial fitting is performed using the average coordinates of each feature point in the world coordinate system. Since the number of feature points such as the "top of the mountain", "valley" and "foot of the mountain" of the mountain to be measured is limited, in order to avoid overfitting, a quadratic polynomial function is used as the ridge line fitting model of the mountain to be measured. This model can fully describe the distribution trend of each feature point in the world coordinate system. The optimal parameters of the quadratic polynomial are obtained by minimizing the error method, and the ridge curve function is constructed.

[0044] S52, setting the range of generating the ridge curve according to the average coordinates of the feature points at the top of the mountain and the feature points at the foot of the mountain in the world coordinate system: , Where: , They are the x-axis and y-axis coordinates of the mountain to be measured in the world coordinate system; is the average coordinate of the feature point 1 at the foot of the mountain in the x-axis direction in the world coordinate system, is the average coordinate of the feature point 2 at the foot of the mountain in the x-axis direction in the world coordinate system, is the average coordinate of the mountain top feature point i in the y-axis direction in the world coordinate system; S is the number of mountain top feature points.

[0045] S53, extracting ridge curve segments according to the ridge curve function and the ridge curve generation range, and connecting the end points of the ridge curve segments to obtain a closed profile of the mountain to be measured.

[0046] S54, according to the inclination angle and inclination direction of the mountain to be measured obtained by the radar, a reduction matrix and a rotation matrix about the y-axis are established to transform the feature points on the profile of the mountain to be measured. The rotation matrix and the reduction matrix are as follows: , , Where: is the rotation matrix about the y-axis; The inclination angle of the mountain to be measured is obtained by the radar; To reduce the matrix; , and They are the reduction ratios in the x, y, and z axis directions respectively; S55, scanning the characteristic points on the transformed profile of the mountain to be measured along the inclined direction to obtain a three-dimensional model of the mountain to be measured.

[0047] S56. The edge of the three-dimensional model of the mountain to be measured obtained by sweeping has some unnatural transition points, and the mountain top feature points are fixed. , valley feature points , foot feature point and At the position in the model, by detecting the edge of the three-dimensional model of the mountain to be tested, Laplace smoothing is used to reduce the unnatural transition of the edge of the three-dimensional model of the mountain to be tested. The formula is: , Where: is the point position that needs to be smoothed; is the point position after smoothing; For point The set of adjacent points of ; For collection size.

[0048] Output the three-dimensional model of the mountain to be tested in STL format after smoothing.

[0049] S6. Use the mountain top feature points and valley feature points as task detection points, calculate the Manhattan distance between each UAV and each task detection point, take the minimum total detection distance as the goal, and use the greedy algorithm to allocate detection tasks so that each task detection point is assigned to a UAV, and the UAVs that are not assigned to the detection task stay at the starting point.

[0050] This step specifically includes the following sub-steps: S61. Create a collection of N drones : , Task checkpoint set: , Where: A set of task detection points; is the mean coordinate of the top feature points of the mountain to be measured, is the mean coordinate of the characteristic points of the depressions in the mountain to be measured.

[0051] S62. Calculate the Manhattan distance between each drone and each detection target point: , Where: For drones Task Checkpoint The Manhattan distance between and For drones and task checkpoints In the world coordinate system The location coordinates of the following.

[0052] S63. Taking the minimum total detection distance as the performance indicator, according to the greedy strategy, select one from the unassigned task detection points, and preferentially assign the selected task detection point to the drone with the shortest Manhattan distance from the point, and add the drone to the set of assigned drones. Repeat the assignment steps until all tasks are assigned.

[0053] S64. UAVs that are not assigned to inspection tasks remain at the starting point and wait.

[0054] S7. Using the three-dimensional model of the mountain to be tested obtained in step S5, selecting Manhattan distance as the evaluation function, and using the A-Star algorithm (A* algorithm) to perform three-dimensional path planning for each UAV assigned to the detection task in step S6.

[0055] This step specifically includes the following sub-steps: S71, rasterizing the three-dimensional space of a certain UAV assigned to the detection task, and determining the initial position coordinates and task detection point coordinates of the UAV; S72, using Manhattan distance as the evaluation function of the A-Star algorithm, starting from the initial position coordinates of the UAV, selecting the node with the minimum evaluation function among its adjacent nodes as the next central node, and repeating this selection method until the task detection point coordinates are reached; S73, trace back from the coordinates of the mission detection point, and construct a track path from the initial position coordinates of the UAV to the coordinates of the mission detection point; use the third-order Bezier curve to smooth the track path, use each node on the track path obtained by the A-Star algorithm as the control point of the third-order Bezier curve, and use 4 consecutive nodes as a group of control points. By adjusting the coordinates of the control points, the track shape is changed to make the output track smoother, ensure the stability of the detection UAV during flight, reduce energy consumption, and avoid problems such as position offset caused by excessive turning angles. The formula of the third-order Bezier curve is as follows: , Where: is the output point of the third-order Bezier curve; , , and are 4 consecutive Bezier control points; is the changing parameter.

[0056] S74. Repeat steps S71 to S73 until all drones assigned to the detection task have completed the corresponding complete path planning.

[0057] This method can be applied to the monitoring of various natural disasters in mountainous areas, such as landslide monitoring, and combined with machine learning models to predict and warn of landslides. The following is a brief explanation using landslide monitoring as an example.

[0058] S8. The drone performs detection of the mountain to be tested along the optimized track path of step S7, and uses the onboard camera to collect image data of the mountain. During the flight, the drone monitors its own position and speed in real time, and compensates for the position, attitude and speed deviations caused by environmental factors through control and adjustment. In order to meet the needs of later image stitching, the overlap rate between adjacent images on the same route must reach 25%, and the camera shooting speed must match the flight speed of the detection drone to ensure full coverage and repetition rate of the image. During the mission, the collected image data is transmitted to the central processing unit in the cabin of the drone cluster in real time through the image transmission module. When the drone flies to the end of the track, the camera stops collecting data on the mountain to be tested, and the drone automatically returns to the cabin along the track path.

[0059] S9. The central processing unit in the cabin of the drone cluster performs pre-processing such as cropping and image enhancement on the received images, and uses the pre-trained mountain crack recognition model to perform crack feature recognition and detection on the pre-processed images. If the collected images contain preset crack features, it is determined that there is a precursor to a landslide in the mountain and the process goes to step S10. If there are no preset crack features, step S9 is continued.

[0060] S10. Stitch the remaining collected images on the route where the crack is located, mark the area where the crack is located, and issue a disaster warning to the outside in a timely manner.

Claims

1. A multi-UAV collaborative path planning method for mountain monitoring, characterized in that: The following steps are involved: S1. Select several drones with the same structure to form a drone cluster. Each drone is equipped with a camera and a radar. The control signal of each drone in the drone cluster is generated based on the set drone kinematic model. S2. Construct a drone cluster formation based on the consistency control protocol, set a drone in the drone cluster as the pilot drone, and set the remaining drones as follower drones. When flying, the follower drones are arranged in a fixed order on the same side of the pilot drone; S3, the drone cluster formation set in control step S2 starts from the starting point, flies at a set speed and approaches the mountain to be measured, uses the camera carried by each drone to collect images of the mountain to be measured at preset time intervals, and marks the corresponding position information on each image of the mountain to be measured, uses the radar carried by each drone to measure the distance between each drone and the mountain to be measured in real time, and obtains the inclination angle and inclination direction of the mountain to be measured; by setting a distance threshold between the drone cluster and the mountain to be measured, when the distance between any drone in the drone cluster and the mountain to be measured is less than or equal to the distance threshold, the drone cluster stops collecting images of the mountain to be measured and returns to the starting point; S4, converting the image of the mountain to be measured obtained in step S3 into a grayscale image, extracting feature points of the mountain to be measured using the Harris corner detection algorithm, and marking the extracted feature points as mountain top feature points, valley feature points and mountain foot feature points according to the direction of gradient change, calculating the depth of each feature point to the corresponding drone according to the camera imaging principle, and obtaining the average coordinates of each feature point in the world coordinate system using the conversion matrix; S5, performing multivariate polynomial fitting according to the coordinates of each feature point obtained in step S4 in the world coordinate system to obtain a ridge curve function of the mountain to be measured, setting a ridge curve generation range, extracting ridge curve segments and connecting the endpoints of each ridge curve segment to form a closed profile of the mountain to be measured, and then performing coordinate transformation on the feature points on the profile of the mountain to be measured according to the inclination angle and inclination direction of the mountain to be measured obtained by the radar, and scanning the transformed feature points along the inclination direction to obtain a three-dimensional model of the mountain to be measured; S6. Take the mountain top feature points and valley feature points as task detection points, calculate the Manhattan distance between each UAV and each task detection point, take the minimum total detection distance as the goal, use the greedy algorithm to allocate detection tasks, so that each task detection point is assigned to a UAV, and the UAVs that are not assigned to the detection task stay at the starting point; S7. Using the three-dimensional model of the mountain to be tested obtained in step S5, selecting Manhattan distance as the evaluation function, and using the A-Star algorithm to perform three-dimensional path planning for each UAV assigned to the detection task in step S6.

2. The multi-UAV collaborative path planning method according to claim 1, characterized in that: The step S1 comprises: The second-order consistent control input of UAV i is obtained by using the second-order motion equation of UAV i. : ， Where: , , They are respectively UAV i in the world coordinate system The position coordinates in the x, y, and z directions, , , are the velocity vectors of drone i in the x, y, and z directions, respectively. , , are the second-order consistency control inputs of UAV i Components in the x, y, and z directions.

3. The multi-UAV collaborative path planning method according to claim 1, characterized in that: The step S2 comprises the following sub-steps: S21, calculating the relative distance between each follower drone and the pilot drone; S22, the pilot drone and each follower drone take off from the starting point in sequence and fly to a preset altitude, and then maintain the preset altitude; S23, setting the speed of each follower drone to be consistent with that of the pilot drone, and controlling the pilot drone to fly on the far left of the formation, and the follower drones are arranged in sequence on the right side of the pilot drone at a fixed distance in the order of numbers. According to the consistency control protocol with spacing in three-dimensional space, the control input of the follower drone is: ， Where: is the control input of the following drone i at time t; is a fixed gain; is the damping gain; is the connection strength between following drone i and following drone j; i and j are drone numbers; is the relative distance between the following UAV i and the pilot UAV at time t; is the relative distance between the following UAV j and the pilot UAV at time t; is the expected relative distance between any two UAVs; The world coordinate system of the following drone i and the pilot drone t at time The position distance on the lower z axis; The world coordinate system of the following drone j and the pilot drone t at all times The position distance on the lower z axis; , are the velocity amplitudes of following UAV i and following UAV j at time t respectively; is the total number of drones in the drone swarm; S24. Under the control input of the following drone, the following drone tracks the desired position by tracking the desired relative distance until the speed and position state of the drone cluster meets the consistency control protocol with spacing in three-dimensional space, thus completing the drone cluster formation.

4. The multi-UAV collaborative path planning method according to claim 1, characterized in that: The step S4 comprises the following sub-steps: S41, converting the image of the mountain to be measured into a grayscale image using a maximum value method; S42, applying the Harris corner detection algorithm to perform image segmentation on the grayscale image converted in step S41, and performing feature point detection on the segmented grayscale image to obtain all feature points in the grayscale image, and then marking the identified feature points as mountain top feature points, valley feature points and foot feature points according to the gradient change direction; S43, randomly selecting two grayscale images captured by two different drone cameras at the same time, using a stereo matching algorithm based on Census transformation to perform stereo matching on the same feature point in the two grayscale images, and obtaining the disparity of the same feature point in the two grayscale images; S44, using the disparity of the same feature point in the two grayscale images obtained in step S43, according to the camera imaging principle and the triangle similarity law, calculate the depth of the feature point from the camera carried by one of the two drones in step S43, and then obtain the three-dimensional coordinates of the feature point in the camera coordinate system, and then use the camera parameters to convert the three-dimensional coordinates of the feature point in the camera coordinate system into coordinates in the world coordinate system; S45, repeating steps S43 and S44, obtaining a coordinate set of the same feature point in the world coordinate system obtained by grayscale images collected by any two drones at each collection time, and calculating the average coordinate of the feature point in the world coordinate system; S46. Repeat steps S43 to S45 to obtain the average coordinates of all feature points in the world coordinate system.

5. The multi-UAV collaborative path planning method according to claim 4, characterized in that: The step S43 includes: using a stereo matching algorithm based on Census transformation to match the grayscale images collected by the two cameras, selecting a pixel corresponding to a feature point in a grayscale image , calculate the Census value and pixel value of each feature point in another grayscale image The Hamming distance of the Census value is obtained with the pixel The feature point pixel with the smallest Hamming distance , realize stereo matching of the same feature point in the grayscale images collected by two cameras, and then obtain the pixel point of the feature point on the two grayscale images , The parallax in the horizontal direction is given by the following formula: ， Where: To explore the feature points of the camera carried by drone i in the kth acquisition The corresponding pixel Census value; It is a bit-by-bit connection operation; For feature points Corresponding pixel Neighborhood window; q is the pixel in the neighborhood window; , The feature points Corresponding pixels And the gray value of the neighborhood pixel q; For comparison operations; ， Where: Pixel and The Hamming distance of To explore the feature points of the camera carried by UAV j in the kth acquisition The corresponding pixel Census value; E is the number of digits of Census transformation; is an XOR operation; It is stipulated that in the two grayscale images to be matched, the exploration drone i should be located on the left side of the exploration drone j; For comparison operations; ， Where: To explore the feature points in the image acquired by the camera carried by UAV i and j for the kth time The matching pixels and parallax; , The image pixel coordinate system matching point under , The horizontal pixel coordinate of .

6. The multi-UAV collaborative path planning method according to claim 5, characterized in that: The step S44 comprises: According to the camera imaging principle and the triangle similarity law, the depth of the feature point from the camera carried by the exploration drone i and its three-dimensional coordinates in the camera coordinate system are calculated. The formula is: ， Where: , , The feature points The grayscale image acquired by cameras i and j in the kth acquisition is obtained in the camera coordinate system The x, y, and z coordinates of the drone. i and j are the numbers of the two drones respectively. f is the camera focal length; is the expected relative distance between each UAV; , The feature points At camera i in the image pixel coordinate system The x and y coordinates of the following: To explore the feature points in the image acquired by the camera carried by UAV i and j for the kth time according to the camera imaging principle and the triangle similarity law The depth of the camera carried by the exploration drone i; , is the principal point coordinate of camera i; The camera's rotation matrix and translation matrix are used to construct a transformation matrix to transform the three-dimensional coordinates of the feature points in the camera coordinate system into coordinates in the world coordinate system: ， Where: For feature points In the kth acquisition, the grayscale image acquired by cameras i and j is obtained in the world coordinate system The coordinates below; For feature points The image acquired by cameras i and j in the kth acquisition is obtained in the camera coordinate system The coordinates below; is the rotation matrix of camera i, is the translation matrix of camera i, obtained from the camera parameters.

7. The multi-UAV collaborative path planning method according to claim 1, characterized in that: The step S5 comprises the following sub-steps: S51, performing multivariate polynomial fitting using the average coordinates of each feature point in the world coordinate system, obtaining optimal parameters of the multivariate polynomial by minimizing the error, and constructing a ridge curve function; S52, setting the range of generating the ridge curve according to the average coordinates of the feature points at the top of the mountain and the feature points at the foot of the mountain in the world coordinate system: ， Where: , They are the x-axis and y-axis coordinates of the mountain to be measured in the world coordinate system; is the average coordinate of the feature point 1 at the foot of the mountain in the x-axis direction in the world coordinate system, is the average coordinate of the feature point 2 at the foot of the mountain in the x-axis direction in the world coordinate system, is the average coordinate of the feature point i on the top of the mountain in the y-axis direction in the world coordinate system; S is the number of feature points on the top of the mountain; S53, extracting ridge curve segments according to the ridge curve function and the ridge curve generation range, and connecting the end points of the ridge curve segments to obtain a closed profile of the mountain to be measured; S54, according to the inclination angle and inclination direction of the mountain to be measured obtained by the radar, a reduction matrix and a rotation matrix about the y-axis are established to transform the feature points on the profile of the mountain to be measured. The rotation matrix and the reduction matrix are as follows: ， ， Where: is the rotation matrix about the y-axis; The inclination angle of the mountain to be measured is obtained by the radar; To reduce the matrix; , and They are the reduction ratios in the x, y, and z axis directions respectively; S55, scanning the characteristic points on the transformed profile of the mountain to be measured along the inclined direction to obtain a three-dimensional model of the mountain to be measured.

8. The multi-UAV collaborative path planning method according to claim 7, characterized in that: The step S5 further comprises: S56. Use Laplace smoothing to reduce the unnatural transition of the edge of the three-dimensional model of the mountain to be measured. The formula is: ， Where: is the point position that needs to be smoothed; is the point position after smoothing; For point The set of adjacent points of ; For collection size.

9. The multi-UAV collaborative path planning method according to claim 1, characterized in that: The step S7 comprises the following sub-steps: S71, rasterizing the three-dimensional space of a certain UAV assigned to the detection task, and determining the initial position coordinates and task detection point coordinates of the UAV; S72, using Manhattan distance as the evaluation function of the A-Star algorithm, starting from the initial position coordinates of the UAV, selecting the node with the minimum evaluation function among its adjacent nodes as the next central node, and repeating this selection method until the task detection point coordinates are reached; S73, tracing back from the coordinates of the mission detection point to construct a track path from the initial position coordinates of the UAV to the coordinates of the mission detection point; S74. Repeat steps S71 to S73 until all drones assigned to the detection task have completed the corresponding complete path planning.

10. The multi-UAV collaborative path planning method according to claim 9, characterized in that: The step S73 further includes: The track path is smoothed using a third-order Bezier curve. Each node on the track path obtained by the A-Star algorithm is used as the control point of the third-order Bezier curve. Four consecutive nodes are used as a group of control points. The formula of the third-order Bezier curve is as follows: ， Where: is the output point of the third-order Bezier curve; , , and are 4 consecutive Bezier control points; is the changing parameter.

Images