Landing control method, device and system based on multi-unmanned aerial vehicle cooperation

Abstract

The invention discloses a landing control method and device based on multi-unmanned aerial vehicle cooperation, and the method comprises the steps: enabling a plurality of unmanned aerial vehicles to execute a flight task, enabling the unmanned aerial vehicles to detect the GPS signal intensity, and switching to receive a Beidou signal when the GPS signal intensity does not meet a preset requirement; based on the Beidou signal, the position state of the unmanned aerial vehicle is calculated by adopting a Kalman filtering algorithm, and the motion state of the unmanned aerial vehicle is controlled based on a PID control algorithm; synchronizing the position state by each unmanned aerial vehicle, and judging whether a landing condition is met or not according to the position state; when the landing condition is met, the to-be-landed unmanned aerial vehicle acquires an environment image through an image acquisition device and identifies a landing identifier; and calculating a relative position with the to-be-landed unmanned aerial vehicle according to the center coordinate of the landing identifier, and controlling the to-be-landed unmanned aerial vehicle to land to a preset position according to the relative position. According to the method, the accuracy of unmanned aerial vehicle landing control can be ensured when the GPS signal is unstable.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a landing control method, device and system based on multi-unmanned aerial vehicle collaboration. Background Art

[0002] Regarding the take-off and landing technology of drones, the composite navigation method of GPS and visual guidance technology is currently mainly used to achieve fully autonomous stable flight and landing in the GPS signal area, which is the case when a single drone performs a flight mission. As the demand for inspection tasks becomes more and more refined, the operation time of drones increases. In order to improve efficiency, it is necessary to reduce the inspection operation time through multi-machine coordinated inspections. In some remote or complex environments of distribution network lines, how to ensure the accurate take-off and landing of drones at airports in multi-machine coordinated tasks has become a major challenge at present.

[0003] At present, the mainstream positioning method of take-off and landing of airport drones is GPS and visual assisted positioning. In the area covered by GPS signals, drones rely on GPS receivers to obtain accurate geographic location information to ensure the accuracy of the flight path. When the drone flies above the airport, it can achieve precise landing by visually identifying the visual markers (ApriTag identification code) that guide the landing. For example, patent text CN114104310A discloses a device and method for visually assisted drone landing based on GPS and AprilTag, in which the ground positioning device cyclically detects its own position and sends the information of the ground positioning device to the flight control device. The flight control device receives the information of the ground positioning device in real time and executes the speed and position series PID control strategy. When approaching the target, the camera detects in real time whether there is an AprilTag target in the field of view. If an AprilTag target appears, the visual and GPS fusion control strategy is executed to control the drone to land in a predetermined posture. If no AprilTag target appears, the AprilTag target is searched according to the position information of the ground positioning device.

[0004] However, when multiple drones are used to carry out inspection tasks in coordination, if the signal is poor or the mission location is in an area without signal, the GPS positioning accuracy is no longer reliable, which will cause a large deviation between the drone's flight position and the actual airport location, and the ApriTag identification code cannot be captured visually, causing the drone to land outside the airport. In addition, there are often large differences in the calibration and identification of specific markers by different drones, and when the flight distance of multiple drones is large, data synchronization is not timely, which can easily cause the drone to deviate and fail to achieve accurate landing. At the same time, inaccurate positioning data will also make it impossible to control the safe distance between drones, resulting in collision accidents. Summary of the invention

[0005] The present invention provides a landing control method, device and system based on multi-drone collaboration of unmanned aerial vehicles, which can ensure the landing accuracy of the unmanned aerial vehicles and avoid collision accidents when the GPS signal is poor.

[0006] A landing control method based on multi-UAV cooperation, comprising:

[0007] Multiple drones perform flight missions. The drones detect the GPS signal strength and switch to receiving Beidou signals when the GPS signal strength does not meet the preset requirements.

[0008] Based on the Beidou signal, the Kalman filter algorithm is used to calculate the position state of the UAV, and the motion state of the UAV is controlled based on the PID control algorithm;

[0009] Each drone synchronizes the position status and determines whether the landing condition is met according to the position status;

[0010] When the landing conditions are met, the UAV to be landed collects environmental images through an image acquisition device and identifies the landing mark;

[0011] The relative position of the landing mark to the unmanned aerial vehicle to be landed is calculated according to the central coordinates of the landing mark, and the unmanned aerial vehicle to be landed is controlled to land at a preset position according to the relative position.

[0012] Furthermore, the Kalman filter algorithm is used to calculate the position state of the drone, including:

[0013] When the Beidou signal is not updated, a state equation for a horizontal position vector of the UAV is calculated based on the acceleration vector and velocity vector of the UAV in the due north and due east directions;

[0014] When the BeiDou signal is updated, using the BeiDou signal as an observed quantity;

[0015] According to the state equation and the observed value, the state of the UAV is predicted and updated based on the Kalman filter to obtain the horizontal position of the UAV.

[0016] Furthermore, the motion state of the drone is controlled based on the PID control algorithm, including:

[0017] Initialize the error and integral terms of the PID controller;

[0018] Based on the current horizontal position of the drone output by the Kalman filter, calculate the error with the expected position;

[0019] The PID controller generates control parameters to control the motion state of the drone based on the error between the drone's current horizontal position and the desired position.

[0020] Furthermore, collecting environmental images and identifying landing marks include:

[0021] Performing grayscale processing and target area extraction on the environment image to generate a grayscale image;

[0022] Performing line segment detection on the grayscale image to obtain a line segment set;

[0023] Fitting the line segment set using the least square method to obtain an edge fitting image;

[0024] Quadrilateral detection is performed on the edge fitting image, and the quadrilateral area is determined to be a landing mark.

[0025] Furthermore, performing line segment detection on the grayscale image to obtain a line segment set includes:

[0026] Calculate the gradient value and gradient direction of each pixel of the grayscale image based on the Sobel operator;

[0027] Connecting every two adjacent pixels in the grayscale image into an edge;

[0028] Calculate the difference between the gradient values ​​of the two pixel points corresponding to each edge to obtain the gradient difference of the corresponding edge;

[0029] Calculate the difference in the gradient direction of the two pixel points corresponding to each edge to obtain the direction difference of the corresponding edge;

[0030] Determine whether the gradient difference and direction difference of the two edges meet the first merging condition. If the first merging condition is met, merge the two edges into an edge set.

[0031] It is determined whether the two edge sets satisfy the second merging condition. If the second merging condition is satisfied, the two edge sets are merged until they cannot be merged any further, thereby obtaining the line segment set.

[0032] Further, judging whether the two edge sets satisfy the second merging condition includes:

[0033] Calculate a first difference between a maximum gradient value and a minimum gradient value in a merged set after the two edge sets are merged, and a second difference between a maximum gradient direction and a minimum gradient direction;

[0034] If the first difference is smaller than the sum of the minimum gradient difference and the first constraint value in the two edge sets before the merger, and the second difference is smaller than the sum of the minimum direction difference and the second constraint value in the two edge sets before the merger, it is determined that the two edge sets meet the second merger condition.

[0035] Further, performing quadrilateral detection on the edge fitting image includes:

[0036] Determining a start end and an end end of each edge in the edge fitting image according to a reference direction;

[0037] Grouping the edges: if the distance between the end of one edge and the start of another edge is within a preset range and the direction of the connection line from the start to the end is the same as the reference direction, then the two edges are merged into one group to obtain a directed edge group;

[0038] All edges are used as first-layer nodes, and the directed edge group is used as lower-layer nodes to establish a directed edge tree;

[0039] The directed edge tree is traversed using a depth-first algorithm. When the last layer of the directed edge tree is traversed, if there is an edge that forms a loop with the first edge, it is determined that the traversed path is a quadrilateral.

[0040] Further, calculating the relative position of the UAV to be landed according to the center coordinates of the landing mark includes:

[0041] Calibrate the focal length of the UAV's image acquisition device in advance;

[0042] According to the focal length of the image acquisition device, a conversion relationship between the camera coordinate system and the image physical coordinate system is obtained;

[0043] According to the conversion relationship, the center coordinates of the landing mark in the camera coordinate system are converted into the center physical coordinates in the image physical coordinate system;

[0044] Convert the center physical coordinates of the landing mark in the image physical coordinate system into the center pixel coordinates in the pixel coordinate system;

[0045] Calculating the position offset of the drone relative to the center pixel coordinates according to the center pixel coordinates;

[0046] The position offset is subjected to attitude correction to obtain the relative position of the center of the landing mark and the UAV to be landed.

[0047] A landing control device based on multi-drone collaboration of unmanned aerial vehicles, comprising:

[0048] The signal detection module is used to detect the GPS signal strength and switch to receiving Beidou signal when it is confirmed that the GPS signal strength does not meet the preset requirements;

[0049] A calculation module, used to calculate the position state of the UAV based on the Beidou signal using a Kalman filter algorithm, and control the motion state of the UAV based on a PID control algorithm;

[0050] A synchronization module, used to synchronize the position status of each drone and determine whether the landing condition is met according to the position status;

[0051] The recognition module is used for the UAV to collect environmental images and recognize the landing mark through the image acquisition device when the landing conditions are met;

[0052] The landing control module is used to calculate the relative position of the drone to be landed according to the central coordinates of the landing mark, and control the drone to be landed to land at a preset position according to the relative position.

[0053] Furthermore, the calculation module uses the Kalman filter algorithm to calculate the position state of the drone, including:

[0054] When the Beidou signal is not updated, a state equation for a horizontal position vector of the UAV is calculated based on the acceleration vector and velocity vector of the UAV in the due north and due east directions;

[0055] When the BeiDou signal is updated, using the BeiDou signal as an observed quantity;

[0056] According to the state equation and the observed value, the state of the UAV is predicted and updated based on the Kalman filter to obtain the horizontal position of the UAV.

[0057] Furthermore, the calculation module controls the motion state of the drone based on the PID control algorithm, including:

[0058] Initialize the error and integral terms of the PID controller;

[0059] Based on the current horizontal position of the drone output by the Kalman filter, calculate the error with the expected position;

[0060] The PID controller generates control parameters to control the motion state of the drone based on the error between the drone's current horizontal position and the desired position.

[0061] Furthermore, the recognition module collects environmental images and recognizes landing marks, including:

[0062] Performing grayscale processing and target area extraction on the environment image to generate a grayscale image;

[0063] Performing line segment detection on the grayscale image to obtain a line segment set;

[0064] Fitting the line segment set using the least square method to obtain an edge fitting image;

[0065] Quadrilateral detection is performed on the edge fitting image, and the quadrilateral area is determined to be a landing mark.

[0066] Furthermore, the recognition module performs line segment detection on the grayscale image to obtain a line segment set, including:

[0067] Calculate the gradient value and gradient direction of each pixel of the grayscale image based on the Sobel operator;

[0068] Connecting every two adjacent pixels in the grayscale image into an edge;

[0069] Calculate the difference between the gradient values ​​of the two pixel points corresponding to each edge to obtain the gradient difference of the corresponding edge;

[0070] Calculate the difference in the gradient direction of the two pixel points corresponding to each edge to obtain the direction difference of the corresponding edge;

[0071] Determine whether the gradient difference and direction difference of the two edges meet the first merging condition. If the first merging condition is met, merge the two edges into an edge set.

[0072] It is determined whether the two edge sets satisfy the second merging condition. If the second merging condition is satisfied, the two edge sets are merged until they cannot be merged any further, thereby obtaining the line segment set.

[0073] Furthermore, the identification module determines whether the two edge sets meet the second merging condition, including:

[0074] Calculate a first difference between a maximum gradient value and a minimum gradient value in a merged set after the two edge sets are merged, and a second difference between a maximum gradient direction and a minimum gradient direction;

[0075] If the first difference is smaller than the sum of the minimum gradient difference and the first constraint value in the two edge sets before the merger, and the second difference is smaller than the sum of the minimum direction difference and the second constraint value in the two edge sets before the merger, it is determined that the two edge sets meet the second merger condition.

[0076] Furthermore, the recognition module performs quadrilateral detection on the edge fitting image, including:

[0077] Determining a start end and an end end of each edge in the edge fitting image according to a reference direction;

[0078] Grouping the edges: if the distance between the end of one edge and the start of another edge is within a preset range and the direction of the connection line from the start to the end is the same as the reference direction, then the two edges are merged into one group to obtain a directed edge group;

[0079] All edges are used as first-layer nodes, and the directed edge group is used as lower-layer nodes to establish a directed edge tree;

[0080] The directed edge tree is traversed using a depth-first algorithm. When the last layer of the directed edge tree is traversed, if there is an edge that forms a loop with the first edge, it is determined that the traversed path is a quadrilateral.

[0081] Furthermore, the landing control module calculates the relative position of the UAV to be landed according to the center coordinates of the landing mark, including:

[0082] Calibrate the focal length of the UAV's image acquisition device in advance;

[0083] According to the focal length of the image acquisition device, a conversion relationship between the camera coordinate system and the image physical coordinate system is obtained;

[0084] According to the conversion relationship, the center coordinates of the landing mark in the camera coordinate system are converted into the center physical coordinates in the image physical coordinate system;

[0085] Convert the center physical coordinates of the landing mark in the image physical coordinate system into the center pixel coordinates in the pixel coordinate system;

[0086] Calculating the position offset of the drone relative to the center pixel coordinates according to the center pixel coordinates;

[0087] The position offset is subjected to attitude correction to obtain the relative position of the center of the landing mark and the UAV to be landed.

[0088] A landing control system based on multi-drone collaboration includes multiple drones, each of which is provided with a processor and a storage device, wherein the storage device stores multiple instructions, and the processor is used to read the instructions and execute the above method.

[0089] The landing control method, device and system based on multi-drone collaboration provided by the present invention have at least the following beneficial effects:

[0090] (1) The Beidou combined with Kalman filtering and PID control method is used to solve the problem of collisions caused by inaccurate positioning when multiple drones land at the same airport after completing inspection tasks in areas without GPS signals;

[0091] (2) The quadrilateral detection algorithm based on pixel gradient can effectively identify landing marks, providing a basis for accurate landing and collision prevention;

[0092] (3) By calculating and controlling the position offset of the UAV, the UAV is ensured to land accurately at the preset position, thus improving the accuracy of the UAV landing control. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 The present invention provides a flowchart of an embodiment of a landing control method based on multi-UAV collaboration.

[0094] Figure 2The present invention provides a flowchart of an embodiment of position calculation and motion state control in a method for coordinated landing of multiple drones.

[0095] Figure 3 A flowchart of an embodiment of identifying a landing mark in a multi-drone coordinated landing control method based on unmanned aerial vehicles provided by the present invention.

[0096] Figure 4 A flowchart of an embodiment of line segment detection in a multi-drone coordinated landing control method based on unmanned aerial vehicles provided by the present invention.

[0097] Figure 5 A flowchart of an embodiment of quadrilateral detection in a multi-drone coordinated landing control method based on unmanned aerial vehicles provided by the present invention.

[0098] Figure 6 The present invention provides a flowchart of an embodiment of landing control in the method for coordinated landing control of multiple drones.

[0099] Figure 7 The present invention provides a flowchart of an embodiment of a landing control device based on multi-UAV collaboration. DETAILED DESCRIPTION

[0100] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0101] refer to Figure 1 In some embodiments, a landing control method based on multi-drone collaboration is provided, comprising:

[0102] S1. Multiple drones perform flight missions. The drones detect the GPS signal strength and switch to receiving Beidou signals when the GPS signal strength does not meet the preset requirements.

[0103] S2. Based on the Beidou signal, the Kalman filter algorithm is used to calculate the position state of the UAV, and the motion state of the UAV is controlled based on the PID control algorithm;

[0104] S3, each drone synchronizes the position status, and determines whether the landing condition is met according to the position status;

[0105] S4. When the landing conditions are met, the UAV to be landed collects environmental images through an image acquisition device and identifies the landing mark;

[0106] S5. Calculate the relative position of the landing mark to the drone to be landed according to the central coordinates of the landing mark, and control the drone to be landed to land at a preset position according to the relative position.

[0107] Specifically, in step S1, multiple drones perform flight missions and fly along pre-planned paths. During the flight, GPS signals are preferentially used for positioning, and GPS signal strength is detected in real time. When it is confirmed that the GPS signal strength does not meet the preset requirements, the Beidou signal is switched to be received. The GPS signal strength does not meet the preset requirements when the GPS signal strength is lower than a preset strength threshold.

[0108] Furthermore, in step S2, the reference direction provided by the Beidou signal includes the due north and due east directions; due to the low refresh rate of the Beidou signal, poor dynamic performance, and susceptibility to interference, it is difficult to provide high-precision navigation information for the aircraft when only using the Beidou signal for navigation. The PID algorithm controls data with a fast update speed and high accuracy in a short time, so in this embodiment, the two are combined to give full play to their respective advantages.

[0109] Specifically, refer to Figure 2 In step S2, the Kalman filter algorithm is used to calculate the position state of the UAV, including:

[0110] S21. When the Beidou signal is not updated, a state equation for a horizontal position vector of the UAV is calculated based on the acceleration vector and velocity vector of the UAV in the due north and due east directions;

[0111] S22. When the BeiDou signal is updated, use the BeiDou signal as an observed value;

[0112] S23. According to the state equation and the observation quantity, the state of the UAV is predicted and updated based on the Kalman filter to obtain the horizontal position of the UAV.

[0113] Specifically, in step S21, before the Beidou signal is updated, the movement of the drone in the navigation system for a short period of time can be regarded as uniformly accelerated linear motion, and the navigation information error of the next moment is not large. The acceleration vector of the drone in the due east and due north directions in the navigation system is defined as a=[a e a n ] T , the velocity vector is v = [v e v n ] T , the horizontal position vector is s=s e S n ] T , the system control period is T, then the state equation of the horizontal position vector at time k is:

[0114] s e (k) = s e (k-1)+v e (k-1)T+0.5a e(k-1)T 2 ; (1)

[0115] s n (k) = s n (k-1)+v n (k-1)T+0.5a n (k-1)T 2 ; (2)

[0116] The velocity vector is:

[0117] v e (k) = v e (k-1)+a e (k-1)T; (3)

[0118] v n (k) = v n (k-1)+a n (k-1)T; (4)

[0119] In the formula, S e (k) is the horizontal position vector in the east direction at time k, S n (k) is the horizontal position vector in the north direction at time k, S e (k-1) is the horizontal position vector in the east direction at time k-1, S n (k-1) is the horizontal position vector in the north direction at time k-1, v e (k) is the velocity vector in the east direction at time k, v n (k) is the velocity vector in the north direction at time k, v e (k-1) is the velocity vector in the east direction at time k-1, v n (k-1) is the velocity vector in the north direction at time k-1, a e (k) is the acceleration vector in the east direction at time k, a e (k) is the acceleration vector in the north direction at time k, a e (k-1) is the acceleration vector in the east direction at time k-1, a e (k-1) is the acceleration vector in the north direction at time k-1, and T represents the system control period.

[0120] Furthermore, in step S22, when the Beidou signal is updated, according to the Kalman filter principle, the horizontal position and speed of the aircraft in the east and north directions are taken as the state quantity x(k), that is:

[0121]

[0122] From equations (1)-(4), we can get the state equation at time k:

[0123] x(k)=Ax(k-1)+Bu(k)+Γw(k); (6)

[0124] Among them, x(k) is the state quantity at time k, u(k) represents the control input matrix, A represents the state transfer matrix, B represents the control matrix, Γ represents the noise matrix, and w(k) is the system process noise, which is the random acceleration brought by the environment.

[0125] Introducing the Beidou signal as the observed quantity y(k), the observation equation is as follows:

[0126] y(k)=Hx(k)+v(k); (7)

[0127] Among them, v(k) is the observation noise, which is a Gaussian white noise with a mean of zero and a variance of R. The variance value can be obtained from a large amount of observation data of the Beidou Power System. H is the observation matrix.

[0128] Further, in step S23, the state of the UAV is predicted and updated based on the Kalman filter to obtain the horizontal position of the UAV, which specifically includes:

[0129] Initialize the state and covariance matrix of the Kalman filter;

[0130] Use Kalman filter to predict the state at time k:

[0131]

[0132] in, represents the estimated value of the state prediction at time k, Represents the state prediction estimate at time k, the covariance matrix of the state prediction estimate at time k for:

[0133]

[0134] Where Q represents the process noise covariance matrix.

[0135] According to the observed value y(k), the residual is calculated with the state prediction estimate, the Kalman gain and covariance matrix are updated, and the updated state x(k|k) is obtained:

[0136] x(k∣k)=x(k∣k-1)+K(k)(y(k)-Hx(k-Δt / T∣k-Δt / T)); (10)

[0137] Among them, x(k|k) represents the updated state quantity, that is, the best estimated state quantity at the current moment, including the horizontal position and speed at the current moment, and K(k) is the Kalman gain.

[0138] Furthermore, in step S2, the motion state of the drone is controlled based on the PID control algorithm, including:

[0139] S24, initializing the error and integral term of the PID controller;

[0140] S25, calculating the error between the current horizontal position of the UAV and the expected position according to the output of the Kalman filter;

[0141] S26, the PID controller generates control parameters to control the motion state of the drone according to the error between the current horizontal position of the drone and the expected position.

[0142] Through the above method, the Beidou signal can be used to achieve stable flight of the drone when the GPS signal strength is poor.

[0143] Furthermore, in step S3, each drone synchronizes its position status, that is, each drone sends its own position status to other drones.

[0144] Specifically, judging whether the landing condition is met according to the position state includes:

[0145] S31. Determine whether the drone has reached a preset safety area based on its own position status;

[0146] S32. If the drone has reached the preset safety area, the drone controls itself to hover and determines whether other drones have reached the preset safety area or have landed.

[0147] S33: If the other UAVs have reached the preset safety area or have completed landing, it is determined that the landing conditions are met.

[0148] Specifically, for each drone, a safety area is set in advance, and the safety area may be an area within a preset range directly above the landing mark.

[0149] Further, refer to Figure 3 In step S4, collecting environmental images and identifying landing marks include:

[0150] S41, performing grayscale processing and target area extraction on the environment image to generate a grayscale image;

[0151] S42, performing line segment detection on the grayscale image to obtain a line segment set;

[0152] S43, fitting the line segment set using the least square method to obtain an edge fitting image;

[0153] S44, performing quadrilateral detection on the edge fitting image, and determining that the quadrilateral area is a landing mark.

[0154] Specifically, in actual applications, most landing marks are quadrilateral patterns, so this embodiment is described using a quadrilateral pattern.

[0155] In step S41, by gray-processing the environment image to generate a gray-scale image, the real-time performance of recognition and positioning can be improved, and problems such as stripe distortion and excessive processor calculation can be avoided. Since the landing mark is a quadrilateral pattern composed of black and white areas, the source image information is basically not lost after being converted into a gray-scale image.

[0156] Further, refer to Figure 4 In step S42, line segment detection is performed on the grayscale image to obtain a line segment set, including:

[0157] S421, calculating the gradient value and gradient direction of each pixel of the grayscale image based on the Sobel operator;

[0158] S422, connecting every two adjacent pixels in the grayscale image into an edge;

[0159] S423, calculating the difference between the gradient values ​​of two pixel points corresponding to each edge, and obtaining the gradient difference of the corresponding edge;

[0160] S424, calculating the difference in the gradient direction of two pixel points corresponding to each edge, and obtaining the direction difference of the corresponding edge;

[0161] S425, determining whether the gradient difference and the direction difference of the two edges satisfy a first merging condition, and if so, merging the two edges into an edge set;

[0162] S426: Determine whether the two edge sets satisfy a second merging condition. If so, merge the two edge sets until they cannot be merged any further, thereby obtaining the line segment set.

[0163] Specifically, in step S421, the Sobel operator is used to calculate the gradient value of each pixel point. For each direction of each pixel point, the direction in which the pixel value changes fastest is the gradient direction.

[0164] In step S422, every two adjacent pixels in the grayscale image can be connected to form an edge.

[0165] In step S423, the difference between the gradient values ​​of two pixel points corresponding to each edge is calculated, that is, the gradient values ​​of the two pixel points are subtracted to obtain the gradient difference of the corresponding edge.

[0166] In step S424, the difference in the gradient direction of two pixel points corresponding to each edge is calculated to obtain the direction difference of the corresponding edge.

[0167] In step S425, it is determined whether the gradient difference and direction difference of the two edges meet the first merging condition, specifically: the difference between the gradient difference and the direction difference of the two edges is calculated. If the difference between the gradient difference of the two edges is less than the first preset threshold, and the difference between the direction difference is less than the second preset threshold, it is determined that the first merging condition is met, and the two edges are merged into an edge set.

[0168] In step S426, determining whether the two edge sets meet the second merging condition includes:

[0169] Calculate a first difference between a maximum gradient value and a minimum gradient value in a merged set after the two edge sets are merged, and a second difference between a maximum gradient direction and a minimum gradient direction;

[0170] If the first difference is smaller than the sum of the minimum gradient difference and the first constraint value in the two edge sets before the merger, and the second difference is smaller than the sum of the minimum direction difference and the second constraint value in the two edge sets before the merger, it is determined that the two edge sets meet the second merger condition.

[0171] Specifically, assume that two edge sets are merged to form a merged set, obtain the maximum gradient value and the minimum gradient value of the pixel points in the merged set, subtract them to obtain the first difference, and then obtain the maximum gradient direction and the minimum gradient direction of the pixel points in the merged set, subtract them to obtain the second difference.

[0172] The minimum gradient value of the pixel points in the two edge sets is taken, and a first constraint value is determined. The first constraint value can be the product of the number of all edges in the merged set and a constant, and the range of the constant is 0-1. Similarly, the minimum gradient direction of the pixel points in the two edge sets is taken, and a second constraint value is determined. The second constraint value can be the product of the number of all edges in the merged set and a constant, and the range of the constant is 0-1, but the constants in the first constraint value and the second constraint value have different values.

[0173] If the first difference is smaller than the sum of the minimum gradient difference and the first constraint value in the two edge sets before the merger, and the second difference is smaller than the sum of the minimum direction difference and the second constraint value in the two edge sets before the merger, it is determined that the two edge sets meet the second merger condition.

[0174] The second merging condition makes the gradient values ​​and gradient directions of the two edge sets closer after the merging. The first constraint value and the second constraint value constrain the merging condition. The number of edges is used for conditional constraints. When the number of edges in the edge set is small, the merging condition is relatively loose. When the number of edges in the edge set is large, the merging condition becomes more stringent.

[0175] Furthermore, in step S43, after obtaining the line segment set, the line segment set is linearly fitted using the least square method to obtain multiple edges, and then obtain an edge-fitting image.

[0176] Further, refer to Figure 5 In step S44, quadrilateral detection is performed on the edge fitting image, including:

[0177] S441, determining a starting end and an ending end for each edge in the edge fitting image according to a reference direction;

[0178] S442, grouping the edges: if the distance between the end of one edge and the start of another edge is within a preset range and the direction of the connection line from the start to the end is the same as the reference direction, then merging the two edges into one group to obtain a directed edge group;

[0179] S443, taking all edges as first-layer nodes and the directed edge groups as lower-layer nodes, and establishing a directed edge tree;

[0180] S444. Use a depth-first algorithm to traverse the directed edge tree. When traversing to the last layer of the directed edge tree, if there is an edge that forms a loop with the first edge, determine that the traversed path is a quadrilateral.

[0181] Specifically, in step S441, the start and end of each edge in the edge fitting image are determined according to a reference direction, and the reference direction may be a clockwise direction or a counterclockwise direction, that is, the start and end of each edge are determined according to the same direction.

[0182] In step S422, by grouping the edges, each group of edges may be a candidate for forming a quadrilateral.

[0183] In steps S423 and S424, a quadrilateral can be identified by establishing a directed edge tree and performing a depth-first traversal.

[0184] Further, refer to Figure 6 In step S5, the relative position of the UAV to be landed is calculated according to the center coordinates of the landing mark, including:

[0185] S51, pre-calibrating the focal length of the image acquisition device of the UAV;

[0186] S52, obtaining a conversion relationship between a camera coordinate system and an image physical coordinate system according to the focal length of the image acquisition device;

[0187] S53, converting the center coordinates of the landing mark in the camera coordinate system into the center physical coordinates in the image physical coordinate system according to the conversion relationship;

[0188] S54, converting the central physical coordinates of the landing mark in the image physical coordinate system into the central pixel coordinates in the pixel coordinate system;

[0189] S55, calculating the position offset of the drone relative to the center pixel coordinates according to the center pixel coordinates;

[0190] S56: Perform attitude correction on the position offset to obtain the relative position of the center of the landing mark and the UAV to be landed.

[0191] Specifically, in steps S52 and S53, it is assumed that the coordinates of the landing mark in the camera coordinate system are Q (x, y, z), where the z value represents the vertical distance between point Q and point 0, and the absolute value of this value is also the height of the drone above the ground; the coordinates of point Q in the image physical coordinate system are Qc (Xc, Yx). Then the conversion relationship between the camera coordinate system and the image physical coordinate system is:

[0192]

[0193] Wherein, f is the focal length of the image acquisition device.

[0194] Furthermore, in order to estimate the relative position of the drone, it is necessary to transform the point Qc in the image physical coordinate system into the pixel coordinate system to obtain its pixel coordinate Q i (u, v), and then the position offset of the drone is calculated by the deviation between the pixel coordinates of the landmark and the pixel coordinates of the center point of the image to complete the relative position estimation of the drone.

[0195] In step S54, the conversion relationship between the image physical coordinate system and the pixel coordinate system is as follows:

[0196]

[0197] Where α and β are scale factors, representing the number of pixels corresponding to the physical distance, and the unit is pixel / mm. u0 and v0 represent the difference in the number of pixels between the pixel coordinates of the center point of the image and the pixel coordinates of the origin in the horizontal and vertical directions, which is usually half of the resolution.

[0198] Substituting formula (12) into formula (11) we obtain:

[0199]

[0200] Expanding formula (13) yields the position offset of the drone relative to the center pixel coordinate of the landing mark:

[0201]

[0202] Where x and y are the horizontal and vertical coordinates of the landmark in the horizontal heading coordinate system, and are also the position offset of the drone relative to the landing mark in the roll and pitch axis directions. When x and y are both zero, it indicates that the drone is directly above the landmark. At this time, the pixel coordinates of the landing mark are the coordinates of the center point of the image; when x or y is not zero, it indicates that the drone is not directly above the landmark. The visual position controller realizes position movement by adjusting the flight attitude, so that the values ​​of x and y gradually tend to zero, so that it can fly directly above the landmark and realize visual guidance of the drone autonomous flight. Considering that the horizontal attitude angle of the drone is not always zero during actual flight, the change of the body attitude will cause the camera angle to change, resulting in the center point of the image not being directly below the body, but the landing mark captured by the camera is in the center of the image. At this time, the position offset is in a non-biased state, no control amount is formed, and visual guidance cannot be performed. Therefore, in step S56, the position offset calculated by equation (14) needs to be corrected. Assume that during the flight, the current roll angle of the drone is θ, and the relative position of the center of the landing mark after attitude correction and the drone to be landed is:

[0203]

[0204] Further, in step S5, the drone to be landed is controlled to land at a preset position according to the relative position, including:

[0205] Keep the direction of the drone's nose unchanged, and use the PID control algorithm to control the drone's pitch angle and roll angle according to the relative position, so that the drone flies to just above the landing mark, and then controls the drone to descend vertically.

[0206] refer to Figure 7 In some embodiments, a landing control device based on multi-drone collaboration is also provided, including:

[0207] The signal detection module 201 is used to detect the GPS signal strength and switch to receiving Beidou signal when it is confirmed that the GPS signal strength does not meet the preset requirements;

[0208] A calculation module 202 is used to calculate the position state of the UAV based on the Beidou signal using a Kalman filter algorithm, and control the motion state of the UAV based on a PID control algorithm;

[0209] A synchronization module 203 is used to synchronize the position status of each drone and determine whether the landing condition is met according to the position status;

[0210] Identification module 204, used for the UAV to be landed to collect environmental images through an image acquisition device and identify landing marks when the landing conditions are met;

[0211] The landing control module 205 is used to calculate the relative position of the drone to be landed according to the central coordinates of the landing mark, and control the drone to be landed to land at a preset position according to the relative position.

[0212] Furthermore, the calculation module 202 uses the Kalman filter algorithm to calculate the position state of the drone, including:

[0213] When the Beidou signal is not updated, a state equation for a horizontal position vector of the UAV is calculated based on the acceleration vector and velocity vector of the UAV in the due north and due east directions;

[0214] When the BeiDou signal is updated, using the BeiDou signal as an observed quantity;

[0215] According to the state equation and the observed value, the state of the UAV is predicted and updated based on the Kalman filter to obtain the horizontal position of the UAV.

[0216] Furthermore, the calculation module 202 controls the motion state of the drone based on the PID control algorithm, including:

[0217] Initialize the error and integral terms of the PID controller;

[0218] Based on the current horizontal position of the drone output by the Kalman filter, calculate the error with the expected position;

[0219] The PID controller generates control parameters to control the motion state of the drone based on the error between the drone's current horizontal position and the desired position.

[0220] Furthermore, the recognition module 204 collects environmental images and recognizes landing marks, including:

[0221] Performing grayscale processing and target area extraction on the environment image to generate a grayscale image;

[0222] Performing line segment detection on the grayscale image to obtain a line segment set;

[0223] Fitting the line segment set using the least square method to obtain an edge fitting image;

[0224] Quadrilateral detection is performed on the edge fitting image, and the quadrilateral area is determined to be a landing mark.

[0225] Furthermore, the recognition module 204 performs line segment detection on the grayscale image to obtain a line segment set, including:

[0226] Calculate the gradient value and gradient direction of each pixel of the grayscale image based on the Sobel operator;

[0227] Connecting every two adjacent pixels in the grayscale image into an edge;

[0228] Calculate the difference between the gradient values ​​of the two pixel points corresponding to each edge to obtain the gradient difference of the corresponding edge;

[0229] Calculate the difference in the gradient direction of the two pixel points corresponding to each edge to obtain the direction difference of the corresponding edge;

[0230] Determine whether the gradient difference and direction difference of the two edges meet the first merging condition. If the first merging condition is met, merge the two edges into an edge set.

[0231] It is determined whether the two edge sets satisfy the second merging condition. If the second merging condition is satisfied, the two edge sets are merged until they cannot be merged any further, thereby obtaining the line segment set.

[0232] Furthermore, the identification module determines whether two edge sets meet the merging condition, including:

[0233] Calculate a first difference between a maximum gradient value and a minimum gradient value in a merged set after the two edge sets are merged, and a second difference between a maximum gradient direction and a minimum gradient direction;

[0234] If the first difference is smaller than the sum of the minimum gradient difference and the first constraint value in the two edge sets before the merger, and the second difference is smaller than the sum of the minimum direction difference and the second constraint value in the two edge sets before the merger, it is determined that the two edge sets meet the second merger condition.

[0235] Furthermore, the recognition module 204 performs quadrilateral detection on the edge-fitted image, including:

[0236] Determining a start end and an end end of each edge in the edge fitting image according to a reference direction;

[0237] Grouping the edges: if the distance between the end of one edge and the start of another edge is within a preset range and the direction of the connection line from the start to the end is the same as the reference direction, then the two edges are merged into one group to obtain a directed edge group;

[0238] All edges are used as first-layer nodes, and the directed edge group is used as lower-layer nodes to establish a directed edge tree;

[0239] The directed edge tree is traversed using a depth-first algorithm. When the last layer of the directed edge tree is traversed, if there is an edge that forms a loop with the first edge, it is determined that the traversed path is a quadrilateral.

[0240] Furthermore, the landing control module 205 calculates the relative position of the UAV to be landed according to the center coordinates of the landing mark, including:

[0241] Calibrate the focal length of the UAV's image acquisition device in advance;

[0242] According to the focal length of the image acquisition device, a conversion relationship between the camera coordinate system and the image physical coordinate system is obtained;

[0243] According to the conversion relationship, the center coordinates of the landing mark in the camera coordinate system are converted into the center physical coordinates in the image physical coordinate system;

[0244] Convert the center physical coordinates of the landing mark in the image physical coordinate system into the center pixel coordinates in the pixel coordinate system;

[0245] Calculating the position offset of the drone relative to the center pixel coordinates according to the center pixel coordinates;

[0246] The position offset is subjected to attitude correction to obtain the relative position of the center of the landing mark and the UAV to be landed.

[0247] In some embodiments, a landing control system based on multi-UAV collaboration is also provided, including multiple UAVs, each of which is provided with a processor and a storage device, the storage device stores multiple instructions, and the processor is used to read the instructions and execute the above method.

[0248] The method and device provided in the above embodiment have at least the following beneficial effects:

[0249] (1) The Beidou combined with Kalman filtering and PID control method is used to solve the problem of collisions caused by inaccurate positioning when multiple drones land at the same airport after completing inspection tasks in areas without GPS signals;

[0250] (2) The quadrilateral detection algorithm based on pixel gradient can effectively identify landing marks, providing a basis for accurate landing and collision prevention;

[0251] (3) By calculating and controlling the position offset of the UAV, the UAV is ensured to land accurately at the preset position, thus improving the accuracy of the UAV landing control.

[0252] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A landing control method based on multi-drone collaboration, characterized in that: include: Multiple drones perform flight missions. The drones detect the GPS signal strength and switch to receiving Beidou signals when the GPS signal strength does not meet the preset requirements. Based on the Beidou signal, the Kalman filter algorithm is used to calculate the position state of the UAV, and the motion state of the UAV is controlled based on the PID control algorithm; Each drone synchronizes the position status and determines whether the landing condition is met according to the position status; When the landing conditions are met, the UAV to be landed collects environmental images through an image acquisition device and identifies the landing mark; The relative position of the landing mark to the unmanned aerial vehicle to be landed is calculated according to the central coordinates of the landing mark, and the unmanned aerial vehicle to be landed is controlled to land at a preset position according to the relative position.

2. The method according to claim 1, characterized in that The Kalman filter algorithm is used to calculate the position status of the drone, including: When the Beidou signal is not updated, a state equation about the horizontal position vector of the UAV is calculated based on the acceleration vector and the velocity vector of the UAV in the due north and due east directions; When the BeiDou signal is updated, using the BeiDou signal as an observed quantity; According to the state equation and the observed value, the state of the UAV is predicted and updated based on the Kalman filter to obtain the horizontal position of the UAV.

3. The method according to claim 2, characterized in that Control the motion state of the drone based on the PID control algorithm, including: Initialize the error and integral terms of the PID controller; Based on the current horizontal position of the drone output by the Kalman filter, calculate the error with the expected position; The PID controller generates control parameters to control the motion state of the drone based on the error between the drone's current horizontal position and the desired position.

4. The method according to claim 1, characterized in that: Collect environmental images and identify landing signs, including: Performing grayscale processing and target area extraction on the environment image to generate a grayscale image; Performing line segment detection on the grayscale image to obtain a line segment set; Fitting the line segment set using the least square method to obtain an edge fitting image; Quadrilateral detection is performed on the edge fitting image, and the quadrilateral area is determined to be a landing mark.

5. The method according to claim 4, characterized in that Performing line segment detection on the grayscale image to obtain a line segment set includes: Calculate the gradient value and gradient direction of each pixel of the grayscale image based on the Sobel operator; Connecting every two adjacent pixels in the grayscale image into an edge; Calculate the difference between the gradient values ​​of the two pixel points corresponding to each edge to obtain the gradient difference of the corresponding edge; Calculate the difference in the gradient direction of the two pixel points corresponding to each edge to obtain the direction difference of the corresponding edge; Determine whether the gradient difference and direction difference of the two edges meet the first merging condition. If the first merging condition is met, merge the two edges into an edge set. It is determined whether the two edge sets satisfy the second merging condition. If the second merging condition is satisfied, the two edge sets are merged until they cannot be merged any further, thereby obtaining the line segment set.

6. The method according to claim 5, characterized in that Determining whether two edge sets satisfy the second merging condition includes: Calculate a first difference between a maximum gradient value and a minimum gradient value in a merged set after the two edge sets are merged, and a second difference between a maximum gradient direction and a minimum gradient direction; If the first difference is smaller than the sum of the minimum gradient difference and the first constraint value in the two edge sets before the merger, and the second difference is smaller than the sum of the minimum direction difference and the second constraint value in the two edge sets before the merger, it is determined that the two edge sets meet the second merger condition.

7. The method according to claim 4, characterized in that Performing quadrilateral detection on the edge-fitted image includes: Determining a start end and an end end of each edge in the edge fitting image according to a reference direction; Grouping the edges: if the distance between the end of one edge and the start of another edge is within a preset range and the direction of the connection line from the start to the end after the connection is the same as the reference direction, then the two edges are merged into one group to obtain a directed edge group; All edges are used as first-layer nodes, and the directed edge group is used as lower-layer nodes to establish a directed edge tree; The directed edge tree is traversed using a depth-first algorithm. When the last layer of the directed edge tree is traversed, if there is an edge that forms a loop with the first edge, it is determined that the traversed path is a quadrilateral.

8. The method according to claim 1, characterized in that Calculating the relative position of the UAV to be landed according to the center coordinates of the landing mark includes: Calibrate the focal length of the UAV's image acquisition device in advance; According to the focal length of the image acquisition device, a conversion relationship between the camera coordinate system and the image physical coordinate system is obtained; According to the conversion relationship, the center coordinates of the landing mark in the camera coordinate system are converted into the center physical coordinates in the image physical coordinate system; Convert the center physical coordinates of the landing mark in the image physical coordinate system into the center pixel coordinates in the pixel coordinate system; Calculating the position offset of the drone relative to the center pixel coordinates according to the center pixel coordinates; The position offset is subjected to attitude correction to obtain the relative position of the center of the landing mark and the UAV to be landed.

9. A landing control device based on multi-drone collaboration, characterized in that: include: The signal detection module is used to detect the GPS signal strength and switch to receiving Beidou signal when it is confirmed that the GPS signal strength does not meet the preset requirements; A calculation module, used to calculate the position state of the UAV based on the Beidou signal using a Kalman filter algorithm, and control the motion state of the UAV based on a PID control algorithm; A synchronization module, used to synchronize the position status of each drone and determine whether the landing condition is met according to the position status; The recognition module is used for the UAV to collect environmental images and recognize the landing mark through the image acquisition device when the landing conditions are met; The landing control module is used to calculate the relative position of the drone to be landed according to the central coordinates of the landing mark, and control the drone to be landed to land at a preset position according to the relative position.

10. A landing control system based on multi-drone collaboration, characterized in that: The method comprises a plurality of drones, each of which is provided with a processor and a storage device, wherein the storage device stores a plurality of instructions, and the processor is used to read the instructions and execute the method according to any one of claims 1 to 8.

Images