Fixed-wing unmanned aerial vehicle cluster formation keeping method based on airborne vision

Abstract

The invention provides a fixed-wing unmanned aerial vehicle cluster formation keeping method based on airborne vision. Local state information is obtained through an airborne vision model; the unmanned aerial vehicles sense the cluster dynamic state in real time based on the airborne camera in an actual environment, each unmanned aerial vehicle receives cluster data with timestamps from all other nodes, and neighborhood unmanned aerial vehicle state information based on the airborne camera is obtained according to an airborne vision model and a cluster data sequence; utilizing short-term historical visual information to predict the speed direction of a neighborhood unmanned aerial vehicle, and utilizing expected formation configuration to construct a behavior rule set supporting airborne vision-based three-dimensional formation maintenance; and finally, solving an expected course in the current decision period according to the behavior rule set, converting the expected course into a waypoint, and sending the waypoint to a self-driving instrument to complete formation keeping. The method does not depend on inter-aircraft communication, so that the problem that the unmanned aerial vehicle cluster formation is difficult to maintain in a complex electromagnetic environment is solved.

Description

Technical Field

[0001] The present invention relates to the field of UAV cluster formation maintenance, and in particular to a fixed-wing UAV cluster formation maintenance method based on airborne vision. Background Art

[0002] Fixed-wing drone swarms demonstrate far superior combat advantages over single units in complex environments. They not only have distributed parallel computing and perception capabilities, but also have strong fault tolerance and robustness, thus achieving excellent combat effects in intelligence, surveillance and reconnaissance (ISR) missions and improving battlefield survivability and combat efficiency. In recent years, as the most common form of drone swarms, drone formations have become a research hotspot in the drone-related field, and formation maintenance is an important part of it. Drone swarm formation maintenance refers to the coordinated flight of two or more drones, which will maintain a certain geometric shape with each other.

[0003] Most existing formation-keeping control methods rely on wireless communication links for two-way information exchange. However, in practical applications, complex electromagnetic environments cannot guarantee effective inter-machine communication. Traditional methods also result in the size of UAV formations being constrained by communication, which seriously affects the stability of decision-making and the safety of the cluster. Researchers in biology and interdisciplinary fields have made great progress in explaining how bird flocks can adaptively fly in formation through visual signals in complex natural environments. Inspired by the visually driven behavior of birds, in recent years, researchers at home and abroad have taken the lead in exploring visual perception-based cluster formations on robot platforms, followed by research on applications on UAV platforms.

[0004] At present, the common methods of maintaining UAV cluster formation based on airborne vision are:

[0005] (1) Most formation control technologies can only be applied to rotary-wing UAV platforms. However, for fixed-wing UAVs, which have underactuated motion characteristics, the control of flight altitude is more complicated. The formation maintenance methods of a few fixed-wing UAVs are also mainly limited to two-dimensional planes, and there is a lack of in-depth research on formation maintenance methods in three-dimensional space.

[0006] (2) Artificial marking is used to assist visual positioning. Visual perception covers nearly omnidirectional vision. Although this improves the accuracy of visual information, it is difficult to install additional marking hardware on small fixed-wing UAVs. In addition, some luminous markers may become conspicuous targets in hostile environments, affecting the concealment of fixed-wing UAVs.

[0007] (3) The modeling is done by simply assuming that local information is available visually, and most of the experiments are verified only through numerical simulations, without considering the noise in the outdoor environment and the errors in visual perception.

[0008] At present, the research on fixed-wing UAV swarm formation maintenance methods based on airborne vision is still in the theoretical and simulation stage. Most methods will not consider the impact of the visual perception part and assume that accurate status information of neighboring UAVs can be obtained. Almost all research that adds front-end visual algorithms is based on rotorcraft UAVs with stronger controllability and more flexible sensor mounting methods. With the increase in environmental complexity, traditional methods based on inter-machine communication have gradually exposed their limitations, which makes the demand for UAV swarm formation maintenance methods based on airborne vision more and more obvious. Future research should focus more on developing UAV swarm control methods that do not rely on external communication, improving the adaptive ability of UAVs in complex environments, and thus achieving safer and more efficient formation maintenance. Summary of the invention

[0009] The purpose of the present invention is to provide a fixed-wing UAV cluster formation maintenance method based on airborne vision, which does not rely on inter-machine communication, so as to solve the problem that UAV cluster formation maintenance is difficult to achieve in complex electromagnetic environments.

[0010] To achieve the above object, the present invention provides a fixed-wing UAV cluster formation maintenance method based on airborne vision, comprising the following steps:

[0011] Firstly, an airborne vision model is established according to the airborne camera parameters, and local state information is obtained through the airborne vision model;

[0012] In the actual environment, drones perceive the cluster dynamics in real time based on their onboard cameras. Each drone receives cluster data with timestamps from all other nodes and obtains the status information of neighboring drones based on the onboard cameras according to the onboard visual model and cluster data sequence.

[0013] The speed and direction of neighboring UAVs are predicted using short-term historical visual information, and a set of behavioral rules that supports three-dimensional formation maintenance based on airborne vision is constructed using the expected formation configuration and local state information. The behavioral rule set includes visual guidance rules, waypoint traction rules, and collision avoidance rules between aircraft.

[0014] Finally, the desired heading in the current decision cycle is solved according to the behavioral rule set, and the desired heading is converted into a waypoint and sent to the autopilot to complete formation maintenance.

[0015] Preferably, the onboard vision model satisfies the visual constraints of the drones in the neighborhood:

[0016]

[0017] In the formula, UAV U i The position coordinates are p i ={x i ,y i ,zi}, UAV j The position coordinates are p j ={x j ,y j ,z j}, D is the maximum viewing distance of the onboard camera, γ h is the lateral field of view, γ v is the vertical field of view, r ij is the Euclidean distance between the two drones.

[0018] Preferably, the local state information includes relative distance and direction information, which is obtained by converting the true value of the drone position obtained in the simulation system:

[0019]

[0020] Among them, r is the estimated distance between the two aircraft, α is the angle between the two aircraft on the horizontal plane, β is the angle between the two aircraft on the vertical plane, and n is the visual noise.

[0021] Preferably, the state information of the neighboring drone based on the onboard camera is obtained, including

[0022] The follower perceives the state of the leader through the onboard visual sensor, obtains its simulated visual information, and estimates the relative distance and direction of the leader in the field of view based on this, and uses the body coordinate system based on the position of the aircraft to perform calculations. The relative distance and direction will be processed by the body coordinate conversion algorithm based on the position of the aircraft and converted into the absolute position coordinates of the neighboring UAV in the inertial coordinate system.

[0023] Preferably, the short-term historical visual information is used to predict the speed direction of the neighboring UAV, and the speed direction of the UAV in front of the field of view is predicted. k Historical location estimates Perform track fitting and Substitute the quasi-uniform B-spline curve, and the tangent line at the end of the curve points to the speed direction of the drone in the latest frame of the image, including

[0024] Quasi-uniform B-spline curve C(m):

[0025]

[0026] Where P i represents the control point, N i,k (m) represents the B-spline basis function;

[0027] UAV i Observation UAV k The historical location of its storage is estimated to be:

[0028]

[0029] If the drone is out of the machine's field of view, the drone will be cleared. i UAV k The short-term historical visual information of the drone u k The estimated speed at the current moment is

[0030] Preferably, visual guidance rules include

[0031] Navigator k Maintain the current heading and speed, and use visual perception to obtain the relative direction between the two aircraft to predict u k Next moment's position:

[0032]

[0033] Where τ is the unit prediction time step, v 0 τ represents the follower u i The radius of the optimal arc of all future state distributions, Indicates that u is in the north-east coordinate system i Estimated u k The location coordinates of Indicates u k Estimation of velocity direction;

[0034] The formation angle constraints δ and δ obtained according to the desired formation configuration v , respectively find the next moment follower u i Position coordinates on the horizontal plane And the position coordinates on the vertical plane

[0035] according to and The height in the three-dimensional space is obtained The heading obtained by the visual guidance rule is It is expressed as:

[0036]

[0037] If the follower drone in the formation does not meet the visual constraints with the friendly individuals in the neighborhood or the distance is too close, the visual guidance rule does not work, that is, the rule output is 0.

[0038] Preferably, the waypoint traction rules include:

[0039] According to the waypoint p M When generating the desired heading, the speed and direction of the drone are restricted to The heading obtained by the waypoint traction rule is described as follows:

[0040]

[0041] In the formula, is the threshold of heading deviation angle in each decision, For followers u i The position of the earth, ∠ is the angle between the velocity vector and the x-axis of the north-east coordinate system, is the final output.

[0042] Preferably, the collision avoidance rules between aircraft include:

[0043] To any follower drone u i and u j , the visual guidance rule will not apply after the collision avoidance is triggered, and the offset v ε , the course of the collision avoidance rules between aircraft for:

[0044]

[0045] In the formula, and are the position coordinates of the follower drone and the estimated position coordinates of the leader, r safe For safe distance between machines, is the velocity vector of the follower;

[0046] The outputs of the three rules are normalized, and the final output is obtained by linear combination based on the collision avoidance conditions between aircraft.

[0047]

[0048] Therefore, the present invention adopts the above-mentioned fixed-wing UAV cluster formation maintenance method based on airborne vision, and the technical effects are as follows:

[0049] (1) Traditional UAV swarm formation-keeping methods usually regard UAVs as being able to accurately obtain neighborhood individual information, and most studies are only focused on rotary-wing UAV platforms. This invention is proposed for fixed-wing UAV swarms, which not only takes into account the accuracy of state prediction under the influence of visual noise and delay, but also the safety between UAVs, and enables the swarm to have the ability to maintain formation in complex environments.

[0050] (2) The behavior-based decision-making method adopted by the present invention does not require two-way information exchange between UAVs and is applicable to situations where communication between UAVs fails in complex electromagnetic environments. Moreover, the algorithm can be completed offline and deployed on the UAV platform.

[0051] (3) The two-by-two leading-following formation control strategy proposed in the present invention can achieve three-dimensional formation maintenance, calculate the desired heading according to the set behavior rules, and add restrictions on altitude changes when converting it into waypoints, thereby ensuring the safety and stability of flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of a fixed-wing UAV cluster formation maintenance method based on airborne vision of the present invention;

[0053] Figure 2 This is a schematic diagram of the value of the simulated noise;

[0054] Figure 3 Configure for the desired formation;

[0055] Figure 4 is a schematic diagram of three-dimensional angle constraint; Figure 4 (a) is a schematic diagram of horizontal plane angle constraint; Figure 4 (b) is a schematic diagram of the vertical plane angle constraint. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0057] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0058] Embodiment 1

[0059] A fixed-wing UAV cluster formation maintenance method based on airborne vision comprises the following steps:

[0060] First, deploy the onboard cameras of fixed-wing drones, such as monocular cameras, binocular cameras, etc., and ensure that the optical parameters of the onboard cameras are correctly calibrated. Since the onboard camera has a limited field of view and cannot capture the positions of all neighboring drones at the same time, it is necessary to establish an onboard visual model that restricts the cluster data sequence through visual constraints.

[0061] Define the set of fixed-wing UAV clusters as U = {u 1,…, u n}, the leader in the cluster is at the front of the formation and cannot obtain the status information of the neighboring drones. The wingman obtains the first-person view image through the onboard camera. For target recognition of the first-person view image, two or more views can only estimate the relative distance and direction information of the neighboring individuals relative to the aircraft in the body coordinate system, that is, the scale factor is unknown and the absolute scale cannot be inferred. i and u j For example, the airborne vision model is that the UAVs in the neighborhood need to satisfy the visual constraints:

[0062]

[0063] In the formula, drone u i The position coordinates are p i ={x i ,y i ,z i}, drone u j The position coordinates are p j ={x j ,y j ,z j}, D is the maximum viewing distance of the onboard camera, γ h is the lateral field of view, γ v is the vertical field of view, r ij is the Euclidean distance between the two drones.

[0064] The simulated visual noise is a noise matrix generated by adding the error values ​​generated by motion blur and Gaussian model, which is recalculated for each decision. In practical applications, Gaussian noise is often used to simulate random disturbances or errors in various signals. The probability density function of Gaussian distribution is as follows:

[0065]

[0066] Where μ is the mean of the Gaussian distribution, σ 2 is the variance and x is the random variable.

[0067] Since the fixed-wing UAV has relative motion with the neighboring aircraft in the field of view during flight, the visual error caused by motion blur cannot be ignored. Assuming that the relative motion is uniform linear motion, the point spread function (PSF), also known as the blur kernel function, is expressed by the following formula:

[0068]

[0069] else:psf(x 1 ,x 2 )=0;

[0070] Where L is the length of the motion blur kernel, which is the motion scale within the exposure time in the pixel coordinate system and is related to the motion rate; (x 1 ,x 2 ) represents the coordinate position of the pixel point on the two-dimensional plane; the slope tanθ is determined by the direction of movement, and θ generally refers to the angle between the direction of movement and the x-axis; the judgment condition in the formula indicates that when the error matrix element is not in the relative motion direction, the value of the matrix element, i.e., the point spread function, is 0.

[0071] The local state information including relative distance and direction information is obtained through the airborne vision model, which is converted from the true value of the UAV position obtained in the simulation system:

[0072]

[0073] Among them, n is the visual noise, α and β are the inputs for visual decision-making in the subsequent formation maintenance model, r is the estimated distance between the two aircraft, α is the angle between the two aircraft on the horizontal plane, and β is the angle between the two aircraft on the vertical plane. The values ​​taken from the noise matrix follow the projection rules in the local field of view, so that the changes in relative position during flight will make the noise values ​​at each moment different, which is closer to the real visual perception error. Figure 2 As shown in the figure, the relative position between each follower UAV and its leader UAV determines the value in the noise matrix. The 11×11 table on the right represents the noise matrix generated by the algorithm. The gray circle represents the leader position in the UAV’s field of view, and the black circle represents its own position. v Represents the projection of the relative distance vector between the leader and the follower in the follower's field of view.

[0074] In the desired formation designed by the present invention, any two adjacent UAVs can form a small leader-follower formation as long as the visual constraints are met. The leader has the ability to move independently and does not need to rely on the feedback information of the follower, thereby enhancing the flexibility and robustness of the overall formation.

[0075] Through the above steps, the follower perceives the state of the navigator through the onboard visual sensor, obtains its simulated visual information, and estimates the position of the navigator in the field of view. In this way, the follower can make autonomous decisions and motion adjustments based on visual information without the need for additional communication. Using the body coordinate system based on the position of the aircraft for calculation makes it more convenient for the follower to process visual information and avoids the complex coordinate conversion process. In actual operation, the follower first determines the relative distance and direction between it and the navigator. This relative information will be processed by the coordinate conversion algorithm and converted into the absolute position coordinates of the neighboring drone in the inertial coordinate system.

[0076] For a small number of fixed-wing UAV clusters, common formations include straight, V-shaped, and triangular. Based on visual constraints, the present invention adopts a three-dimensional V-shaped formation as the formation configuration that is expected to be maintained, ensuring that any UAV in the formation and the UAVs in the neighborhood can form a small pilot-follower formation, such as Figure 3 shown.

[0077] Take the three-dimensional V-shaped formation of five drones as an example. 1 As the lead aircraft, the mathematical formula of its expected formation configuration can be expressed as:

[0078]

[0079] In the formula, (x iv ,y iv ,z iv ) is the drone u i The desired position in the formation, θ is the horizontal formation angle of the V-shaped formation, θ v is the vertical formation angle of the V-formation, r v is the Euclidean distance between UAVs in the formation, which can be called the desired formation distance between UAVs.

[0080] Since fixed-wing UAVs need to adapt to dynamically changing environments during flight, it is considered to introduce short-term historical frame images (short-term historical frame images) to provide more accurate decision-making information. Compared with other curve fitting methods, the quasi-uniform B-spline curve sets the repetition k at both ends of the node vector, so that it can show better local controllability at both ends of the curve. At the same time, the equidistant distribution of internal nodes makes the curve less likely to change drastically and can maintain a certain uniformity. Therefore, the quasi-uniform B-spline fitting method is used to fit the motion trend curve. As a special form of B-spline curve, the quasi-uniform B-spline fitting method generates a curve by parameterizing a set of control points and then using the B-spline basis function for weighted summation.

[0081] Define the spline basis function N i,k (m), for the 0th order basis function, satisfies the following equation:

[0082]

[0083] For the k-order basis function (k>0), the recursive formula is:

[0084]

[0085] In the formula, k represents the spline order, m i is an element in the node vector M.

[0086] The vector M can be expressed by the following mathematical formula:

[0087] M=a,…,a,m k ,…,m n+1 ,b,…,b;

[0088] Where a and b are boundary values, usually a=0, b=1; n+1 represents the number of control points; m k ,…,m n+1 It means the points to be fitted, also called control points, are generally distributed equidistantly.

[0089] The quasi-uniform B-spline curve C(m) is obtained by the B-spline basis function and the control points P i It is defined by the weighted sum of:

[0090]

[0091] Define u i Observation u k The historical location of its storage is estimated to be:

[0092]

[0093] According to the front view of the drone u k Historical location estimates Perform track fitting and Substitute the quasi-uniform B-spline curve, and the tangent line at the end of the curve points to the speed direction of the drone in the latest frame image. If the drone leaves the field of view of the aircraft, its historical frame information is cleared. At the same time, to ensure that the drone can be stably observed within the field of view, according to experience, the continuous observation time is at least 2 seconds. Finally, the drone u k The estimated velocities at the current moment are

[0094] The following is an introduction to the designed behavioral rule set:

[0095] (1) Visual guidance rules

[0096] The visual guidance rule mainly makes the relative angles of the follower and the leader in the UAV formation converge to the desired formation angle to achieve formation maintenance based on visual perception.

[0097] Predict the time step in units of τ and predict the followers u i All future states are distributed within a radius of v 0 On the superior arc of τ, the reverse extension of its own velocity vector in any state passes through u i Current position, assuming the navigator u k Keeping the current heading and speed, the relative direction between the two aircraft obtained by visual perception can predict u k Next moment's position:

[0098]

[0099] In the formula, Indicates that u is in the north-east coordinate system i Estimated u k The location coordinates of Indicates u k The estimation of speed direction is obtained by estimating the movement trend of the leader in the algorithm.

[0100] Taking the control on the horizontal plane as an example, assuming that the leader and the follower have the same altitude, the formation angle constraint δ is half of the desired formation angle, and the sign determination method of δ is determined by the follower UAV u i In the formation, relative to the leader u k The mathematical formula is as follows:

[0101]

[0102] In the formula, ||·|| represents the Euclidean norm, Represents u i current location.

[0103] To extend the control method on the two-dimensional plane to three dimensions, it is necessary to constrain δ according to the relative azimuth and formation angle on the vertical plane. v Determine the follower drone u i The height of the two-dimensional plane u i The prediction for the next moment is the same, such as Figure 4 As shown, the red dotted line represents u i To the leader u k The motion trajectory prediction for the next unit time step, the black dotted line represents the prediction of its future state.

[0104] In the three-dimensional direction, the formation angle constraints v and δ are obtained according to the desired formation configuration. v , respectively find the next moment follower u i Position coordinates on the horizontal plane And the position coordinates on the vertical plane

[0105]

[0106] In the formula, and for u i The projections of the current position coordinates on the horizontal and vertical planes, and for u i Prediction k The projections of the next position coordinates on the horizontal and vertical planes respectively, and ||·|| represents the Euclidean norm.

[0107] according to and The height in the three-dimensional space is obtained The heading obtained by the visual guidance rule is It can be expressed as:

[0108]

[0109] It should be noted that if the follower drone in the formation does not meet the visual constraints with the friendly individuals in the neighborhood or the distance is too close, the visual guidance rule will not work, that is, the rule output is 0.

[0110] (2) Waypoint traction rules

[0111] According to the waypoint p M When generating the desired heading, the speed and direction of the drone are limited to prevent the leader from disappearing from the followers’ field of view. The heading obtained by the waypoint traction rule is described as follows:

[0112]

[0113] In the formula, is the threshold of heading deviation angle in each decision, For followers u i The position of the earth, ∠ is the angle between the velocity vector and the x-axis of the north-east coordinate system, is the final output.

[0114] (3) Rules for avoiding collisions between aircraft

[0115] For the leader-follower UAV formation, an improved control rule is proposed to avoid collisions between drones. For distributed formations, if the collision avoidance rules are simply designed based on the relative distance between drones, individuals in the formation may avoid each other continuously, which increases the flight path of the drone cluster and affects the formation maintenance effect. In order to solve this problem, some researchers have added a hierarchical warning mechanism to the drone cluster inspired by the behavior rules of pigeon flocks, that is, low-level drones in the formation avoid high-level drones, while high-level drones do not participate in obstacle avoidance behavior. Therefore, based on the visual perception model, an obstacle avoidance mechanism is designed in which only followers avoid drones according to the relative position of the drones in the field of view, avoiding the chain reaction of drones avoiding each other.

[0116] For any drone u i and u j , the visual guidance rule will not apply after the collision avoidance is triggered. Assuming the offset v ε , the course of the collision avoidance rules between aircraft for:

[0117]

[0118] In the formula, and are the position coordinates of the follower drone and the estimated position coordinates of the leader, r safe For safe distance between machines, is the velocity vector of the follower;

[0119] It should be noted that the relative distance indicated here is obtained by simulated visual estimation and has certain errors.

[0120] The outputs of the three rules are normalized, and the final output is obtained by linear combination based on the collision avoidance conditions between aircraft.

[0121]

[0122] After obtaining the heading information through rule-based decision-making, the waypoints are calculated by multiplying the fixed time step with the unit velocity vector. In addition to adapting to the track on the horizontal plane, the control method of the three-dimensional formation also needs to consider the height change in the vertical direction. Because the algorithm takes into account the time delay of visual perception in actual situations, the drone cannot respond to the decision immediately, and it is more sensitive to height changes, and may climb or descend too much. Therefore, adding a limit on the amplitude of the waypoint height change in the algorithm improves the stability of the algorithm in three-dimensional control and ensures the safety of the drone during flight.

[0123] The evaluation indicators of drone cluster formation maintenance in traditional methods include the distance between aircraft, the error value between the actual position and the expected position, and the integral accumulation of the absolute value of the error over time. For the fixed-wing drone cluster formation maintenance method based on airborne vision proposed in this invention, only relative direction information is used for decision making in the algorithm, so position-related evaluation indicators are not used. Therefore, in order to evaluate the performance of the three-dimensional formation maintenance algorithm based on visual perception, the design indicators are as follows:

[0124] (1) Minimum distance between machines

[0125]

[0126] Minimum distance between machines r min is the minimum distance between a UAV and any UAV in its neighborhood during the formation maintenance process. If r min If the value of is always greater than twice the length of the wingspan, it means that the algorithm can achieve safe formation maintenance and there is no risk of collision between aircraft during formation flight.

[0127] (2) Extreme value of formation angle error

[0128] The formation angle error refers to the difference between the actual relative angle between the leader and the follower and the expected formation angle. Taking the minimum value as an example, the UAV u i The extreme value of the formation angle error can be obtained by the following formula:

[0129]

[0130] In the formula, drone ui and droneu j Satisfying the visual constraints, the leader and follower are θ and θ respectively. v are the UAVs u in the expected formation configuration i The smaller the formation angle error extreme value, the stronger the reliability of the formation keeping algorithm.

[0131] (3) Root mean square error of formation angle

[0132] The root mean square error is the square root of the ratio of the actual value to the expected value and the number of squares of the values, expressed as θ i and Represents drone u i The actual value of the formation angle, m is the number of times the value is taken, and the index can be obtained by the following formula:

[0133]

[0134] The smaller the RMS error of the formation angle, the higher the accuracy of the formation keeping algorithm.

[0135] Therefore, the present invention adopts the above-mentioned fixed-wing UAV cluster formation maintenance method based on airborne vision, which does not rely on inter-machine communication, to solve the problem that UAV cluster formation maintenance is difficult to achieve in complex electromagnetic environments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A fixed-wing UAV cluster formation maintenance method based on airborne vision, characterized in that: The following steps are involved: Firstly, an airborne vision model is established according to the airborne camera parameters, and local state information is obtained through the airborne vision model; In the actual environment, drones perceive the cluster dynamics in real time based on their onboard cameras. Each drone receives cluster data with timestamps from all other nodes and obtains the status information of neighboring drones based on the onboard cameras according to the onboard visual model and cluster data sequence. The speed and direction of neighboring UAVs are predicted using short-term historical visual information, and a set of behavioral rules that supports three-dimensional formation maintenance based on airborne vision is constructed using the expected formation configuration and local state information. The behavioral rule set includes visual guidance rules, waypoint traction rules, and collision avoidance rules between aircraft. Finally, the desired heading in the current decision cycle is solved according to the behavioral rule set, and the desired heading is converted into a waypoint and sent to the autopilot to complete formation maintenance.

2. According to the method of maintaining fixed-wing UAV cluster formation based on airborne vision in claim 1, it is characterized in that: The onboard vision model satisfies the visual constraints for drones in the neighborhood: In the formula, UAV U i The position coordinates are p i ={x i ,y i ,z i }, UAV j The position coordinates are p j ={x j ,y j ,z j }, D is the maximum viewing distance of the onboard camera, γ h is the lateral field of view, γ v is the vertical field of view, r ij is the Euclidean distance between the two drones.

3. The method for maintaining fixed-wing UAV cluster formation based on airborne vision according to claim 1 is characterized in that: The local state information includes relative distance and direction information, which is obtained by converting the true value of the drone position obtained in the simulation system: Among them, r is the estimated distance between the two aircraft, α is the angle between the two aircraft on the horizontal plane, β is the angle between the two aircraft on the vertical plane, and n is the visual noise.

4. The method for maintaining fixed-wing UAV cluster formation based on airborne vision according to claim 1 is characterized in that: Obtain the status information of the neighboring drone based on the onboard camera, including The follower perceives the state of the leader through the onboard visual sensor, obtains its simulated visual information, and estimates the relative distance and direction of the leader in the field of view based on this, and uses the body coordinate system based on the position of the aircraft to perform calculations. The relative distance and direction will be processed by the body coordinate conversion algorithm based on the position of the aircraft and converted into the absolute position coordinates of the neighboring UAV in the inertial coordinate system.

5. The method for maintaining fixed-wing UAV cluster formation based on airborne vision according to claim 1 is characterized in that: The short-term historical visual information is used to predict the speed direction of the neighboring UAVs. k Historical location estimates Perform track fitting and Substitute the quasi-uniform B-spline curve, and the tangent line at the end of the curve points to the speed direction of the drone in the latest frame of the image, including Quasi-uniform B-spline curve C(m): Where P i represents the control point, N i，k (m) represents the B-spline basis function; UAV i Observation UAV k The historical location of its storage is estimated to be: If the drone is out of the machine's field of view, the drone will be cleared. i UAV k The short-term historical visual information of the drone u k The estimated speed at the current moment is 6. The method for maintaining fixed-wing UAV cluster formation based on airborne vision according to claim 1 is characterized in that: Visual guidance rules, including Navigator k Maintain the current heading and speed, and use visual perception to obtain the relative direction between the two aircraft to predict u k Next moment's position: Where τ is the unit prediction time step, v0τ represents the follower u i The radius of the optimal arc of all future state distributions, Indicates that u is in the north-east coordinate system i Estimated u k The location coordinates of Indicates u k Estimation of velocity direction; The formation angle constraints δ and δ obtained according to the desired formation configuration v , respectively find the next moment follower u i Position coordinates on the horizontal plane And the position coordinates on the vertical plane according to and The height in the three-dimensional space is obtained The heading obtained by the visual guidance rule is It is expressed as: If the follower drone in the formation does not meet the visual constraints with the friendly individuals in the neighborhood or the distance is too close, the visual guidance rule does not work, that is, the rule output is 0.

7. The method for maintaining fixed-wing UAV cluster formation based on airborne vision according to claim 1 is characterized in that: Waypoint towing rules, including According to the waypoint p M When generating the desired heading, the speed and direction of the drone are restricted to The heading obtained by the waypoint traction rule is described as follows: Where θ is the threshold of heading deviation angle in each decision, For followers u i The position of the earth, ∠ is the angle between the velocity vector and the x-axis of the north-east coordinate system, is the final output.

8. The method for maintaining fixed-wing UAV cluster formation based on airborne vision according to claim 1 is characterized in that: Rules for avoiding collisions between aircraft include: To any follower drone u i and u j , the visual guidance rule will not apply after the collision avoidance is triggered, and the offset v ε , the course of the collision avoidance rules between aircraft for: In the formula, and are the position coordinates of the follower drone and the estimated position coordinates of the leader, r safe For safe distance between machines, is the velocity vector of the follower; The outputs of the three rules are normalized, and the final output is obtained by linear combination based on the collision avoidance conditions between aircraft.

Images