An indoor UAV formation control method based on relative positioning of airborne view

Abstract

The present invention belongs to the field of drone control technology, and specifically is a method for controlling indoor drone formations based on relative positioning of airborne perspective. The method of the present invention includes: the ground station predetermines the desired drone formation during indoor flight, including setting the desired distance and angle between the follower and the navigator, and planning the navigator's flight path according to the pre-set flight mission. The follower completes the relative positioning of the navigator's airborne tag during dynamic flight through the first-person perspective of the airborne camera, and does not require global positioning information. The system includes: a ground control station, a drone, and a communication device. The ground control station includes a visual image processing system and a formation controller, and the communication device includes a transceiver and a router. Compared with traditional formation schemes, the present invention only uses airborne visual sensors, has low hardware cost, good stability and a wide range of applications, and is a lightweight drone formation system.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle control, and in particular relates to an indoor unmanned aerial vehicle formation control method based on relative positioning of airborne viewing angles. Background Art

[0002] In recent years, with the increase in the diversity of drone application scenarios and missions, single drones are unable to meet increasingly complex needs. The concept of swarm intelligence was proposed in the 1980s, inspired by the group behavior patterns of social biological groups in nature, such as ant colonies and bee colonies. In 2020, the "White Paper on the Development of Artificial Intelligence" issued by the Chinese Academy of Sciences listed "swarm intelligence technology" as one of the eight key technologies in the field of artificial intelligence. As an important form of swarm intelligence, drone swarms integrate aerial robot technology with swarm intelligence technology, providing a new idea for the development of drones.

[0003] A drone swarm refers to a group of drones that complete tasks by interacting with other drones and the surrounding environment. It has redundancy, robustness, and scalability. A drone formation refers to adjusting the position state of each drone through interaction between drones during flight to achieve a specified geometric shape. To perform tasks based on drone formations, the information collection range can be optimized and the air resistance of the fleet can be reduced by designing the formation, making the drone swarm safer and more orderly, thereby improving the efficiency of mission execution. Drone formations include issues such as formation design, formation maintenance, relative positioning, information fusion, and formation controller design. Drone formations are the basis for studying drone swarms, so research on multi-drone formations has important theoretical and application value.

[0004] At present, for the research on indoor drone formations, since there is no GPS positioning system, the positioning of drones in the formation often adopts UWB positioning or multi-complex sensor fusion positioning. Although the above methods have high positioning accuracy, the positioning equipment is complicated to build, and the drones are equipped with many sensor devices, which easily causes unnecessary power consumption and increases the power requirements for drone flight. In addition, they can only fly in fixed areas with positioning systems. Since the load capacity and endurance of micro-single drones are limited to a certain extent, completing complex flight missions poses a huge challenge for single drones, especially micro-single drones. Therefore, it is difficult for traditional drone formations to solve simple and lightweight drone formation flight, and it is difficult to achieve stable flight in indoor areas. Summary of the invention

[0005] In order to solve the above problems, the present invention provides an indoor UAV formation control method based on relative positioning of airborne perspective to solve the problem that traditional UAV formations are simple and light, and lightweight UAV formation flight is difficult to achieve stable flight in indoor areas.

[0006] The indoor UAV formation control method based on airborne visual angle relative positioning provided by the present invention only uses a lightweight sensor, namely an airborne visual sensor, to obtain the relative position of the UAV, and designs a formation flight controller by controlling the sampling frequency and the flight speed, and completes the relative positioning by identifying the tag of the navigator, thereby realizing stable flight of the UAV formation; wherein:

[0007] The ground station communicates with each drone via wireless transmission, receives the drone’s flight status, and sends control instructions;

[0008] During the UAV formation flight, the UAV obtains the relative position information of the leader through its own onboard first-time visual sensor, and then guides the relative distance and angle between the followers and the leader through the formation control system to form a stable flight formation and complete the task together;

[0009] The formation controller obtains the current relative position and posture of the follower and the navigator based on the visual image processing system and calculates the speed information; according to the current speed and the field of view of the drone, the video image acquisition and transmission frequency is adjusted to ensure that the navigator always appears in the follower's field of view, thereby controlling the relative position and posture of the follower and the navigator through the kinematic model in the formation controller and the control method of removing the saturation limit, calculating the control amount required by the drone in real time and transmitting it to the actuator to ensure that the formation shape flies within a certain error range;

[0010] Based on this, the UAV formation can carry out smooth coordinated flight in an indoor environment with good WIFI communication.

[0011] The specific steps are:

[0012] Step S1, create different Apriltag (visual reference system) positioning identifiers through a printer and stick them on all drones in the formation.

[0013] Step S2: The drone takes off, opens the camera during flight to obtain the Apriltag positioning identifier of the navigator, completes relative positioning, and transmits the video stream information back to the ground station; the specific steps are as follows:

[0014] Step S21, sending a control command through the ground station to make all UAVs in the formation start flying at an initial speed, with the navigator flying according to a preset desired trajectory;

[0015] Step S22, the follower turns on the camera and transmits the visual information collected within a certain time interval to the ground station;

[0016] In step S23, the follower communicates with the ground station via wireless WIFI and transmits the video stream information to the ground station. The ground station uses the visual image processing system to obtain the flight parameters of the navigator in the current frame image, including the relative position of the navigator and the follower drone.

[0017] Step S3: The graphics workstation and formation controller located at the ground station convert the command into a control instruction and send it to the followers; the specific steps are:

[0018] Step S31, the visual image processing system transmits the analyzed relative position information of the leader and the follower to the formation controller, the input of the formation controller is the position information of the leader and the follower, and the output is the speed compensation amount;

[0019] In step S32, the ground station converts the output of the formation controller into a control command and sends it to the follower via wireless transmission. After receiving the command, the follower performs a corresponding action.

[0020] Step S4, during the flight, the sampling frequency of the image is adaptively adjusted according to the current formation flying speed to ensure that the leader always appears in the field of vision of the followers.

[0021] In the present invention, the ground station communicates with each drone through wireless transmission, receives the flight status of the drone, and sends control instructions. The communication system is stable and not easily interfered by external signals. The drones have fast response speed and good formation coordination.

[0022] In the present invention, the UAV obtains the relative position information of the leader through the first-time visual sensor on board during the formation flight, and then guides the relative distance and relative angle between the followers and the leader through the formation control system to form a stable flight formation and complete the task together. The UAV equipment is simple, low-load, does not require global positioning information, and can still maintain formation flight through visual positioning when subject to light or electromagnetic interference, which can improve the anti-interference ability of the formation flight control system and the efficiency of completing the task, and has broad application prospects.

[0023] In the present invention, the formation controller obtains the current relative position and posture of the follower and the navigator according to the visual image processing system, and calculates the speed information. According to the current speed and the field of view angle range of the drone, the video image acquisition and transmission frequency is adjusted to ensure that the navigator always appears in the follower's field of view, so as to control the relative position and posture of the follower and the navigator through the kinematic model in the formation controller and the control method of removing the saturation limit, calculate the control amount required by the drone in real time and transmit it to the actuating mechanism, so as to ensure that the formation shape flies within a certain error range.

[0024] The formation flight method of the present invention can realize a smooth coordinated flight of a UAV formation in an indoor environment with good WIFI communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is the drone formation according to an embodiment of the present invention.

[0026] Figure 2 It is the human-machine viewing angle of the embodiment of the present invention.

[0027] Figure 3 It is the Apriltag positioning identifier of the embodiment of the present invention.

[0028] Figure 4 The drone according to the embodiment of the present invention is affixed with an Apriltag positioning identifier.

[0029] Figure 5 The figure is a flowchart of the relative posture detection process of the UAV according to an embodiment of the present invention.

[0030] Figure 6 Schematic diagram of the visual positioning algorithm of the UAV formation according to an embodiment of the present invention.

[0031] Figure 7 1. It is a control diagram of the UAV formation controller according to an embodiment of the present invention.

[0032] Figure 8 The framework diagram of the indoor UAV formation control method based on relative positioning of the airborne perspective according to an embodiment of the present invention.

[0033] Fig. 9 This is a diagram showing the effect of determining the relative position of a formation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The indoor UAV formation control method based on relative positioning of airborne viewpoint of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0035] The ground station communicates with each drone via wireless transmission, receives the drone’s flight status, and sends control instructions;

[0036] During the UAV formation flight, the UAV obtains the relative position information of the leader through its own onboard first-time visual sensor, and then guides the relative distance and angle between the followers and the leader through the formation control system to form a stable flight formation and complete the task together;

[0037] The formation controller obtains the current relative position and posture of the follower and the navigator based on the visual image processing system and calculates the speed information; according to the current speed and the field of view of the drone, the video image acquisition and transmission frequency is adjusted to ensure that the navigator always appears in the follower's field of view, thereby controlling the relative position and posture of the follower and the navigator through the kinematic model in the formation controller and the control method of removing the saturation limit, calculating the control amount required by the drone in real time and transmitting it to the actuator to ensure that the formation shape flies within a certain error range;

[0038] Based on this, the UAV formation can carry out smooth coordinated flight in an indoor environment with good WIFI communication.

[0039] Step S1, create different Apriltag positioning identifiers through a printer and attach them to all drones in the formation.

[0040] Step S2: The drone takes off, opens the camera during flight to obtain the Apriltag positioning identifier of the navigator, completes relative positioning, and transmits the video stream information back to the ground station; the specific steps are as follows:

[0041] Step S21, sending a control command through the ground station to make all UAVs in the formation start flying at an initial speed, with the navigator flying according to a preset desired trajectory;

[0042] The kinematic model of the drone is:

[0043] Horizontal direction:

[0044]

[0045] Vertical direction:

[0046]

[0047] Assume that in the coordinate system of the drone body, the drone nose is forward as the X direction, the upward direction is the Z direction, and the Y direction is determined by the right-hand rule. Among them, X, Y, and Z are the coordinate positions in the x direction, y direction, and z direction respectively, υ and ω are the flight linear velocity and angular velocity of the drone, ψ is the angle between the flight direction of the drone and the X direction, i is the i-th drone (i=1, 2...n), and · represents the derivative.

[0048] Mathematical modeling of drone formations:

[0049]

[0050] See also Figure 1 , Figure 1 The figure shows the UAV formation model. d , θ are the expected distance and angle between the follower and the leader respectively. f, s are the actual distance, e f 、e s 、e ψ is the error between the expected and actual situation, f d for l d Component along the leader's flight direction, s d For l d In a right triangle with the hypotenuse as the angle and θ as the included angle, the right angle side corresponding to θ, x jis the x-coordinate, y-coordinate of the j-th follower j is the y coordinate of the jth follower, ψ j is the yaw angle of the jth follower.

[0051] In step S22, the follower turns on the camera and transmits the collected visual information to the ground station within a certain time interval.

[0052] See also Figure 2 , model the UAV vision, assuming that the UAV field of view is 60°, that is, 30° on the left and 30° on the right. The relationship between the angles can be described as:

[0053]

[0054] Among them, γ max is the maximum field of view of the drone. γ is the angle between the follower and the center of the camera. β is the angle between the flight direction and the x direction. α is the angle opposite to s in the right triangle formed by f and s. The subscript i represents the i-th follower (i = 0, 1, ..., n-1).

[0055] β i =ψ i -ψ j

[0056]

[0057] ψ i , j are the yaw angles of the leader and follower respectively. ω i ,ω j are the angular velocities of the leader and follower, respectively.

[0058] α i -γ max ≤β i ≤α i +γ max ; (6)

[0059] make

[0060] At the initial moment, the following conditions must be met:

[0061]

[0062] At any time during formation flying, the following conditions must be met:

[0063]

[0064] Taking the derivatives of both sides and simplifying them, we get:

[0065] α i -w max ≤wi ≤α i +w max , (10)

[0066] ω max For 50,000 people and maximum angular velocity.

[0067] The angular velocity saturation limit of the drone is:

[0068]

[0069] Step S23: The follower communicates with the ground station via WIFI and transmits the video stream information to the ground station. The ground station uses the visual image processing system to obtain the flight parameters of the navigator in the current frame image, including the relative position coordinates and flight attitude of the navigator and the follower drone. Figure 3 .

[0070] The Apriltag created by the printer is of the appropriate size (see Figure 4 ) is attached to the pilot UAV to be tracked. The follower transmits the collected visual information containing Apriltag to the ground visual image processing system at a certain time interval t. After the Apriltag is identified, encoded and decoded, it is assumed that the relative position of the pilot and the follower at time k is (x1, y1, z1), and the relative position at time k+t is (x2, y2, z2), where:

[0071] y2=y1+v y t

[0072] x2=x1+v x t; (12)

[0073] k moment:

[0074]

[0075] k+t moment:

[0076]

[0077] Limited by the drone's field of view, the following conditions must be met:

[0078]

[0079] According to the properties of the arctanx function, the above formula can be further simplified to:

[0080]

[0081] У1 and У2 represent the relative positions in the y direction, and У2 is eliminated by equation (12).

[0082] Since x1, y1, and ψ are constants, the relationship between the current flight speed V and the image sampling transmission time interval t can be calculated.

[0083] Step S3: The graphics workstation and formation controller located at the ground station convert the command into a control instruction and send it to the followers; the specific steps are:

[0084] Step S31, the visual image processing system transmits the analyzed relative position information of the leader and the follower to the formation controller, the input of the formation controller is the position information of the leader and the follower, and the output is the speed compensation amount;

[0085] The navigator flies along the ideal path, and the followers exchange information with the navigator through vision and compensate for speed based on the relative position. Each follower only needs to interact with the navigator and maintain its relative position with the host. Figure 5 .

[0086] According to the PD controller (see Figure 6 ), where K p is the proportionality coefficient, K d is the differential coefficient. The proportional differential (PD) controller is simplified by removing the influence of the integral (I) from the PID controller and combining only the proportional module and the differential module. The proportional differential controller can be used to control the track error, reduce the influence of the accumulated error during the flight, and provide good performance. Since the proportional differential controller needs to collect position information at a higher frequency for real-time feedback, and this method uses a vision-based positioning method to process the image, the image processing frequency is reduced in combination with the relationship between the flight speed and the frequency of adoption in S4 to meet the system performance requirements.

[0087] The follower adopts a discrete controller, and its control strategies are as follows. The input of the controller is the deviation between the distance between the two UAVs at time k and the ideal distance, and the output is the follower speed compensation.

[0088] x-direction:

[0089]

[0090] V x (k) = V x0 +K p ·[ΔV x +K d (ΔV x (k)-ΔV x (k-1)]; (19)

[0091] y direction:

[0092]

[0093] V y (k) = V y0 +K p ·[ΔV y +K d (ΔV y (k)-ΔV y (k-1)]; (21)

[0094] z-direction:

[0095] ΔV z =(z i (k)-z j (k)), (22)

[0096] V z (k) = V z0 +K p ·[ΔV z +K d (ΔV z (k)-ΔV z (k-1)], (23)

[0097] Angular velocity:

[0098] Δw i =(ψ i (k)-ψ j (k)), (24)

[0099] w z (k) = w i0 +K p ·[Δw i +K d (Δw i (k)-Δw i (k-1)], (25).

[0100] In step S32, the ground station uses the position-speed-attitude method to calculate the speed compensation and angular velocity compensation of the follower according to the error between the current position of the drone and the ideal position in S31, and sends the control instructions generated according to the compensation to the follower. After receiving the instructions, the follower performs the corresponding action.

[0101] Step S4, during the flight, the sampling frequency of the image is adaptively adjusted according to the current formation flying speed to ensure that the leader always appears in the field of vision of the followers.

[0102] According to the indoor UAV formation control method based on relative positioning of the airborne view provided in the above embodiment, the relative position between the leader and the follower can be quickly located in any indoor scene, so as to perform formation design. Fig. 9 .

[0103] Matters not covered by the present invention are known technologies.

[0104] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for controlling an indoor UAV formation based on relative positioning of airborne view, characterized in that: The relative position of the UAVs is obtained by using lightweight sensors, namely airborne visual sensors, and the formation flight controller is designed by controlling the sampling frequency and flight speed. The relative positioning is completed by identifying the leader's tag, and the stable flight of the UAV formation is realized; among them: The ground station communicates with each drone via wireless transmission, receives the drone’s flight status, and sends control instructions; During the UAV formation flight, the UAV obtains the relative position information of the leader through its own onboard first-time visual sensor, and then guides the relative distance and angle between the followers and the leader through the formation control system to form a stable flight formation and complete the task together; The formation controller obtains the current relative position and posture of the follower and the navigator based on the visual image processing system and calculates the speed information; according to the current speed and the field of view of the drone, the video image acquisition and transmission frequency is adjusted to ensure that the navigator always appears in the follower's field of view, thereby controlling the relative position and posture of the follower and the navigator through the kinematic model in the formation controller and the control method of removing the saturation limit, calculating the control amount required by the drone in real time and transmitting it to the actuator to ensure that the formation shape flies within a certain error range; Based on this, the UAV formation can achieve smooth coordinated flight in an indoor environment with good WIFI communication; The specific steps are: Step S1, creating different Apriltag positioning identifiers through a printer and attaching them to all drones in the formation; Step S2: The drone takes off, opens the camera during flight to obtain the Apriltag positioning identifier of the navigator, completes relative positioning, and transmits the video stream information back to the ground station; the specific steps are as follows: Step S21, sending a control command through the ground station to make all UAVs in the formation start flying at an initial speed, with the navigator flying according to a preset desired trajectory; Step S22, the follower turns on the camera and transmits the visual information collected within a certain time interval to the ground station; Step S23, the follower communicates with the ground station via wireless WIFI and transmits the video stream information to the ground station. The ground station uses the visual image processing system to obtain the flight parameters of the navigator in the current frame image, including the relative position of the navigator and the follower drone; Step S3, the graphics workstation and formation controller located at the ground station convert the command into a control instruction and send it to the followers; the specific steps are: Step S31, the visual image processing system transmits the analyzed relative position information of the leader and the follower to the formation controller, the input of the formation controller is the position information of the leader and the follower, and the output is the speed compensation amount; Step S32, the ground station converts the output of the formation controller into a control command, and sends it to the follower via wireless transmission. After receiving the command, the follower performs a corresponding action; Step S4, during the flight, the sampling frequency of the image is adaptively adjusted according to the current formation flying speed to ensure that the leader always appears in the field of vision of the followers; The process of step S2 is: S21, sending a control command through the ground station to make all the UAVs in the formation start flying at an initial speed, with the leader flying according to a pre-set desired trajectory; The kinematic model of the drone is: Horizontal direction: Vertical direction: Assume that in the coordinate system of the drone body, the drone nose is facing forward as the X direction, the upward direction is the Z direction, and the Y direction is determined by the right-hand rule; where X, Y, and Z are the coordinate positions in the x direction, y direction, and z direction, respectively, υ and ω are the flight linear velocity and angular velocity of the drone, ψ is the angle between the flight direction of the drone and the X direction, i is the i-th drone, i = 1, 2…n, and n is the number of drones; · represents the derivative; Mathematical modeling of drone formations: ideal: actual: error: l d , θ are the expected distance and angle between the follower and the leader respectively; f, s are the actual distance, e f 、e s 、e ψ is the error between the expected and actual situation, f d for l d Component along the leader's flight direction, s d For l d In a right triangle with the hypotenuse as the angle and θ as the included angle, the right angle side corresponding to θ, x j is the x-coordinate, y-coordinate of the j-th follower j is the y coordinate of the jth follower, ψ j is the yaw angle of the jth follower; S22, the follower turns on the camera and transmits the collected visual information to the ground station within a certain time interval; Model the drone vision. Assume that the drone field of view is 60°, 30° on each side. The relationship between the angles is described as: b i +g i =a i ,≤γ max =30° Among them, γ max is the maximum field of view of the drone, γ is the angle between the follower and the center of the camera; β is the angle between the flight direction and the x direction, α is the angle opposite to s in the right triangle formed by f and s as right-angled sides, and the subscript i represents the i-th follower, i = 0, 1, ..., n-1; b i =ψ i -ψ j ψ i , j are the yaw angles of the leader and follower respectively; ω i ,ω j are the angular velocities of the leader and follower respectively; a i -c max ≤β i ≤α i +g max (6) make At the initial moment, it satisfies: At any time during formation flying, the following conditions are met: Taking the derivatives of both sides and simplifying them, we get: a i -w max ≤ω i ≤α i +oh max , (10) ω max For 50,000 people and maximum angular velocity; The angular velocity saturation limit of the drone is: S23, the follower communicates with the ground station via WIFI and transmits the video stream information to the ground station. The ground station uses the visual image processing system to obtain the flight parameters of the navigator in the current frame image. The flight parameters include the relative position coordinates and flight attitude of the navigator and the follower drone; The Apriltag of appropriate size created by the printer is attached to the pilot drone to be tracked. The follower transmits the collected visual information containing the Apriltag to the ground visual image processing system at a certain time interval t. After the Apriltag is identified, encoded and decoded, it is assumed that the relative position of the pilot and the follower at time k is (x1, y1, z1), and the relative position at time k+t is (x2, y2, z2), where: y2=y1+v y t x2=x1+v x t; (12) k moment: k+t moment: Limited by the field of view of the drone, it meets the following requirements: According to the properties of the arctanx function, the above formula can be further simplified to: Since x1, y1, and ψ are constants, the relationship between the current flight speed V and the image sampling transmission time interval t can be calculated.

2. The indoor UAV formation control method according to claim 1, characterized in that: The process of step S3 is: S31, the visual image processing system transmits the analyzed relative position information of the leader and the follower to the formation controller, the input of the formation controller is the position information of the leader and the follower, and the output is the speed compensation amount; The leader flies along the ideal path, and the follower exchanges information with the leader through vision and compensates for speed based on the relative position; Each follower only needs to interact with the leader to maintain its relative position with the host; The formation controller adopts PD controller, assuming K p is the proportionality coefficient, K d is the differential coefficient; The follower adopts a discrete controller, and its control strategies are as follows. The input of the controller is the deviation between the distance between the two drones and the ideal distance at time k, and the output is the speed compensation of the follower: x-direction: V x (k)=V x0 +K p ·[ΔV x +K d (ΔV x (k)-ΔV x (k-1)]; (19) y direction: V y (k)=V y0 +K p ·[ΔV y +K d (ΔV y (k)-ΔV y (k-1)]; (21) z-direction: ΔV z =(z i (k)-z j (k)), (22) V z (k)=V z0 +K p ·[ΔV z +K d (ΔV z (k)-ΔV z (k-1)], (23) Angular velocity: Δw i =(ψ i (k)-ψ j (k)), (24) w z (k)=wi0+K p ·[Δw i +K d (Δw i (k)-Δw i (k-1)], (25) In S32, the ground station uses the position-speed-attitude method to calculate the speed compensation and angular velocity compensation of the follower according to the error between the current position of the drone and the ideal position in S31, and sends the control instructions generated according to the compensation to the follower. After receiving the instructions, the follower performs the corresponding action.

Images