A method for relative positioning of unmanned aerial vehicle formation

Abstract

This invention discloses a method for relative positioning of UAV formations, comprising the following steps: S1, installing an industrial camera on the UAV; after installation, using a three-dimensional measurement method to obtain the positional relationship between the UAV surface feature points and the aircraft center; using the industrial camera to acquire initial images; and performing a self-test on the industrial camera to obtain a UAV swarm equipped with an image acquisition device; S2, using the UAV swarm equipped with the image acquisition device to acquire images, obtaining continuous flight images; S3, performing data processing based on the initial images and continuous flight images, and completing the relative positioning of the UAV formation based on the data processing results. This invention only needs to extract the three-dimensional coordinates of necessary feature points from two corresponding images, requiring less information and having a fast processing speed, which can well meet real-time requirements and is especially suitable for dynamic situations.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a method for relative positioning of UAV formations. Background Technology

[0002] Due to their unique advantages such as fast response speed, low operating cost, and flexible deployment, drones are widely regarded as an important platform for future information technology development. However, due to environmental complexity and mission diversity, a single drone often cannot meet many practical mission requirements. Furthermore, due to limitations in the number of onboard devices, perception viewpoint, and range, a single drone is typically unable to perform tasks such as continuous target tracking and all-around saturation attacks. Therefore, drone swarms working collaboratively to execute missions are gradually becoming a trend.

[0003] The existing solutions for the relative positioning of drone swarms are mainly solved through wireless communication. However, due to the scarcity of communication resources and the presence of dense electromagnetic interference, existing solutions are usually slow and lack stealth, and cannot adequately meet the positioning needs of drone swarms. Summary of the Invention

[0004] In view of the above-mentioned shortcomings in the prior art, the present invention provides a method for relative positioning of UAV formations, which solves the problems of slow processing speed and poor concealment of the existing solutions.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: A method for relative positioning of unmanned aerial vehicle (UAV) formations is provided, comprising the following steps:

[0006] S1. Install an industrial camera on the drone. After installation, use a 3D measurement method to obtain the positional relationship between the drone's surface feature points and the aircraft's center. Use the industrial camera to acquire initial images and perform a self-test on the industrial camera to obtain a drone swarm equipped with image acquisition equipment.

[0007] S2. Use a swarm of drones equipped with image acquisition devices to acquire images and obtain continuous flight images;

[0008] S3. Perform data processing based on the initial image and continuous flight images, and complete the relative positioning of the UAV formation based on the data processing results.

[0009] Further: S11, Securely install an industrial camera on each of the front two sides of each drone, and after installation, a drone equipped with an image acquisition device is obtained;

[0010] S12. Use a scanner to acquire 3D point cloud data of a single drone, and use an additional industrial camera of the same specifications to capture images of the single drone from different angles as initial images.

[0011] S13. Perform SURF feature recognition on the initial image, map the recognized UAV fuselage surface feature points to the three-dimensional feature point data of a single UAV obtained in step S1, and obtain the correspondence function between the surface feature points of a single UAV and the center of the aircraft.

[0012] S14. Establish the coordinate system D-XYZ for the navigation drone and the coordinate system D′-X for the non-navigation drone. t Y t Z t and camera coordinate system S-xyz;

[0013] Among them, the D-XYZ coordinate system of the navigation drone takes the center of the navigation drone as the origin, the longitudinal axis of the drone is the X-axis, the Y-axis is perpendicular to the X-axis and points to the left wing, and the Z-axis is perpendicular to the XY plane and points upwards as positive;

[0014] Non-navigational UAV coordinate system D′-X t Y t Z t With the center of the non-navigation drone as the origin, and the longitudinal axis of the aircraft as X... t axis, Y t The axis is perpendicular to X. t The axis points to the left wing, Z t The axis is perpendicular to X. t Y t A plane pointing upwards is considered positive.

[0015] Camera coordinate system S-xyz: The camera coordinate system is established with the center of the onboard camera as the origin; the x-axis and y-axis are parallel to the x-axis and y-axis of the physical image coordinate system, and the z-axis coincides with the optical axis;

[0016] S15. Calculate the coordinates of each surface feature point of the pilot drone in the pilot drone coordinate system based on the correspondence function between the surface feature points of a single drone and the center of the aircraft, and pre-set the coordinates of each surface feature point of the pilot drone in the pilot drone coordinate system and the initial image in each drone.

[0017] S16. Perform a self-test on the onboard industrial camera to confirm that the data can be returned correctly, thus obtaining the drone swarm equipped with image acquisition equipment.

[0018] Further: In step S11, the installation requirements for the industrial cameras are: the observation range of the two industrial cameras of each UAV can cover a 180-degree field of view in front of the UAV and they are installed symmetrically.

[0019] The beneficial effects of the above-mentioned further solutions are: using industrial cameras has the advantages of being lightweight, low-power, and small in size, and visual positioning relies solely on natural light, which also provides high concealment.

[0020] Further: Step S3 includes the following sub-steps:

[0021] Step S3 includes the following sub-steps:

[0022] S33. Based on the SURF method, perform feature matching between the flight images of the other non-leader drones and the initial image to obtain the three-dimensional coordinates of some feature points on the surface of the leader drone in the leader drone coordinate system.

[0023] The coordinates of the three-dimensional coordinates are represented as: (X i ,Y i Z i (i = 1, 2, 3...);

[0024] S34. Based on the coordinates of no fewer than 6 feature points in the navigation UAV coordinate system and the corresponding image coordinates in the image coordinate system, determine the external parameters X of the computer-mounted camera. S ,Y S Z S ,ω, κ;

[0025] Among them, (X) S ,Y S Z S ( ) represents the coordinates of the airborne camera's photography center in the coordinate system of the navigation drone. These are the three rotation angles from coordinate system S-xyz to coordinate system D-XYZ;

[0026] S35. Based on the relationship between the center of the airborne camera and the center of the UAV, perform a transformation to obtain the coordinate information (X) of the UAV's own center in the coordinate system of the pilot UAV. St ,Y St Z St ) and from coordinate system D′-X t Y t Z t The three corners to coordinate system D-XYZ And according to (X) St ,Y St Z St )and Complete the relative positioning of the drone formation.

[0027] The advantages of the above-mentioned further solution are: there is no need for UAVs to communicate through inter-UAV channels, which reduces the consumption of communication resources; and only the necessary three-dimensional coordinates of feature points need to be extracted from two corresponding images, which requires less information and has a fast processing speed, and can well meet the real-time requirements, especially suitable for dynamic situations.

[0028] Further: Step S34 includes the following sub-steps:

[0029] S32-1. Using the collinearity equation of photographic transformation, the coordinates of any point in the world coordinate system and its corresponding image coordinates in the image coordinate system are expressed by the following formula:

[0030]

[0031] in, Let these be the homogeneous coordinates of the object point in the world coordinate system. Z represents the homogeneous coordinates of the corresponding image point in the image coordinate system. C M is the object distance from a point in space to the center of the camera's image, and AB is the projection matrix. The intrinsic parameter matrix, The extrinsic parameter matrix;

[0032] Among them, F x =f / dx, F y = f / dy, where f represents the camera's focal length, dx and dy represent the length and width per pixel, respectively, and C x C y R represents the coordinates of the principal point of the image in the image coordinate system; R and t are the rotation and translation matrices in the camera's extrinsic parameters, and T represents the transpose.

[0033] S32-2. The projection matrix M is represented using the relationship between the world coordinate system and the image coordinate system. Its expression is as follows:

[0034]

[0035] Among them, l1-l 12 Let l be the coefficient relating the world coordinate system and the image coordinate system. 12 =1;

[0036] S32-3. Substituting the relationship between the world coordinate system and the image coordinate system used in step S32-2 into the collinearity equation of the photographic transformation, we obtain the expanded collinearity equation, the expression of which is:

[0037] DL-C = 0

[0038]

[0039]

[0040]

[0041] Where D is a matrix composed of the coordinates of six surface feature points of the navigation drone in the navigation drone coordinate system, and L is l1-l 11 The matrix formed by these coordinates is C, which is the matrix composed of the image coordinates of the six surface feature points of the pilot drone in the pilot drone coordinate system and the corresponding image coordinates in the image coordinate system.

[0042] S32-4. Based on the relationship between L, D, and C, calculate the value of L using the following formula:

[0043] L=(D T D) -1 (D T C);

[0044] S32-5, According to l1-l in L 11 The value of the external parameter X of the computer-mounted camera. S ,Y S Z S ,ω, κ.

[0045] Further: the external parameter X of the computer-mounted camera in step S32-5 S ,Y S Z S ,ω, The formula for κ is:

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061] Among them, a3, b3, c3, b1, b2, f x ,dβ,x0,y0,ds,A ds B ds and C ds All of these are intermediate solution variables.

[0062] The beneficial effects of this invention are as follows:

[0063] (1) It reduces the consumption of communication resources, visual positioning relies only on natural light, and has a high degree of concealment;

[0064] (2) Industrial cameras have the advantages of being lightweight, having low power consumption, and being small in size;

[0065] (3) Only the necessary feature points' three-dimensional coordinates need to be extracted from two corresponding images. The amount of information is small and the processing speed is fast, which can well meet the real-time requirements and is especially suitable for dynamic situations.

[0066] (4) Unlike existing visual formation studies, where the detection and positioning of the navigator relies on special markers on the fuselage, this invention identifies and extracts the features of the UAV surface itself based on the SURF method, without the need for special markings, making the operation simpler. Attached Figure Description

[0067] Figure 1 This is a flowchart of the UAV formation positioning method described in this invention.

[0068] Figure 2 This is a schematic diagram of the first rotation according to the angle κ from coordinate system S-xyz to coordinate system D-XYZ.

[0069] Figure 3 This is a schematic diagram of the second rotation according to the angle ω from coordinate system S-xyz to coordinate system D-XYZ.

[0070] Figure 4 The angle of rotation from coordinate system S-xyz to coordinate system D-XYZ A schematic diagram of the third rotation. Detailed Implementation

[0071] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0072] like Figure 1 As shown,

[0073] In one embodiment of the present invention, a method for relative positioning of unmanned aerial vehicle (UAV) formations is provided, comprising the following steps:

[0074] S1. Install an industrial camera on the drone. After installation, use a 3D measurement method to obtain the positional relationship between the drone's surface feature points and the aircraft's center. Use the industrial camera to acquire initial images and perform a self-test on the industrial camera to obtain a drone swarm equipped with image acquisition equipment.

[0075] S2. Use a swarm of drones equipped with image acquisition devices to acquire images and obtain continuous flight images;

[0076] S3. Perform data processing based on the initial image and continuous flight images, and complete the relative positioning of the UAV formation based on the data processing results.

[0077] In this embodiment, S11, an industrial camera is fastened to each of the two front sides of each drone, and after installation, a drone equipped with an image acquisition device is obtained.

[0078] In step S11, the installation requirements for the industrial cameras are: the observation range of the two industrial cameras of each drone can cover a 180-degree field of view in front of the drone and they are installed symmetrically.

[0079] S12. Use a scanner to acquire 3D point cloud data of a single drone, and use an additional industrial camera of the same specifications to capture images of the single drone from different angles as initial images.

[0080] S13. Perform SURF feature recognition on the initial image, and map the recognized UAV fuselage surface feature points to the three-dimensional feature point data of a single UAV obtained in step S1, to obtain the correspondence function between the surface feature points of a single UAV and the center of the aircraft.

[0081] S14. Establish the coordinate system D-XYZ for the navigation drone and the coordinate system D′-X for the non-navigation drone. t Y t Z t and camera coordinate system S-xyz;

[0082] Among them, the D-XYZ coordinate system of the navigation drone takes the center of the navigation drone as the origin, the longitudinal axis of the drone is the X-axis, the Y-axis is perpendicular to the X-axis and points to the left wing, and the Z-axis is perpendicular to the XY plane and points upwards as positive;

[0083] Non-navigational UAV coordinate system D′-X t Y t Z t With the center of the non-navigation drone as the origin, and the longitudinal axis of the aircraft as X... t axis, Y t The axis is perpendicular to X.t The axis points to the left wing, Z t The axis is perpendicular to X. t Y t A plane pointing upwards is considered positive.

[0084] Camera coordinate system S-xyz: The camera coordinate system is established with the center of the onboard camera as the origin; the x-axis and y-axis are parallel to the x-axis and y-axis of the physical image coordinate system, and the z-axis coincides with the optical axis;

[0085] S15. Calculate the coordinates of each surface feature point of the pilot drone in the pilot drone coordinate system based on the correspondence function between the surface feature points of a single drone and the center of the aircraft, and pre-set the coordinates of each surface feature point of the pilot drone in the pilot drone coordinate system and the initial image in each drone.

[0086] S16. Perform a self-test on the onboard industrial camera to confirm that the data can be returned correctly, thus obtaining the drone swarm equipped with image acquisition equipment.

[0087] In this embodiment, the image acquisition method in step S2 is as follows: a swarm of drones equipped with image acquisition devices flies, and during the flight, each drone takes pictures of the front of the drone through an industrial camera to obtain continuous flight images, thus completing the image acquisition.

[0088] In this embodiment, step S3 includes the following sub-steps:

[0089] Step S3 includes the following sub-steps:

[0090] S31. Based on the SURF method, perform feature matching between the flight images of the other non-leader drones and the initial image to obtain the three-dimensional coordinates of some feature points on the surface of the leader drone in the leader drone coordinate system.

[0091] The coordinates of the three-dimensional coordinates are represented as: (X i ,Y i Z i (i = 1, 2, 3...);

[0092] S32. Based on the coordinates of no fewer than 6 feature points in the navigation UAV coordinate system and the corresponding image coordinates in the image coordinate system, determine the external parameters X of the computer-mounted camera. S ,Y S Z S ,ω, κ;

[0093] Among them, (X) S ,Y S Z S ( ) represents the coordinates of the airborne camera's photography center in the coordinate system of the navigation drone. These are the three rotation angles from coordinate system S-xyz to coordinate system D-XYZ;

[0094] like Figures 2-4 The figures shown represent the three rotations from coordinate system S-xyz to coordinate system D-XYZ. A schematic diagram of the rotation process;

[0095] S33. Based on the relationship between the center of the airborne camera and the center of the UAV, perform a transformation to obtain the coordinate information (X) of the UAV's own center in the coordinate system of the navigation UAV. St ,Y St Z St ) and from coordinate system D′-X t Y t Z t The three corners to coordinate system D-XYZ And according to (X) St ,Y St Z St )and Complete the relative positioning of the drone formation.

[0096] Step S32 includes the following sub-steps:

[0097] S32-1. Using the collinearity equation of photographic transformation, the coordinates of any point in the world coordinate system and its corresponding image coordinates in the image coordinate system are expressed by the following formula:

[0098]

[0099] in, Let these be the homogeneous coordinates of the object point in the world coordinate system. Z represents the homogeneous coordinates of the corresponding image point in the image coordinate system. C M is the object distance from a point in space to the center of the camera's image, and AB is the projection matrix. The intrinsic parameter matrix, The extrinsic parameter matrix;

[0100] Among them, F x =f / dx, F y = f / dy, where f represents the camera's focal length, dx and dy represent the length and width per pixel, respectively, and C x C y R represents the coordinates of the principal point of the image in the image coordinate system; R and t are the rotation and translation matrices in the camera's extrinsic parameters, and T represents the transpose.

[0101] S32-2. The projection matrix M is represented using the relationship between the world coordinate system and the image coordinate system. Its expression is as follows:

[0102]

[0103] Among them, l1-l 12 Let l be the coefficient relating the world coordinate system and the image coordinate system. 12 =1;

[0104] S32-3. Substituting the relationship between the world coordinate system and the image coordinate system used in step S32-2 into the collinearity equation of the photographic transformation, we obtain the expanded collinearity equation, the expression of which is:

[0105] DL-C = 0

[0106]

[0107]

[0108]

[0109] Where D is a matrix composed of the coordinates of six surface feature points of the navigation drone in the navigation drone coordinate system, and L is l1-l 11 The matrix formed by these coordinates is C, which is the matrix composed of the image coordinates of the six surface feature points of the pilot drone in the pilot drone coordinate system and the corresponding image coordinates in the image coordinate system.

[0110] S32-4. Based on the relationship between L, D, and C, calculate the value of L using the following formula:

[0111] L=(D T D) -1 (D T C);

[0112] S32-5, According to l1-l in L 11 The value of the external parameter X of the computer-mounted camera. S ,Y S Z S ,ω, κ;

[0113] In step S32-5, the external parameter X of the computer-mounted camera S ,Y S Z S ,ω, The formula for κ is:

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129] Among them, a3, b3, c3, b1, b2, f x ,dβ,x0,y0,ds,A ds B ds and C ds All of these are intermediate solution variables.

[0130] Considering the extreme case where it is impossible to obtain the coordinate information of at least six feature points on the surface of the lead drone, each non-lead drone can also roughly solve its relative position to the lead drone based on the pinhole imaging model. After completing the feature point pairing, the actual distance D between two feature points and the coordinates (u) of their corresponding two image points in the image coordinate system are given by the three-dimensional feature point model obtained in step S1. i ,v i (i=1,2), the coordinates (x, y) of the two feature points in the camera coordinate system can be calculated using the following formula. i ,y i ,z i (i = 1, 2), its expression is:

[0131]

[0132] in, f is the focal length of the camera;

[0133]

[0134] (X,Y,Z) can be considered as the approximate coordinates of the center of the navigation drone in the camera coordinate system. Through certain translation and rotation transformations, the coordinate information of the center of the navigation drone in the non-navigation drone coordinate system can be obtained.

[0135] In the description of this invention, it should be understood that the terms "center," "thickness," "upper," "lower," "horizontal," "top," "bottom," "inner," "outer," and "radial," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying the relative importance or the number of technical features implicitly specified. Therefore, a feature defined by "first," "second," and "third" may explicitly or implicitly include one or more of that feature.

Claims

1. A method for relative positioning of unmanned aerial vehicle (UAV) formations, characterized in that, Includes the following steps: S1. Install an industrial camera on the drone. After installation, use a 3D measurement method to obtain the positional relationship between the drone's surface feature points and the aircraft's center. Use the industrial camera to acquire initial images and perform a self-test on the industrial camera to obtain a drone swarm equipped with image acquisition equipment. Step S1 includes the following sub-steps: S11. Securely install an industrial camera on each of the two front sides of each drone. After installation, a drone equipped with an image acquisition device is obtained. S12. Use a scanner to acquire 3D point cloud data of a single drone, and use an additional industrial camera of the same specifications to capture images of the single drone from different angles as initial images. S13. Perform SURF feature recognition on the initial image, and map the recognized UAV fuselage surface feature points to the three-dimensional feature point data of a single UAV obtained in step S1, so as to obtain the correspondence function between the surface feature points of a single UAV and the center of the aircraft. S14. Establish the coordinate system D-XYZ for the navigation drone and the coordinate system for the non-navigation drone. and camera coordinate system S-xyz; Among them, the D-XYZ coordinate system of the navigation drone takes the center of the navigation drone as the origin, the longitudinal axis of the drone is the X-axis, the Y-axis is perpendicular to the X-axis and points to the left wing, and the Z-axis is perpendicular to the XY plane and points upwards as positive; Non-navigation drone coordinate system With the center of the non-navigation drone as the origin, the longitudinal axis of the aircraft is... axis, Axis perpendicular to The axis points to the left wing. Axis perpendicular to A plane pointing upwards is considered positive. Camera coordinate system S-xyz: The camera coordinate system is established with the center of the onboard camera as the origin; the x-axis and y-axis are parallel to the x-axis and y-axis of the physical image coordinate system, and the z-axis coincides with the optical axis; S15. Calculate the coordinates of each surface feature point of the pilot drone in the pilot drone coordinate system based on the correspondence function between the surface feature points of a single drone and the center of the aircraft, and pre-set the coordinates of each surface feature point of the pilot drone in the pilot drone coordinate system and the initial image in each drone. S16. Perform a self-test on the onboard industrial camera to confirm that the data can be returned correctly, and obtain the drone swarm equipped with image acquisition equipment; S2. Use a swarm of drones equipped with image acquisition devices to acquire images and obtain continuous flight images; S3. Perform data processing based on the initial image and continuous flight images, and complete the relative positioning of the UAV formation based on the data processing results; Step S3 includes the following sub-steps: S31. Based on the SURF method, perform feature matching between the flight images of the other non-leader drones and the initial image to obtain the three-dimensional coordinates of some feature points on the surface of the leader drone in the leader drone coordinate system. The coordinates of the three-dimensional coordinates are represented as follows: ; S32. Based on the coordinates of no fewer than 6 feature points in the navigation UAV coordinate system and the corresponding image coordinates in the image coordinate system, determine the external parameters of the computer-mounted camera. ; in, The coordinates of the airborne camera's photography center in the coordinate system of the navigation drone. These are the three rotation angles from coordinate system S-xyz to coordinate system D-XYZ; S33. Based on the relationship between the center of the airborne camera and the center of the UAV, perform a transformation to obtain the coordinate information of the UAV's own center in the coordinate system of the navigation UAV. and from coordinate system The three corners to coordinate system D-XYZ and according to and Complete the relative positioning of the drone formation.

2. The method for relative positioning of UAV formations according to claim 1, characterized in that, In step S11, the installation requirements for the industrial cameras are: the observation range of the two industrial cameras of each drone can cover a 180-degree field of view in front of the drone and they are installed symmetrically.

3. The method for relative positioning of UAV formations according to claim 1, characterized in that, The image acquisition method in step S2 is as follows: a swarm of drones equipped with image acquisition devices flies, and during the flight, each drone takes pictures of the front of it through an industrial camera to obtain continuous flight images, thus completing the image acquisition.

4. The method for relative positioning of UAV formations according to claim 1, characterized in that, Step S32 includes the following sub-steps: S32-1. Using the collinearity equation of photographic transformation, the coordinates of any point in the world coordinate system and its corresponding image coordinates in the image coordinate system are expressed by the following formula: in, Let these be the homogeneous coordinates of the object point in the world coordinate system. These are the homogeneous coordinates of the corresponding image points in the image coordinate system. The distance between a point in space and the center of the camera's field of view. Let be the projection matrix. The intrinsic parameter matrix, The extrinsic parameter matrix; in, , , f Indicates the camera's focal length. dx and dy These represent the length and width per unit pixel, respectively. , R represents the coordinates of the principal point of the image in the image coordinate system; R and t are the rotation and translation matrices in the camera's extrinsic parameters, and T represents the transpose. S32-2. The projection matrix M is represented using the relationship between the world coordinate system and the image coordinate system. Its expression is as follows: in, - The coefficients representing the relationship between the world coordinate system and the image coordinate system are: ; S32-3. Substituting the relationship between the world coordinate system and the image coordinate system used in step S32-2 into the collinearity equation of the photographic transformation, we obtain the expanded collinearity equation, the expression of which is: in, D This is a matrix composed of the coordinates of six surface feature points of the navigation drone in the navigation drone coordinate system. L for - The matrix formed C This is a matrix composed of the image coordinates of the six surface feature points of the navigation drone in the navigation drone coordinate system and the corresponding image coordinates in the image coordinate system. S32-4, according to L , D and C Relationship, calculation L The value of is given by the formula: ； S32-5, according to L middle - The value of the external parameters of the computer-mounted camera. .

5. The UAV formation relative positioning method according to claim 4, characterized in that, The external parameters of the computer-mounted camera in step S32-5 The formula is: in, a 3 、b 3 、c 3 、b 1 、b 2 、f x , dβ, x 0 、y 0 ,ds,A ds 、B ds and C ds All of these are intermediate solution variables.

Images