Vision-based Collaborative Conflict Avoidance

Abstract

A method and apparatus for drone collision avoidance are described, including capturing an image of an environment using a camera. The camera is oriented in a direction. Second feature information is received from an external source. The second feature information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction. First feature information is extracted from the captured image. The first feature information is matched with the second feature information. A second local reference frame of the second feature information is transformed to a first local reference frame of the first feature information to determine a position of the external source. Determining whether a collision with the external source will occur based on the position of the external source and a current flight trajectory.

Description

Technical Field

[0001] The embodiments described herein generally relate to collision avoidance for aircraft, such as drones. The embodiments are implemented in hardware, software, or a mixture of both hardware and software. Background Art

[0002] Current collision avoidance capabilities for area aircraft (such as drones) have various limitations. For example, collision avoidance methods may be limited to detecting large objects (such as walls), have slow reaction times, and / or rely on 3D cameras to locate nearby objects. These capabilities may not be applicable to proxy-proxy collision avoidance. A typical scenario where existing methods fail is when two drones fly in a collision path at 90 degrees. This can occur in open spaces or areas such as corridors.

[0003] Another known method of collision avoidance relies on external positioning or simultaneous localization and mapping (SLAM) algorithms. However, this method only works after the aircraft has a public map. In addition, real-time SLAM algorithms require powerful computers that are typically available on drones. SLAM algorithms may also not be reliable in dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 Capturing images for collision avoidance according to an exemplary embodiment is illustrated.

[0005] Figure 2 Three drones are illustrated in an environment according to some embodiments.

[0006] Figures 3A-3F A cooperative collision avoidance technique in accordance with some embodiments is illustrated.

[0007] Figure 4 Illustration of a drone aligning its camera to point toward a future location in accordance with some embodiments.

[0008] Figure 5 Illustrates two drones that can locate each other according to some embodiments.

[0009] Figure 6 Illustrating micro-neighbor positioning according to some embodiments.

[0010] Figure 7 is a flow chart for neighbor positioning according to some embodiments.

[0011] Figure 8 Determining the distance between two drones is illustrated in accordance with some embodiments.

[0012] Fig. 9 Sharing of neighbor information according to some embodiments is illustrated.

[0013] Fig.10 The use of virtual forces to avoid collisions in accordance with some embodiments is illustrated.

[0014] Fig.11 An example of using virtual forces according to some embodiments is illustrated.

[0015] Fig.12 is a flow chart for performing posture adjustments using virtual forces according to some embodiments.

[0016] Fig.13 A hierarchical trajectory plan is illustrated in accordance with some embodiments.

[0017] Fig.14 is a flow chart illustrating the use of hierarchical trajectory planning according to certain embodiments.

[0018] Fig.15 Trajectory planning based on a sampling technique according to some embodiments is shown.

[0019] Fig.16 A block diagram illustrating a computing device within which a set or sequence of instructions may be executed to cause the device to perform an example of any of the methodologies discussed herein. DETAILED DESCRIPTION

[0020] Many current applications of drones, such as precision agriculture, inspection, search and rescue, warehouse and inventory systems, can benefit from the interaction of multiple drones. For example, drones can reduce the time required to complete a job when using a single drone. In addition, there are many uses for drones in close proximity to each other. For example, multiple drones can be operated in the same enclosed space or in a small outdoor area. Accordingly, there is an interest in achieving autonomous flight. For safe autonomous flight, drones should avoid collisions with each other or other objects.

[0021] In an example, the disclosed collision avoidance procedure allows multiple drones to safely coexist in a cluttered environment (such as, interior) or in low-altitude flight. Thus, new applications and increased efficiency are achieved through multi-drone systems. The proposed procedure is simpler and faster than current SLAM-based approaches, which require agents (e.g., drones, stationary objects, ships, etc.) to be registered in a global map before any collaborative maneuvers can be planned. Various disclosed collision avoidance techniques do not require a common reference frame and an environmental map. The disclosed technology improves simple reaction methods based on detection and tracking of dynamic obstacles by allowing agents to anticipate potential collision avoidance situations, and can be equipped with monocular (combined with inertial measurements), stereo or 3D cameras. The data association between sensor observations obtained by different agents can be parsed and used to determine the relative positioning between two agents and corresponding neighbors.

[0022] In an example, a down-facing camera is oriented to ensure that two drones in a collision trajectory will detect a possible collision X / 2 seconds in advance, locate and track each other to plan accordingly. X is an amount of time that can be calculated based on the maneuverability and / or reaction time of the drone. For example, X can be the amount of time a drone needs to move a specific distance to avoid a collision. In various embodiments, X can be changed based on the speed of the drone. Collision detection and avoidance can be performed in situations where there is no line of sight between drones (such as in a corridor). In another example, the camera can be oriented upward. In various embodiments, the orientation of the drone's camera is the same. In addition, common features in the two images are used to locate the position. Accordingly, any orientation and features within the image can be used for the disclosed collision avoidance technology.

[0023] In various embodiments, the proposed reactive collision avoidance considers the position and velocity of neighboring agents to create virtual forces. The virtual forces and their reaction forces can be used by the low-level controller to steer the drone to avoid collisions with neighboring agents. In some embodiments, the hierarchical trajectory planning algorithm considers static and dynamic objects in the environment to plan collision-free trajectories by considering dynamic objects and other agents with higher priority as obstacles in 4D (3D space + time).

[0024] Embodiments relate to cooperative collision avoidance for drone-drone communication based on sparse visual features captured with a camera, such as with a pan / tilt camera. In an embodiment, the pan / tilt camera is oriented in such a way that it points to an area on the ground that the drone plans to fly over in the next X seconds. X can be, for example, 1, 3, 5, 15, 30 seconds. The value X can be set based on the maneuverability and reaction time of the drone. For example, if a drone requires 12 seconds to be able to move a distance equal to the maximum size of one, two, three, etc. drones, then X can be set to 12 seconds. This ensures that the drone is able to move out of the way of an approaching drone or obstacle. In an example, the value of X is set to twice the time required to move a certain amount of space. Setting the value to twice or more of this amount takes into account examples where two drones are flying directly toward each other.

[0025] Figure 1 An image 110 captured for collision avoidance is illustrated according to an exemplary embodiment. Drone 102 may autonomously fly along path 104 to destination 106. Destination 106 will be reached at a time calculated based on the speed of the drone along path 104, the expected speed, and path 104. When drone 102 reaches destination 106, drone 103 will be directly above point 108. Image 110 includes a plurality of obstacles or features 120A-120E.

[0026] Obstacles 120A-120E may be used by another drone to determine the location of drone 102. Drone 102 may share images 110 or features associated with one or more of obstacles 120A-120E with another drone. Another drone may use images from its camera to determine the location of drone 102. Using the location and additional information (such as a destination, path 104, etc.), another drone may change its path to avoid colliding with drone 102.

[0027] Sharing imagery and / or feature information allows both drones to "see" a common area in their planned trajectories identified by multiple features corresponding to their respective views, and will be able to locate each other. This allows either drone to determine if it is on a collision trajectory with the other drone. If a future collision based on the current planned trajectory is determined, a collision avoidance maneuver is planned to avoid the collision. In addition, the drone may change information about the trajectories of other drones or objects in its field of view so that their presence can be taken into account when replanning the trajectory. Optionally, other onboard cameras of the drone may be used to verify the presence of other drones. In addition, cameras on stationary objects may also be used. In this example, the drone may determine if its trajectory will collide with a stationary object and may replan the trajectory as needed.

[0028] In a dynamic environment including multiple drones, each drone may not be able to see or locate every other drone. However, each drone can share the position of its neighbors relative to its own reference frame. In this way, one drone can detect unseen drones and plan accordingly.

[0029] Figure 2 Three drones 202, 204, and 206 are illustrated in an environment according to some embodiments. In this environment, an image 212 from a camera of a first drone 202 does not overlap with an image 216 from a third drone 206. However, an image 214 from a second drone 204 overlaps with images 212 and 216. Accordingly, the first drone 202 will not be able to locate the position of the third drone 206 from images 212 and 216. However, the second drone 204 can locate the positions of both the first drone 202 and the third drone 206. The second drone 204 can share its own position and the position of the third drone 206 relative to its own reference frame with the first drone 202. The first drone 202 can then determine its position relative to the second drone 204, and then use the shared information to determine its position relative to the third drone.

[0030] One collision avoidance technique is based on placing virtual forces (which are functions of position and velocity) on the locations of neighboring drones and embedding the reactions to these forces into the trajectory tracking controller. In a second technique, each drone can access the planned trajectories of other drones in its neighborhood via drone-drone communication. Using the information shared between drones, agent B's plan can be placed in the reference frame of agent A. Possible future collisions can be determined from agent B's flight plan and agent A's flight plan. If a future collision is detected, agent A can use a hierarchical planner to update its expected trajectory.

[0031] Figures 3A-3F The collaborative collision avoidance technique according to some embodiments is illustrated. The collaborative collision avoidance technique allows for fast neighbor location and online planning and collision avoidance. Figures 3A-3F In the example, the first drone 302 has two-way communication with the second drone 304. Two-way communication is not required. For example, the second drone 304 may provide its data to the first drone 302, but not receive any data from the first drone 302. In this example, the first drone 302 will change its own trajectory to avoid the second drone 304. This allows the first drone 302 to not include a transmitter function and transmission hardware.

[0032] return Figures 3A-3F , both the first drone 302 and the second drone 304 are able to transmit their position data. Accordingly, the present technology utilizes drone-to-drone communications that may include image data from a gimbal-controlled downward-facing camera. The position of the camera may be a function of speed, maneuverability, and the drone's intended trajectory in a manner that the camera captures an area on the ground that the drone plans to fly over in X seconds. For example, image 312 is captured using a camera from the first drone 302. Image 314 is captured using a camera from the second drone 304. In an embodiment, image 312 or 314 itself may be transmitted. In another embodiment, extracted features may be transmitted instead of the entire image.

[0033] Features in each image 312, 314 are then extracted using known methods and broadcast to neighboring drones using short-range or wireless network communications. By sharing these features, neighboring drones can analyze feature matches, detect if parts of the features are common (there is image overlap), and use geometric techniques of multiple views (in this case, views from different cameras) to locate the neighboring drone with respect to its own reference frame. Using the views of multiple neighbors, the relative speed of the drones can be calculated. In another embodiment, the drone can communicate its speed.

[0034] Furthermore, since the geometry from the different views can also be applied to the sequence of its own views, the drone can calculate its own velocity in its own reference frame and extract the velocities of neighbors. This technique solves the problem of associating the location of a neighbor with the neighbor's ID and / or communication channel, which is a common problem in V2X communications where the infrastructure receives the information but cannot distinguish the transmitting aircraft.

[0035] Figure 3A The locations of the first drone 302 and the second drone 304 are shown along with images 312 and 314 captured by each of the drones 302 and 304. The images 312 and 314 are centered on the location where each drone will be at a particular time in the future. Figure 3B Images 312 and 314 acquired from the first drone 302 and the second drone 304, respectively, are shown. Images 312 and 314 include common objects. Figures 3A-3F In the example of FIG. 3 , the camera on each of drones 302 and 304 is pointed downward. In other examples, the camera may be pointed upward, east, west, north, south, etc. Each of drones 302 and 304 extracts features from its image. Figure 3C Features extracted from each of images 312 and 314 are shown as circles. This feature information can be transmitted to other drones and other nearby drones or receivers. In addition, other information can be transmitted, such as other drones' positions, speeds, current flight paths, times when drones will pass through the center of the image, etc. Figure 3D The matching of features generated from two images 312 and 314 is shown.

[0036] Features may be matched using known feature matching techniques. For example, each feature may have a descriptor that may be obtained by a feature extraction technique such as Scale Invariant Feature Transform (SIFT) or Speeded Up Robust Features (SURF). An example of a brute force approach for matching features is to take a feature from one image and compare it to all such features in other images, selecting the one with the smallest distance or within a threshold. Another example is to use geometric constraints between the two images, locate the feature in one image and look for it in the epipolar lines defined in the second image. Yet another option is to use a matcher based on Fast Approximate Nearest Neighbor Search (FLANN). To remove outliers, a Random Sample Consensus (RANSAC) algorithm may be used afterwards.

[0037] Using the common features, each of drones 302 and 304 can calculate the location of the other drones. Furthermore, the location is paired with the flight path and / or speed, and each drone can calculate whether a collision will occur when the drone moves over the location pointed by the camera.

[0038] If a collision is predicted on the current flight paths of drones 302 and 304, one or both flight paths may be changed. In the example, each drone 302 and 304 has a priority. This priority information may be shared and communicated with other drones. Drones with lower priorities change their flight paths to avoid collisions, while drones with higher priorities do not change their flight paths. This helps eliminate the situation where both drones change their flight paths, but in such a way that collisions are not avoided. Figure 3E Drones 302 and 304 are shown on a collision course. Drone 304 has a flight path 344 and a higher priority than drone 302. Drone 302 has a flight path (not shown) that directly faces drone 304. After sharing the flight path and priority information, drone 302 determines that a collision with drone 304 is predicted based on the current flight path. Since drone 302 has a lower priority, drone 302 changes its flight path to a new flight path 342 that avoids the collision.

[0039] Figure 3F Another collision avoidance technique is illustrated. This technique is a reactive collision avoidance using the velocities of drones 302 and 304 to determine virtual forces 362 and 364. These virtual forces 362 and 364 indicate the space that drones 302 and 304 may occupy in the future. In the example, virtual forces 362 and 364 are extended to three dimensions. Virtual forces 362 and 364 may be communicated to other drones, which may then be used to determine if a collision is predicted.

[0040] In one example, a stationary receiver receives images or features extracted from images from one or more drones. The stationary receiver may then perform trajectory planning and collision determination. If a possible collision is detected, the stationary receiver may provide an updated flight path or flight instructions to avoid the collision. For example, a drone may be instructed to reduce or increase its speed to avoid a collision. As another example, a drone may be instructed to change its altitude to avoid a collision. In an example, a stationary receiver develops flight plans for multiple drones in an area, allowing the drones to have less hardware and be lighter than if all the hardware required to perform flight planning onboard was present.

[0041] Camera orientation facing downward

[0042] As described above, use cameras that each have the same orientation (e.g., downward, upward). The following describes a downward-facing camera orientation, but can also be used with upward-facing cameras. Assume that the drone is at position (x 0 ,y 0 ,z 0 ) T , and has a planned position (x p ,y p ,z p) T The drone camera can then be controlled to point to (x p ,y p ,z g ) T , where z g is ground, e.g. 0. Figure 1 In the figure, the drone 102 is pointing at its (x p ,y p ,z g ) T Position 108. Order From (x 0 ,y 0 ,z 0 ) T Point to (x p ,y p ,z g ) T The unitary vector of :

[0043]

[0044] Let R(ψ,φ,θ)=R(t) be the rotation matrix defining the transformation from the initial orientation of the drone to the current orientation given φ,θ,ψ (roll, pitch, yaw, Euler angles).

[0045]

[0046] also, represents the unit vector pointing from the drone's takeoff point to the front of the drone, and is the unit vector pointing in the direction of the drone’s front at a given time t.

[0047]

[0048] Correspondingly, there exists a transformation matrix R G (ψ G ,φ G ,θ G ), the gimbal can perform this transformation matrix to move the camera from pointing to Directed to point to This relationship can be written as:

[0049]

[0050] Figure 4 4 illustrates a drone aligning its camera to point to a future location according to some embodiments. At a first point in time, drone 402A has Vector aligns camera 404. At a future time T, drone 402A calculates that it will pass through point 406. Accordingly, to capture a picture that includes point 406, drone 402A aligns its camera 404 toward point 406. Using the above transformations, drone 402A can calculate its rotation and other flight parameters. After such a change in their relative positions, drone 402B now has its camera 404 pointing toward point 406, which is along vector

[0051] Without loss of generality, can be defined as a unit vector pointing in the x direction, so Note that since φ, θ, ψ are known from the sensors, we can use known methods to solve To obtain the components of G. Finally, from this component, the gimbal camera ψ can be obtained G ,φ G ,θ G For example, if G has components G ij ,but:

[0052]

[0053]

[0054] θ G = -arcsin(G 31 )

[0055] Geometry of multiple views

[0056] Figure 5Two drones 502 and 504 are illustrated that can locate each other according to some embodiments. Drones 502 and 504 can locate each other by matching features on their images with those shared by their neighbors. Each drone 502 and 504 can share features on its own reference frame. In addition, drones 502 and 504 can use continuous image information to calculate their relative speed. Since each drone 502 and 504 can calculate its own range, the velocity vector of drone 504 in the reference frame of drone 502 can be obtained. Image 512 is acquired by a camera on drone 502. As described above, the camera can point to the location where drone 502 will be at some point in the future. Object 520 within the environment can become part of image 512 as represented object 522. The same object 520 can also be part of image 514 acquired by drone 504. Object 524 can be captured within image 514. Each drone 502 and 504 can perform feature extraction, where features of object 520 and other objects can be obtained. This feature information can be shared between drones 502 and 504. Accordingly, drone 502 has its image 512, feature information from image 512, and feature information from image 514. Drone 502 can compare feature information from both images 512 and 514 and match its version 522 of object 520 with version 524 from drone 504. In addition, drone 502 can match other objects between the two images 512 and 514. This allows drone 502 to determine the location of drone 504. Traditionally, transformations from different views of the same drone can be used to calculate visual inertial odometry. Although as described, views from different drones can be used to locate other drones and then use the locations and flight plans of the other drones to plan avoidance maneuvers as needed.

[0057] Neighbor Positioning

[0058] Figure 6 1 illustrates neighbor positioning according to some embodiments. As described above, drones 602 and 604 orient their cameras to where they will fly over in the future, ensuring that the two drones in the collision path will calculate common areas within the image of the ground at least X / 2 seconds in advance. Drones 602 and 604 may then capture images 612 and 614 of environment 616, respectively. Agent A, drone 602, may then share information of the image and / or information of features extracted from image 612 with nearby neighbors. Images 612 and 614 capture certain identical objects, such as object 620. This object is represented as object 622 and object 624 in images 612 and 614, respectively. This information allows neighbor agent B, drone 604, to match information (such as information related to object 620) to detect image overlap and calculate a transformation (e.g., homography) representing the translation and orientation of agent A's reference frame with respect to agent B's reference frame.

[0059] Figure 7 702 is a flowchart for neighbor positioning according to some embodiments. This flowchart illustrates the operation of a drone to locate a neighbor. At operation 702, the drone captures an image. Before capturing the image, the drone can calculate where the drone will be at a future time and orient its camera to point to this location on the ground or ceiling. Subsequently, at operation 704, image information (such as features, lines, etc.) can be extracted from the image. At operation 706, the image information in the local frame of the drone can be broadcast so that a nearby drone or receiver can receive the image information. At operation 708, the drone can also receive drone image information and positions of neighbors within the neighbor's frame. In an example, the image information can come from a stationary object or another drone. At operation 710, the image information from the drone image can be matched with the received image information. For example, features can be compared and matched. At operation 712, overlaps in the image information are determined. If there is no overlap, the drone determines that there is no collision when the drone flies to its future position.

[0060] If there is an image overlap, there is a possibility that the drones will collide as they fly to their future positions. At operation 714, the transformation between the two views is calculated. The first transformation obtains the translation and rotation from one camera view to the other drone camera view. This transformation transforms between the camera system of the drone and the camera system of the drone from which the image information is received. Then at operation 716, the transformation between the local systems is completed, which allows the drone to use the trajectory information about other drones and any neighbor drone information. The first transformation can be connected with the rotation and translation of the camera view about its local system to obtain a second transformation. This transformation can be used by one drone to locate the neighbors of the neighbors in its local reference system. Accordingly, the transformation allows the drone to determine the position of other drones relative to its system.

[0061] At operation 720, the drone may receive the positions of other drones that have been located from neighboring drones. This allows the drone to account for the flight paths of drones that do not have image overlap. Information is provided about the positions of nearby drones (e.g., drones that have image overlap with the current drone or image overlap with neighboring drones). At operation 722, the positions of other drones are aggregated. The drone may then use this information to update its tracking algorithm at operation 724 to determine if a collision is predicted. Various collision algorithms are disclosed below. The drone may then capture another image and restart the process so that the drone can continue to fly without colliding with other drones or obstacles.

[0062] Fusion of vision-based estimation and distance measurement

[0063] If the features in the image are obtained from a monocular camera and there is no depth information, the relative position between the drones can be calculated until an unknown zoom factor can be recovered. Using a communication device for sharing information between agents, the distance from the receiver to the transmitter can be measured. For example, a preamble can be included in the communication packet to implement an RTT (round trip time) protocol, which can be used to estimate the distance. Similarly, other methods such as RSS (received signal strength) can be used to determine the distance. It is also possible to use ultra-wideband radio for accurate distance measurement.

[0064] Figure 8 The distance between two drones 802 and 804 is determined according to some embodiments. To fuse this information, the vector t between the two drones 802 and 804 is 12 820 defines a linear curve in the space where another agent can be located. The vector t can be obtained from the transformation described above with respect to transformation 714 12 On the other hand, the distance d to the drone 804 that can be obtained from the radio communication defines a sphere 810 in the space in which the drone 804 can be located. The intersection of the vector 820 with the sphere 810 is unique and corresponds to an estimate of the position of the drone 804.

[0065] Obtain neighbor and trajectory information from UAV 2 in UAV 1's reference frame

[0066] Fig. 9 Sharing neighbor information according to some embodiments is illustrated. Drone 902 may capture image 912 and extract features from image 912. For example, object 930 may be captured in image 912 and features related to object 930 or other objects may be extracted from image 912. These features may be shared with drone 904. Drone 904 has acquired its own image 914, which also captures object 930. Features that may be extracted from image 914 may be shared with drone 902. A third drone 906 may also acquire an image (not shown) that does not include any or sufficient features for matching its image with image 912 from drone 902. However, third drone 906 may be matched with drone 904, so that drone 904 may be able to determine the location of drone 906. The location of drone 906 may then be shared with drone 902 via drone 904.

[0067] Drone 906 may share its flight plan with drone 904. Drone 904 may then share its flight plan 924 and the position of drone 906 and flight plan 926 with drone 902. Drone 904 receives the flight plan of drone 906 relative to the reference frame of drone 906. After calculating the position of drone 906, drone 904 may transform the flight plan of drone 906 into the reference frame of drone 904. This transformed flight plan may be shared with drone 902. Alternatively, the position of drone 906 relative to drone 904 may be shared with drone 902, and drone 902 may then use the positions of drones 904 and 906 to perform a transformation to transform flight plan 926 to the reference frame of drone 902.

[0068] Thus, drone 902 may be able to consider flight plan 926 of drone 906. The flight plan for a drone may be one or more three-dimensional points relative to the drone's coordinates. Drone 902 may then consider the flight plans of both drones 904 and 906 to determine its flight path and whether there are any possible collisions. Drone 904 may share the position of drone 906 relative to drone 904 with drone 902. Drone 902 may then obtain the transformation [R 12 |t 12 ], which transforms between the camera systems of drone 902 and drone 904. 12 is the rotation between the reference frames of the two camera views. The vector t 12 is the distance vector between the two drones 902 and 904. Subsequently, [R 12 |t 21 ] to transform the trajectory information of drone 904 and drone 906, which allows drone 902 to know where drone 906 is flying.

[0069] Virtual force based reactive collision avoidance

[0070] Fig.10 Illustrated using virtual forces to avoid collisions in accordance with some embodiments. Three agents, or drones 1002, 1004, and 1006, are flying within an environment. Each of these drones 1002, 1004, and 1006 may have virtual forces 1012, 1014, and 1016, respectively. The virtual forces may be based on the velocity of the drone. In an embodiment, the virtual forces may be visualized as spheres. The virtual forces may also be used to adjust the position of the drones to avoid collisions. Drones 1002 and 1004 may have velocities with respect to the reference frame of drone 1002. and As mentioned above, the relative speed can be obtained And the relative position The virtual force can be based on the following formula:

[0071]

[0072] where λ is a design parameter defining the size of the potential field, and has a nominal / center value of 1. The direction of the force points away from drone 1004, and due to A factor whose magnitude decreases with distance and inversely / squarely to the direction of relative velocity. This term decreases as drone 1004 moves away from drone 1002. This value is 1 when drone 1004 moves parallel to drone 1002. This term increases as drone 1004 moves toward drone 1002.

[0073] Virtual forces can be applied to any number of drones or other objects, causing drone 1002 to generate virtual forces 1014 and 1016 for each of the drones and / or other drones. This virtual force can then be added to the default position control implemented by the drone. In an embodiment, there is a position control on top of the attitude control, where the former generates commands for the latter. These virtual forces can be added to the position control to generate new attitude commands. The mixer takes the attitude commands and generates corresponding powers to be applied to the motors. Thus, the virtual forces can affect the acceleration of the drone. This change generates a change in the reference of the attitude control, which changes the movement of the drone.

[0074] Fig.11 An example of using virtual forces in accordance with some embodiments is illustrated. Drone 1102 and drone 1104 are flying such that without adjusting the flight paths, drones 1102 and 1104 will collide at a point in the future. To avoid the collision, one or both of drones 1102 and 1104 may use virtual forces to adjust their flight paths. As described above, the velocities of both drones 1102 and 1104 with respect to the reference frame of drone 1102 are known or may be calculated. The relative velocity may then be calculated. and location The virtual force 1112 of drone 1102 with respect to drone 1104 may be calculated as described above. This virtual force may then be used to update the flight path of drone 1102 by adjusting the input to the attitude controller. The same calculation of the reference from drone 1104 may be performed to calculate the virtual force 1114. When drones 1102 and 1104 fly toward each other, drones 1102 and 1104 will repel each other based on their calculated virtual forces 1112 and 1114. From an external vantage point, drones 1102 and 1104 will appear to repel each other when they fly close to each other. Each drone 1102 and 1104 will adjust its flight path to avoid collision. When drones 1102 and 1104 fly over each other, the virtual force value will decrease, and each of drones 1102 and 1104 will have less impact on the flight path of the other drone.

[0075] Fig.12 1 is a flow chart for using virtual forces to make attitude adjustments according to some embodiments. A drone has a flight plan 1202 that is input to a position controller 1206. Without additional input, the position controller 1206 will provide commands to an attitude controller 1212. The attitude controller 1212 provides commands to a mixer 1214, which determines the appropriate power and applies the appropriate power to a motor 1216. Thus, the drone can fly along its planned route 1202. However, without additional input, the drone may collide with other drones or objects. To avoid other drones, the relative position and relative velocity of the drone 1204 are provided to a virtual force generator 1208. Virtual forces may be generated for drones located near the drone. The virtual forces may then be added 1210 to the commands from the position controller 1206. Thus, the drone may deviate from the plan 1202 based on the position and velocity of nearby drones (e.g., drones that may collide with the current drone).

[0076] Hierarchical trajectory planning with collision avoidance

[0077] Fig.13 Hierarchical trajectory planning according to some embodiments is illustrated. Hierarchical trajectory planning can be used instead of or in addition to reactive collision avoidance. In an example, hierarchical trajectory planning can be used when a local minimum is detected. A local minimum occurs when other drones or obstacles block a drone from its destination. In this case, the drone can stay in place until the obstacle moves or another trajectory plan (such as a hierarchical trajectory plan) is used. Partial or complete trajectories of drones can be transmitted to neighboring drones in their own reference frame.

[0078] In an embodiment, each drone has three associated spheres. The collision sphere is the smallest sphere that is c The radius of the sphere is:

[0079]

[0080] where r c is the radius of the collision sphere, p 12 ,v 12 ,a 12 are the relative position, velocity and acceleration of the center of mass with respect to other drones. h is the discretized time step and t r is the time it takes the drone to calculate the new trajectory. The observation sphere encloses the replanning sphere and its radius is given by:

[0081] r o =r r +v limit t o

[0082] where t o is the time it takes for the UAV to receive the trajectory and align it with its reference frame, and v limit (v 限制 ) is the maximum speed allowed for all drones in the environment.

[0083] Each 1302, 1304, and 1306 may calculate its own sphere. For example, drone 1302 calculates the radius of its replanning sphere 1312, its observation sphere 1332, and its collision sphere 1342. Each of drones 1302, 1304, and 1306 calculates its relative position. When the collision sphere 1342 of drone 1302 intersects the observation sphere 1314 of drone 1304, drone 1302 sends its trajectory 1324 and its priority to drone 1304. If drone 1302 has a higher priority than drone 1304, the reference frame of drone 1304 is aligned with the reference frame of drone 1302. The trajectory 1322 of drone 1302 is compared with the trajectory 1324 of drone 1304 in the aligned reference frame to determine whether the collision sphere 1342 of drone 1302 intersects the collision sphere 1344 of drone 1304. If a collision is predicted, drone 1304 may replan its trajectory to avoid drone 1302 by considering the trajectory of drone 1302 as an obstacle.

[0084] Fig.1414 is a flow chart illustrating the use of hierarchical trajectory planning in accordance with certain embodiments. At operation 1402, the drone calculates the relative positions of its neighbors. The drone has received positions from its neighbors. At operation 1404, the drone determines whether any neighbors are within its observation sphere. If there are no drones or obstacles in its observation sphere, the drone continues on its flight path without change and waits to re-perform operation 1402. For example, the drone may determine whether a neighbor is within its observation sphere on a time schedule based on travel distance, based on the number of neighbors, or a combination of these.

[0085] If there are neighbors within its observation sphere, the drone calculates its priority as needed in operation 1408. In the example, the drone calculates its priority as a random number with uniform density between 0 and 1. The drone can then transmit its priority and current planned trajectory so that its neighboring drones can receive the priority and the trajectory. If the drone determines that the neighboring drone has the same priority, the drone can calculate a new priority and transmit the new priority to the neighboring drone. The neighboring drone can also determine that the drone is within its observation sphere and send its priority and trajectory. At operation 1406, the drone can thus receive the priority of its neighbor and the current planned trajectory. At operation 1410, the drone compares its priority with the priority of its neighbor. If the drone has a higher priority than the neighboring drone, the drone can continue its trajectory while the neighboring drone will determine whether a collision will occur and adjust its trajectory accordingly.

[0086] If the drone has a lower priority than its neighbors, the drone will adjust its trajectory if a future collision is detected. At operation 1412, the drone aligns its trajectory with the trajectory of its neighbor drones. Once the trajectories are aligned, the drone may determine at operation 1414 whether a future collision with a neighbor drone will occur. If there is no collision, the drone may continue with its flight plan unchanged. If there is a detected collision, at operation 1416, the drone may replan its trajectory to avoid the collision. There are known planning algorithms that can generate a new trajectory in a fraction of a second.

[0087] Fig.15Trajectory planning based on sampling techniques according to some embodiments is shown. Trajectory planning takes into account the trajectories of other drones. Two drones 1502 and 1504 fly in the same environment and share trajectory information. Drone 1504 can receive the trajectory of drone 1502. In this example, drone 1504 will update its trajectory to avoid a possible collision with drone 1502. The trajectory received by drone 1504, which will replan its flight path, is parameterized with respect to its discretized time step. For example, drone 1504 receives the trajectory of drone 1502 and is parameterized using the time step of drone 1504. This trajectory is shown as flight path 1512, which is essentially the current flight path of drone 1502. Subsequently, a path plan can be generated using a sampling-based method to assign a time step to each segment. If the time step of any generated segment is equal to the segment of the other drone, this segment is discarded because a collision will occur. In this example, a possible collision will occur in segment 1556 because the time segments are equal and intersecting, such as 4h and intersecting. In this way, the proposed trajectory planning algorithm incorporates dynamic obstacles and treats the drone 1502 trajectory as a "temporary obstacle", thereby obtaining one or more collision-free trajectories even in the presence of conflicting paths. For example, trajectory 1514 is an example of avoiding a detected collision along segment 1556.

[0088] Example Machine Description

[0089] Fig.16 A block diagram of a computing device 1600 is illustrated, within which an instruction set or sequence of instructions may be executed to cause the device to perform an example of any of the methods discussed herein. In alternative embodiments, the machine 1600 may be operated as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 1600 may operate within the capabilities of a server, a client, or both in a server-client network environment. In an example, the machine 1600 may act as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. The machine 1600 may be a user equipment (UE), an evolved node B (eNB), a Wi-Fi access point (AP), a Wi-Fi station (STA), a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a smart phone, a network device, a network router, a switch or a bridge, or any machine capable of (sequentially or otherwise) executing instructions specifying actions taken by the machine. Further, while a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations.

[0090] Examples as described herein may include logic or multiple components, modules or mechanisms, or may operate on logic or multiple components, modules or mechanisms. A module is a tangible entity (e.g., hardware) that can perform a specified operation, and may be configured or arranged in a particular manner. In an example, a circuit (e.g., internally or relative to an external entity such as other circuits) may be arranged as a module in a specified manner. In an example, all or part of one or more computer systems (e.g., independent client or server computer systems) or one or more hardware processors may be configured by firmware or software (e.g., instructions, application parts, or applications) to operate a module for performing a specified operation. In an example, software may reside on a machine-readable medium. In an example, the software, when executed by the underlying hardware of a module, causes the hardware to perform a specified operation.

[0091] Thus, the term "module" is understood to encompass a tangible entity that is physically constructed, specifically configured (e.g., hardwired), or temporarily (e.g., transiently) configured (e.g., programmed) to operate in a specified manner or to perform part or all of any operations described herein. Considering examples in which modules are temporarily configured, each of these modules need not be instantiated at any one time. For example, where the modules include a general-purpose hardware processor configured using software, the general-purpose hardware processor may be configured as various different modules at different times. The software may configure the hardware processor accordingly, for example, so as to constitute a particular module at one time instance and to constitute different modules at different time instances.

[0092] The machine (e.g., computer system) 1600 may include a hardware processor 1602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 1604, and a static memory 1606, some or all of which may communicate with each other via an interconnect (e.g., a bus) 1608. The machine 1600 may also include a display unit 1610, an alphanumeric input device 1612 (e.g., a keyboard), and a user interface (UI) navigation device 1614 (e.g., a mouse). In an example, the display unit 1610, the input device 1612, and the UI navigation device 1614 may be a touch screen display. The machine 1600 may additionally include a storage device (e.g., a drive unit) 1616, a signal generating device 1618 (e.g., a speaker), a network interface device 1620, and one or more sensors (such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors). The machine 1600 may include an output controller 1628 that connects to or controls one or more peripheral devices (e.g., a printer, a card reader, etc.), such as a serial (e.g., universal serial bus (USB)), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection.

[0093] The storage device 1616 may include a machine-readable medium 1624 on which is stored one or more sets of data structures or instructions 1622 (e.g., software) embodied or utilized by any one or more of the techniques or functionality described herein. The instructions 1624 may also reside, completely or at least partially, within the main memory 1604, within the static memory 1606, or within the hardware processor 1602 during execution thereof by the machine 1600. In an example, one or any combination of the hardware processor 1602, the main memory 1604, the static memory 1606, or the storage device 1616 may constitute a machine-readable medium.

[0094] Although the machine-readable medium 1622 is illustrated as a single medium, the term “machine-readable medium” may include a single medium or multiple media (eg, a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 1624 .

[0095] The term "machine-readable medium" includes any medium capable of storing, encoding, or carrying instructions for execution by the machine 1600 and causing the machine 1600 to perform any one or more of the techniques of the present disclosure, or capable of storing, encoding, or carrying data structures used by or associated with such instructions. Non-limiting examples of machine-readable media may include solid-state memory and optical and magnetic media. Specific examples of machine-readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; random access memory (RAM); and CD-ROM and DVD-ROM disks. In some examples, the machine-readable medium may include non-transient machine-readable media. In some examples, the machine-readable medium may include machine-readable media that is not a transient propagating signal.

[0096] The instructions 1624 may be further sent or received via a communication network 1626 using a transmission medium via a network interface device 1620 utilizing any of a number of transmission protocols (e.g., frame relay, Internet Protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), a mobile telephone network (e.g., a cellular network), a plain old telephone (POTS) network, and a wireless data network (e.g., a wireless network called The Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards, known as The network interface device 520 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connecting to the communication network 526. In an example, the network interface device 520 may include multiple antennas that communicate wirelessly using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. In some examples, the network interface device 520 may communicate wirelessly using multi-user MIMO technology. The term "transmission medium" shall include any intangible medium that is capable of storing, encoding, or carrying instructions executed by the machine 500, and "transmission medium" includes digital or analog communication signals or other intangible media for facilitating the communication of such software.

[0097] Additional Notes and Examples:

[0098] Example 1 is a system for drone collision avoidance, the system comprising: a camera for capturing an image of an environment, wherein the camera is oriented in a direction; a receiver for receiving second feature information from an external source, wherein the second feature information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction; a processing circuit for: extracting first feature information from the captured image; matching the first feature information with the second feature information; transforming a second local reference system of the second feature information to a first local reference system of the first feature information to determine a position of the external source; and determining whether a collision with the external source will occur based on the position of the external source and a current flight trajectory.

[0099] In Example 2, the subject matter of Example 1 includes, wherein a collision with an external source is determined and wherein the processing circuit is further configured to change a current flight trajectory of the drone based on the collision.

[0100] In Example 3, the subject matter of Examples 1-2 includes, wherein the external source is a second drone.

[0101] In Example 4, the subject matter of Examples 1-3 includes, wherein the external source is a stationary object.

[0102] In Example 5, the subject matter of Examples 1-4 includes a transmitter for broadcasting first characteristic information.

[0103] In Example 6, the subject matter of Examples 1-5 includes, wherein the receiver is further configured to receive the position of a third source, the third source does not have a feature that overlaps with the first feature information, and wherein the processing circuit is further configured to: determine whether a collision with the third source will occur based on the position of the third source and the current flight trajectory; and change the current flight trajectory based on the collision with the third source.

[0104] In Example 7, the subject matter of Examples 1-6 includes, wherein the receiver is further configured to receive trajectory information of an external source from the external source.

[0105] In Example 8, the subject matter of Example 7 includes, wherein the processing circuit is further configured to: determine a relative velocity of an external source based on trajectory information of the external source; determine a virtual force based on the relative velocity of the external source; add the virtual force to a position based on a current flight trajectory to create an attitude controller input; and provide the attitude controller input to the attitude controller, wherein the virtual force changes the acceleration of the drone from the acceleration of the current flight trajectory.

[0106] In Example 9, the subject matter of Examples 7-8 includes that the receiver is further configured to receive external source priority information from an external source, and wherein in order to determine whether a collision with the external source will occur, the processing circuit is configured to: determine an observation sphere of the external source based on the position of the external source; determine a collision sphere of the drone based on the current position of the drone; determine an intersection between the observation sphere of the external source and the collision sphere of the drone; align the current flight trajectory with the flight trajectory of the external source based on the intersection of the observation sphere of the external source and the collision sphere of the drone; and determine that a collision between the drone and the external source will occur based on the intersection of the aligned trajectories.

[0107] In Example 10, the subject matter of Examples 7-9 includes, wherein in order to determine whether a collision with an external source will occur, the processing circuit is configured to: determine a first position of the drone based on the current flight trajectory for multiple time points; determine a second position of the external source based on trajectory information of the external source for multiple time points; if paths between consecutive points from the first position and the second position intersect at the same time point, it is determined that a collision will occur.

[0108] In Example 11, the subject matter of Examples 1-10 includes, wherein the direction is upward.

[0109] In Example 12, the subject matter of Examples 1-11 includes, wherein the direction is downward.

[0110] Example 13 is a machine-implemented method for drone collision avoidance, the method comprising: capturing an image of an environment using a camera, wherein the camera is oriented in a direction; receiving second feature information from an external source using a receiver, wherein the second feature information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction; extracting first feature information from the captured image using a processing circuit; matching the first feature information with the second feature information; transforming a second local reference system of the second feature information to a first local reference system of the first feature information to determine a position of the external source; and determining whether a collision with the external source will occur based on the position of the external source and the current flight trajectory.

[0111] In Example 14, the subject matter of Example 13 includes, wherein the method further includes changing the current flight trajectory of the drone based on determining that a collision with the external source will occur.

[0112] In Example 15, the subject matter of Examples 13-14 includes, wherein the external source is a second drone.

[0113] In Example 16, the subject matter of Examples 13-15 includes, wherein the external source is a stationary object.

[0114] In Example 17, the subject matter of Examples 13-16 includes broadcasting the first characteristic information using a transmitter.

[0115] In Example 18, the subject matter of Examples 13-17 includes receiving the position of a third source, the third source not having a feature that overlaps with the first feature information; determining whether a collision with the third source will occur based on the position of the third source and the current flight trajectory; and changing the current flight trajectory based on the collision with the third source.

[0116] In Example 19, the subject matter of Examples 13-18 includes receiving trajectory information from an external source from an external source.

[0117] In Example 20, the subject matter of Example 19 includes determining a relative velocity of an external source based on trajectory information of the external source; determining a virtual force based on the relative velocity of the external source; adding the virtual force to a position based on a current flight trajectory to create an attitude controller input; and providing the attitude controller input to the attitude controller, wherein the virtual force changes the acceleration of the drone from the acceleration of the current flight trajectory.

[0118] In Example 21, the subject matter of Examples 19-20 includes receiving external source priority information from an external source, and wherein determining whether a collision with the external source will occur includes: determining an observation sphere of the external source based on the position of the external source; determining a collision sphere of the drone based on the current position of the drone; determining an intersection between the observation sphere of the external source and the collision sphere of the drone; aligning a current flight trajectory with a flight trajectory of the external source based on the intersection of the observation sphere of the external source and the collision sphere of the drone; and determining that a collision between the drone and the external source will occur based on the intersection of the aligned trajectories.

[0119] In Example 22, the subject matter of Examples 19-21 includes, wherein determining whether a collision with an external source will occur includes: determining a first position of the drone based on a current flight trajectory for multiple time points; determining a second position of the external source based on trajectory information of the external source for multiple time points; and determining that a collision will occur if paths between consecutive points from the first position and the second position intersect at the same time point.

[0120] In Example 23, the subject matter of Examples 13-22 includes, wherein the direction is upward.

[0121] In Example 24, the subject matter of Examples 13-23 includes, wherein the direction is downward.

[0122] Example 25 is at least one computer-readable medium for drone collision avoidance, comprising instructions that, when executed by a machine, cause the machine to perform the following operations: capture an image of an environment, wherein a camera is oriented in a direction; receive second feature information from an external source, wherein the second feature information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction; extract first feature information from the captured image; match the first feature information with the second feature information; transform a second local reference system of the second feature information to a first local reference system of the first feature information to determine a position of the external source; and determine whether a collision with the external source will occur based on the position of the external source and the current flight trajectory.

[0123] In Example 26, the subject matter of Example 25 includes, wherein the instructions further include changing a current flight trajectory of the drone based on determining that a collision with an external source will occur.

[0124] In Example 27, the subject matter of Examples 25-26 includes, wherein the external source is a second drone.

[0125] In Example 28, the subject matter of Examples 25-27 includes, wherein the external source is a stationary object.

[0126] In Example 29, the subject matter of Examples 25-28 includes, wherein the instructions further include broadcasting the first characteristic information using a transmitter.

[0127] In Example 30, the subject matter of Examples 25-29 includes, wherein the instructions further include: receiving the position of a third source, the third source not having a feature that overlaps with the first feature information; determining whether a collision with the third source will occur based on the position of the third source and the current flight trajectory; and changing the current flight trajectory based on the collision with the third source.

[0128] In Example 31, the subject matter of Examples 25-30 includes, wherein the instructions further include receiving trajectory information from an external source from an external source.

[0129] In Example 32, the subject matter of Examples 30-31 includes, wherein the instructions further include: determining a relative velocity of an external source based on trajectory information of the external source; determining a virtual force based on the relative velocity of the external source; adding the virtual force to a position based on a current flight trajectory to create an attitude controller input; and providing the attitude controller input to the attitude controller, wherein the virtual force changes the acceleration of the drone from the acceleration of the current flight trajectory.

[0130] In Example 33, the subject matter of Examples 30-32 includes, wherein the instructions further include receiving external source priority information from an external source, and wherein the instructions for determining whether a collision with the external source will occur include: determining an observation sphere of the external source based on the position of the external source; determining a collision sphere of the drone based on the current position of the drone; determining an intersection of the observation sphere of the external source with the collision sphere of the drone; aligning a current flight trajectory with a flight trajectory of the external source based on the intersection of the observation sphere of the external source with the collision sphere of the drone; and determining that a collision between the drone and the external source will occur based on the intersection of the aligned trajectories.

[0131] In Example 34, the subject matter of Examples 30-33 includes, wherein the instructions for determining whether a collision with an external source will occur include: determining a first position of the drone based on a current flight trajectory for multiple time points; determining a second position of the external source based on trajectory information of the external source for multiple time points; and determining that a collision will occur if paths between consecutive points from the first position and the second position intersect at the same time point.

[0132] In Example 35, the subject matter of Examples 25-34 includes, wherein the direction is upward.

[0133] In Example 36, the subject matter of Examples 25-35 includes, wherein the direction is downward.

[0134] Example 37 is a device for aircraft-to-aircraft communication, the device comprising: a device for capturing an image of an environment, wherein the camera is oriented in a direction; a device for receiving second characteristic information from an external source, wherein the second characteristic information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction; a device for extracting first characteristic information from the captured image; a device for matching the first characteristic information with the second characteristic information; a device for transforming a second local reference system of the second characteristic information to a first local reference system of the first characteristic information to determine a position of an external source; and a device for determining whether a collision with the external source will occur based on the position of the external source and a current flight trajectory.

[0135] In Example 38, the subject matter of Example 37 includes means for changing a current flight trajectory of the drone based on determining that a collision with an external source will occur.

[0136] In Example 39, the subject matter of Examples 37-38 includes, wherein the external source is a second drone.

[0137] In Example 40, the subject matter of Examples 37-39 includes, wherein the external source is a stationary object.

[0138] In Example 41, the subject matter of Examples 37-40 includes means for broadcasting the first characteristic information using a transmitter.

[0139] In Example 42, the subject matter of Examples 37-41 includes a device for receiving the position of a third source, the third source not having a feature that overlaps with the first feature information; a device for determining whether a collision with the third source will occur based on the position of the third source and the current flight trajectory; and a device for changing the current flight trajectory based on the collision with the third source.

[0140] In Example 43, the subject matter of Examples 37-42 includes means for receiving trajectory information of an external source from an external source.

[0141] In Example 44, the subject matter of Examples 19-43 includes a device for determining a relative velocity of an external source based on trajectory information of the external source; a device for determining a virtual force based on the relative velocity of the external source; a device for adding a virtual force to a position based on a current flight trajectory to create an attitude controller input; and a device for providing the attitude controller input to the attitude controller, wherein the virtual force changes the acceleration of the drone from the acceleration of the current flight trajectory.

[0142] In Example 45, the subject matter of Examples 19-44 includes a device for receiving external source priority information from an external source, and wherein the device for determining whether a collision with the external source will occur includes: a device for determining an observation sphere of the external source based on the position of the external source; a device for determining a collision sphere of the drone based on the current position of the drone; a device for determining the intersection of the observation sphere of the external source and the collision sphere of the drone; a device for aligning the current flight trajectory with the flight trajectory of the external source based on the intersection of the observation sphere of the external source and the collision sphere of the drone; and a device for determining that a collision between the drone and the external source will occur based on the intersection of the aligned trajectories.

[0143] In Example 46, the subject matter of Examples 19-45 includes, wherein the device for determining whether a collision with an external source will occur includes: a device for determining a first position of the drone based on a current flight trajectory for multiple time points; a device for determining a second position of the external source based on trajectory information of the external source for multiple time points; and a device for determining that a collision will occur when paths between consecutive points from the first position and the second position intersect at the same time point.

[0144] In Example 47, the subject matter of Examples 37-46 includes, wherein the direction is upward.

[0145] In Example 48, the subject matter of Examples 37-47 includes, wherein the direction is downward.

[0146] Example 49 is at least one machine-readable medium comprising instructions that, when executed by a processor circuitry, cause the processor circuitry to perform operations to implement any of Examples 1-48.

[0147] Example 50 is an apparatus comprising means for implementing any of Examples 1-48.

[0148] Example 51 is a system for implementing any of Examples 1-48.

[0149] Example 52 is a method for implementing any of Examples 1-48.

[0150] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The accompanying drawings illustrate specific embodiments that can be implemented by way of illustration. In this article, these embodiments are also referred to as "examples". Such examples may include elements other than the elements shown or described. However, examples including the elements shown and described are also contemplated. In addition, examples using any combination or arrangement of those elements shown or described (or one or more aspects thereof) are also contemplated, either with reference to a specific example (or one or more aspects thereof), or with reference to other examples (or one or more aspects thereof) shown or described herein.

[0151] The publications, patents, and patent documents referred to in this document are incorporated herein by reference in their entirety, as if individually incorporated by reference. In the event of inconsistent usages between this document and those documents incorporated by reference, the usage in the incorporated references should be considered supplementary to that of this document; for irreconcilable inconsistencies, the usage in this document controls.

[0152] In this document, the terms "a" or "an" are used to include one or more than one as is common in patent documents, regardless of any other examples or uses of "at least one" or "one or more". In this document, the term "or" is used to indicate a non-exclusive or, such that "A or B" includes "A but not B", "not A", and "A and B", unless otherwise indicated. In the appended claims, the terms "including" and "in which" are used as the equivalents of the terms "comprising" and "wherein" in plain English, respectively. Also, in some claims, the terms "including" and "comprising" are open-ended, that is, the system, device, article, or process includes elements other than those listed later, and such terms in the claims are still deemed to fall within the scope of the claims. In addition, in the appended claims, the terms "first", "second", and "third", etc., are used only as labels and are not intended to indicate numerical order to their objects.

[0153] The above embodiments may be implemented in various hardware configurations, which may include a processor for executing instructions for implementing the above techniques. Such instructions may be included in a readable medium (such as a suitable storage medium or memory) or other processor executable medium.

[0154] The above description is intended to be exemplary and non-restrictive. For example, the above examples (or one or more aspects thereof) may be used in combination with each other. Other embodiments such as those implemented by a person of ordinary skill in the art after carefully reading the above description may also be used. The abstract allows the reader to quickly determine the nature of the present technology disclosure. It should be understood that the abstract will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above specific embodiments, various features may be grouped together to make the disclosure smooth. However, the claims may not state every feature disclosed herein, because the embodiments may characterize a subset of the features. Further, the embodiments may include fewer features than those disclosed in a particular example. Therefore, the attached claims are thus incorporated into the specific embodiments, and a claim exists independently as a separate embodiment. The scope of the embodiments disclosed herein should be determined with reference to the attached claims together with the full scope of equivalents to which such claims are entitled.

Claims

1. A device for avoiding collision of unmanned aerial vehicles, the device comprising: Processing circuitry for: extracting first feature information from an image, the image being captured from a first camera oriented in a direction; receiving second feature information from an external source, wherein the second feature information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction; Matching the first feature information with the second feature information; transforming a second local reference system of the second feature information to a first local reference system of the first feature information to determine a position of the external source; as well as A determination is made based on the location of the external source and the current flight trajectory whether a collision with the external source will occur.

2. The device according to claim 1, characterized in that A collision with the external source is determined and wherein the processing circuit is further configured to change the current flight trajectory based on the collision.

3. The device according to claim 1, characterized in that The external source is a second drone.

4. The device according to claim 1, characterized in that The external source is a stationary object.

5. The device according to claim 1, characterized in that Further comprising a transmitter for broadcasting the first characteristic information.

6. The device according to claim 1, characterized in that The processing circuit is further configured to receive a location of a third source, the third source having no characteristics overlapping with the first characteristic information, and wherein the processing circuit is further configured to: determining whether a collision with the third source will occur based on the location and current flight trajectory of the third source; and The current flight trajectory is altered based on a determination that a collision with the third source will occur.

7. The device according to any one of claims 1 to 6, characterized in that The processing circuit is further configured to receive trajectory information of the external source from the external source.

8. The device according to claim 7, characterized in that The processing circuit is further configured to: determining a relative speed of the external source based on the trajectory information of the external source; determining a virtual force based on the relative velocity of the external source; adding the virtual force to a position based on the current flight trajectory to create a gesture controller input; as well as The attitude controller input is provided to an attitude controller, wherein the virtual force changes an acceleration of the drone from an acceleration of the current flight trajectory.

9. The device according to claim 7, characterized in that The processing circuit is further configured to receive external source priority information from the external source, and wherein to determine whether a collision with the external source will occur, the processing circuit is configured to: determining an observation sphere of the external source based on the position of the external source; determining a collision sphere for the drone based on a current position of the drone; Determining that the observation sphere of the external source intersects the collision sphere of the drone; aligning a current flight trajectory with a flight trajectory of the external source based on an intersection of the observation sphere of the external source and the collision sphere of the drone; as well as A determination is made based on the intersection of the aligned trajectories that a collision between the drone and the external source will occur.

10. The device according to claim 7, characterized in that To determine whether a collision with the external source will occur, the processing circuit is configured to: determining a first position of the drone based on the current flight trajectory for a plurality of time points; determining a second position of the external source based on the trajectory information of the external source for the plurality of time points; as well as If paths between consecutive points from the first location and the second location intersect at the same point in time, it is determined that a collision will occur.

11. The device according to any one of claims 1 to 6, characterized in that The direction is upwards.

12. The device according to any one of claims 1 to 6, characterized in that The direction is downward.

13. A machine-implemented method for drone collision avoidance, the method comprising: capturing an image of an environment using a camera, wherein the camera is oriented in a direction; receiving second feature information from an external source using a receiver, wherein the second feature information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction; extracting first feature information from the captured image using processing circuitry; Matching the first feature information with the second feature information; transforming a second local reference system of the second feature information to a first local reference system of the first feature information to determine a position of the external source; as well as A determination is made based on the location of the external source and the current flight trajectory whether a collision with the external source will occur.

14. The method according to claim 13, characterized in that The method further includes changing the current flight trajectory of the drone based on determining that a collision with the external source will occur.

15. The method according to claim 13, characterized in that The external source is a second drone.

16. The method according to claim 13, characterized in that The external source is a stationary object.

17. The method according to claim 13, characterized in that Further comprising broadcasting the first characteristic information using a transmitter.

18. The method according to claim 13, characterized in that Further including: receiving a location of a third source, the third source having no characteristics overlapping with the first characteristic information; determining whether a collision with the third source will occur based on the position of the third source and the current flight trajectory; as well as The current flight trajectory is altered based on a determination that a collision with the third source will occur.

19. The method of claim 13, wherein: Further comprising receiving, from a second source, trajectory information of the second source.

20. The method of claim 19, wherein: Further including: determining a relative speed of the external source based on the trajectory information of the external source; determining a virtual force based on the relative velocity of the external source; adding the virtual force to a position based on the current flight trajectory to create a gesture controller input; as well as The attitude controller input is provided to an attitude controller, wherein the virtual force changes an acceleration of the drone from an acceleration of the current flight trajectory.

21. The method of claim 19, wherein: external source priority information is received from the external source, and wherein determining whether a collision with the external source will occur comprises: determining an observation sphere of the external source based on the position of the external source; determining a collision sphere for the drone based on a current position of the drone; Determining that the observation sphere of the external source intersects the collision sphere of the drone; aligning a current flight trajectory with a flight trajectory of the external source based on an intersection of the observation sphere of the external source with the collision sphere of the drone; and A determination is made based on the intersection of the aligned trajectories that a collision between the drone and the external source will occur.

22. The method of claim 19, wherein: Determining whether a collision with the external source will occur includes: determining a first position of the drone based on the current flight trajectory for a plurality of time points; determining a second position of the external source based on the trajectory information of the external source for the plurality of time points; If paths between consecutive points from the first location and the second location intersect at the same point in time, it is determined that a collision will occur.

23. The method of claim 13, wherein: The direction is downward.

24. At least one computer-readable medium for drone collision avoidance, comprising instructions that, when executed by a machine, cause the machine to: capturing an image of an environment, wherein the camera is oriented in a direction; receiving second feature information from an external source, wherein the second feature information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction; extracting first feature information from the captured image; Matching the first feature information with the second feature information; transforming a second local reference system of the second feature information to a first local reference system of the first feature information to determine a position of the external source; as well as A determination is made based on the location of the external source and the current flight trajectory whether a collision with the external source will occur.

25. An apparatus for aircraft-to-aircraft communication, the apparatus comprising: Means for capturing an image of an environment, wherein the camera is oriented in a direction; means for receiving second characteristic information from an external source, wherein the second characteristic information is extracted from a second image of the environment, the second image being captured by a second camera oriented in the direction; means for extracting first feature information from the captured image; means for matching the first characteristic information with the second characteristic information; means for transforming a second local reference frame of the second characteristic information to a first local reference frame of the first characteristic information to determine a position of the external source; as well as Means for determining whether a collision with the external source will occur based on the location and current flight trajectory of the external source.

Images