Projection-based cooperative collision avoidance

Abstract

Methods and apparatus for drone collision avoidance. The drone's processing circuitry extracts information from an encoded image captured by a detection device having a field of view overlapping the encoded image. Based on the extracted information, a determination is made as to whether a collision with an external source will occur on the drone's flight path. The drone's flight path is then altered to avoid the collision.

Description

[0001] Priority application

[0002] This application claims the benefit of priority to U.S. Application Serial No. 15 / 941,363, filed on March 30, 2018, which is incorporated herein by reference in its entirety. Technical Field

[0003] The embodiments described herein generally relate to collision avoidance for aircraft (such as drones). Embodiments are implemented in hardware, software, or a combination of both. Background Technology

[0004] Current collision avoidance capabilities for area-specific aircraft (such as drones) have various limitations. For example, collision avoidance methods may be limited to detecting large objects (such as walls), have slow reaction times, and / or rely on 3D cameras to locate nearby objects. These capabilities may not be suitable for agent-to-agent collision avoidance. A typical scenario where existing methods fail is when two drones are flying in collision paths at a 90-degree angle. This can occur in open areas or areas such as corridors.

[0005] Another known method for collision avoidance relies on external localization or simultaneous localization and mapping (SLAM) algorithms. However, this method only works after the aircraft have a shared map. Furthermore, real-time SLAM algorithms require powerful computers that are typically not available on UAVs. SLAM algorithms can also be unreliable in dynamic environments.

[0006] Another approach to collision avoidance is based on sharing sparse visual features to achieve relative positioning and cooperative planning for collision avoidance maneuvers. Such avoidance is then provided through radio communication between adjacent drones. However, the reliance on radio communication and coordination between the individual agents presents coordination challenges. Attached Figure Description

[0007] Figure 1 The illustration shows a captured projected image for collision avoidance according to some embodiments.

[0008] Figures 2A-2B The illustration depicts a cooperative collision avoidance technique according to some embodiments.

[0009] Figure 3 The illustration depicts a cooperative collision avoidance technique according to some embodiments.

[0010] Figures 4A-4B The illustration shows a cooperative collision avoidance technique for three drones according to some embodiments.

[0011] Figure 5A flowchart for attitude adjustment according to some embodiments is shown.

[0012] Figure 6 The illustration shows a projected image for collision avoidance according to an illustrative embodiment.

[0013] Figure 7 The illustration shows a projected image for collision avoidance according to an illustrative embodiment.

[0014] Figure 8 The illustration shows two drone projection images for a cooperative collision avoidance technology according to some embodiments.

[0015] Figure 9 This is a flowchart for nearest neighbor localization according to some embodiments.

[0016] Figure 10 A block diagram of a computing device is shown, within which an example of an instruction set or sequence of instructions can be executed to cause the device to perform any of the methods discussed herein. Detailed Implementation

[0017] Many current applications of drones—such as precision agriculture, inspection, search and rescue, and warehousing and inventory systems—can benefit from the interaction of multiple drones, including reducing the time required to complete such tasks. However, communication between drones is often challenging due to obstacles obstructing the line of sight and causing multiple reflections of radio frequency (RF) signals. Therefore, such obstacles limit the line of sight and communication between drones when attempting to perform work and tasks. Even for drone enthusiasts, similar problems arise when other drones are present in the area.

[0018] In the example, the disclosed collision avoidance procedure allows multiple drones to coexist safely and operate in an environment containing numerous obstacles. The camera system also captures images of other drones projecting within the environment. The projected images include encoded data associated with the drones. Such data includes, but is not limited to, flight history, flight path, flight trajectory, drone speed, drone position data, etc. For collision avoidance purposes, the camera system captures images and decodes the encoded data. Specifically, based on the decoded information associated with the projecting drone, the system captures the drones to change their flight paths accordingly.

[0019] The proposed process is simpler and faster than current SLAM-based methods, which require registering agents (e.g., drones, stationary objects, ships, etc.) in a global map before any coordinated actions can be planned. Various publicly available collision avoidance techniques do not require a common frame of reference and an environmental map. The proposed technique improves upon simple reaction-based methods based on the detection and tracking of dynamic obstacles by allowing agents to anticipate potential collision avoidance scenarios. Data acquired by different agents from encoded projections can be parsed and used to determine the relative localization of two agents with their respective neighbors. Furthermore, the proposed process avoids the loss of communication and coordination due to reflected RF signals by eliminating the need for RF signals while still allowing for relative localization between the two agents.

[0020] Figure 1 The illustration shows an image of a projected image captured for collision avoidance according to some embodiments. The drone operating environment 100 includes a first drone 105 and a second drone 110. Each drone 105 and 110 can be of any type, and in this example, each drone includes a frame 115 and a propeller 120, allowing autonomous flight over an environment 125 that includes numerous obstacles 130. Obstacles 130 may include walls, ceilings, aircraft, trees, other drones, etc. Each drone 105 and 110 also includes a collision avoidance system 135.

[0021] The collision avoidance system 135 includes processing circuitry that operates to prevent drones 105 and 110 from colliding with external sources such as walls, buildings, ceilings, obstacles, other drones, etc. The collision avoidance system 135 includes a projection device 140 that receives information related to drones 105 or 110 in real time and encodes the projected image 145. This information includes, but is not limited to, drone speed, drone orientation, drone rotation, drone acceleration, the projected drone path, and the projected drone destination. The encoded projection can be of any type, including QR codes, CCTag projections, barcodes, spectral imaging, patterned imaging, combinations of these projections, etc. The projection device 140 is any of the following: a tilt / pan camera or projector, a 3D camera (such as Kinect). TM RealSense TM Structural sensors, Asus Xtion TM Examples of such devices include: light-emitting diodes (LEDs), red, green, and blue (RGB) cameras or projectors, etc. Specifically, the projection device 140 can project not only downwards onto the ground, but also in multiple directions, such as onto objects including buildings and other obstacles, or upwards onto the ceiling, etc.

[0022] The collision avoidance system 135 also includes a detection device 150 having a field of view 155 for detecting the encoded image 145 to retrieve encoded messages for decoding. The detection device 150 in this example is a tilt / translation 3D camera. Such 3D cameras include, but are not limited to, Kinect. TM RealSense TM Structural sensors, Asus Xtion TM Therefore, the first UAV 105 projects an encoded image 145 using a projection device 140, and the encoded image 145 is detected by a detection device 150 of the second UAV 110 for decoding.

[0023] Drone 105 or 110 detects image 145 based on encoded image 145 and determines the rotation and translation between projection device 140 and detection device 150. In this way, the drones position themselves relative to each other. Similarly, in embodiments where the encoded image 145 is patterned and certain features of the pattern (such as squares) are aligned, transformation data is obtained. Specifically, once the pattern is aligned, data embedded within the pattern is extracted. Such data or information includes, but is not limited to, the expected trajectory for the next few seconds, the expected trajectory of neighboring drones for the next few seconds, the drone's target, the targets of neighboring drones, the latest mapping of the drone's surrounding environment, the drone's altitude, the current translation, tilt, or roll of projection device 140, etc.

[0024] As an example of positioning and alignment, one or more controllers decode the message, and drones 105 and 110 use this information to perform positioning. In this example, both the projection device 140 and the detection device 150 are cameras with a downward-facing camera orientation. In this example, the drone is at position (x0, y0, z0). T The location, and has a planned location in the next X seconds (x p y p , z p ) T Then the drone camera can be controlled to point (x) p y p , z g ) T , where z g It's the ground, for example, 0. In Figure 1 In the middle, the first drone 105 is pointing towards it (x) p y p , z g ) T Location. Let... It is from (x0, y0, z0) T Pointing to (x) p y p, z g ) T unitary vector:

[0025]

[0026] Let R(ψ, φ, θ) = R(t) be the rotation matrix that defines the transformation from the initial orientation of the UAV to the current orientation given φ, θ, ψ (Euler angles for roll, pitch, and yaw).

[0027]

[0028] also, This represents a unit vector pointing from the point of takeoff of the drone towards the direction in front of the drone, and It is a unit vector pointing forward from the drone at a given time t. Note:

[0029]

[0030] Therefore, there exists a transformation matrix R. G (ψ G φ G θ G The gimbal can perform the transformation matrix R. G (ψ G φ G θ G To move the camera from the direction of Orientation is pointing This relationship can be written as:

[0031]

[0032] Therefore, the positioning and alignment between multiple drones were completed.

[0033] Therefore, one or more controllers, including processing circuitry, alter the drone's flight path to avoid collisions without using radio frequency. In this example, the flight path is automatically altered by one or more controllers. These controllers utilize any method for collision avoidance, including but not limited to reactive collision avoidance algorithms, conflict resolution and hierarchical techniques, and virtual force techniques.

[0034] Figure 2A and Figure 2B The illustration depicts a cooperative collision avoidance technique according to some embodiments. In some examples, this technique comprises a collision avoidance system (such as...) Figure 1 The collision avoidance system 135 is implemented. Figure 2AIn this scenario, a first drone 200 and a second drone 205 traverse an environment 210. In this example, the flight path 215 of the first drone 200 overlaps with the flight path 220 of the second drone, indicating a collision path 225. In the environment 210, an encoded image 230 projected from the camera of the first drone 200 has a projection area 235 overlapping with the field of view 240 of the camera of the second drone 205. The projection area 235 is controlled by turning the camera based on the speed and maneuverability of the first drone 200. The encoded image 230 includes encoded information related to trajectory, flight path, speed, and other similar information that can be used to determine the relative position of the first drone 200, represented by the rotation and transformation of the first drone 200. The camera of the second drone 205 detects and decodes the encoded information. Based on this information, the second drone selects an alternative flight path 245 to avoid a collision (e.g., ...). Figure 2B (As shown).

[0035] In the example, the encoded image 230 is a pattern, and the camera with a field of view 240 is a translation / tilt camera. The translation / tilt camera can be a visible spectrum camera, an infrared camera, etc. The projection area 235 is on the ground, ceiling, or object that the first drone 200 plans to fly over within X seconds. In this way, the nearby agent, i.e., the drone, observes the image at least X / 2 seconds before a collision occurs. Additionally, to allow for localization, the projection 230 includes embedded data such as the expected trajectory, the positions of neighboring drones, the latest mapping of the local area, the current altitude, the current speed, the current translation / tilt / roll that the first drone 200 will traverse in the next few seconds, etc. In this way, radio communication is unnecessary, and the detection speed is enhanced, thus improving collision avoidance.

[0036] Example encoded image 230 is illustrated as a QR-like pattern. This QR-like pattern can utilize different patterns and colors to ensure when more than one projected image exists in an area where the detection device can distinguish between different projections. Alternatively, projected images from different drones are provided intermittently, and the timing of the projection is adjusted based on information received from each encoded image to prevent synchronization of intermittent projections, thereby ensuring that each encoded image is projected at different times. Another example of code separation is the redundancy of information encoded within the projected image. In this way, only portions of the encoded image need to be in the field of view to receive the encoded information and locate other drones.

[0037] Figure 3The illustration depicts a cooperative collision avoidance technique according to some embodiments. In this example, a first human-machine interface 305 and a second drone 300 do not have a line of sight to each other. In this example, a building 310 separates the drones 300 and 305. The first drone 305 projects an encoded image 315, which is partially projected onto the building 310 and partially onto the ground 320. The field of view 325 of the camera of the second drone 300 captures a portion of the partially projected image 315 on the ground 320. The encoded image 315 is encoded with multiple redundancies of data, such that the portion of the partially projected image 315 in the camera's field of view 325 contains data or information that can be decoded by the second drone 300 and used to avoid collisions with the first drone 305.

[0038] Figures 4A-4B The illustration depicts a cooperative collision avoidance technique for three drones according to some embodiments. Three agents in the environment, namely drones 400, 405, and 410, are on collision trajectories, allowing drones 400, 405, and 410 to pre-position themselves with each other and apply virtual forces to the other drones to avoid collisions using a reactive collision avoidance algorithm.

[0039] Each of these drones 400, 405, and 410 may have virtual forces 415, 420, and 425, respectively. The virtual forces may be based on the drone's speed. In an embodiment, the virtual forces may be detected or determined based on encoded images 430, 435, and 440 of each drone 400, 405, and 410. Drones 405 and 410 have speeds relative to a reference frame of drone 400. and And relative velocity can be obtained. and relative position Virtual force can be based on the following formula:

[0040]

[0041] Where λ is a design parameter constraining the size of the potential field, and its nominal / center value is 1. The direction of the force is opposite to the direction pointing towards the UAV 405, and because The factor, the magnitude of the force, decreases inversely / squarely with distance and in the direction of relative velocity. This term decreases as drone 405 moves away from drone 400. When drone 405 moves parallel to drone 400, the value is 1. This term increases as drone 405 moves toward drone 400.

[0042] Virtual forces 415, 420, and 425 can be applied to any number of drones or other objects, causing drone 400 to generate a virtual force 415 for each of the drones and / or other objects. This virtual force 415 can then be added to the default position control implemented by the drone. In an embodiment, the position control exists on top of the attitude control, where the former generates commands for the latter. These virtual forces 415, 420, and 425 can be added to the position control to generate new attitude commands. The mixer takes the attitude commands and generates the corresponding power to be applied to the motors. Therefore, the virtual forces 415, 420, and 425 can affect the acceleration of the drone. This change generates a change in the reference attitude control, which alters the movement of drones 400, 405, and 410.

[0043] As an example, virtual forces 415, 420, and 425 can then be used to update the flight path of drone 400 by adjusting the input to the attitude controller. The same calculations can be performed using a reference from drone 405 to calculate virtual force 420. As drones 400 and 405 fly toward each other, they will repel each other based on their calculated virtual forces 415 and 420. From an externally favorable perspective, drones 400 and 405 appear to repel each other when flying close together. Each drone 400 and 405 adjusts its flight path to avoid a collision. As drones 400 and 405 fly past each other, the virtual force values ​​decrease, and each of drones 400 and 405 will have less impact on the flight path of the other drone.

[0044] Figure 5 A flowchart illustrating attitude adjustment according to some embodiments is shown. The drone has a flight plan 505 input to a position controller 515. Without additional input, the position controller 515 provides commands to an attitude controller 530. The attitude controller 530 provides commands to a mixer 535, which determines appropriate power and applies that appropriate power to a motor 540. Thus, the drone can fly along its planned route 505. However, without additional input, the drone may collide with other drones or objects.

[0045] To avoid other drones, the relative position and relative velocity 510 of the drone are received from an encoded image projected by another drone, detected and decoded by that drone, and provided to the virtual force generator 520. Virtual forces can be generated for drones positioned near the first drone. These virtual forces can then be added 525 to commands from the position controller 515. Thus, the drone can deviate from its planned flight path 505 based on the position and velocity of nearby drones (e.g., drones that might collide with the current drone). While the flowchart focuses on attitude adjustment using virtual forces, other attitude adjustment methods are also conceivable. Specifically, as a result of detecting and decoding information from the encoded projection, collision avoidance methods can be implemented based on this information without using radio frequency.

[0046] Figure 6 The illustration depicts a projected image for collision avoidance according to an illustrative embodiment. An example projected image 600 from a first drone can be used to embed information for detection and utilization by a second drone. In one embodiment, the encoded image 600 is about... Figures 1-4B The described drone uses one of the projections. Image 600 includes both pattern 605 and object 610. This includes a three-dimensional object generated by the projection device. Pattern 605 includes a square 615 and other two-dimensional identifying patterns. Object 610 includes a three-dimensional shape. Pattern 605 and object 610 can be of any size or color. In the example, redundant information is embedded in the pattern and object, including by providing repeating patterns, objects, and colors in the projection. Redundancy is used to ensure that the required information can be shared even if only a portion of the encoded image 600 is within the field of view of the detection device.

[0047] Furthermore, as the characteristics of the drone (such as speed, trajectory, flight plan, etc.) change, the pattern, objects, and colors change to present an updated image 600. Specifically, such changes include, but are not limited to, pattern color, pattern shape, pattern arrangement, object shape, object color, object position, etc. Additionally, in some embodiments, the collision avoidance system detects the surface color, shadows, texture, etc., of the projected area and alters the characteristics of the encoded image 600 (such as color, pattern, objects, etc.) to make the projection more easily detectable by other drones and users. Similarly, in some embodiments, the collision avoidance system detects overlapping projection areas and alters image characteristics to avoid confusion and increase the detectability of the projected image.

[0048] Although Figures 1-4BThe projected image 600 is typically illustrated as a QR code, but different images including different patterns and / or colors can be envisioned. In some embodiments, the example image may embed information based on visual robustness (such as scale invariance, orientation invariance, and illumination invariance). In some embodiments, the example image may embed redundant information in any way, including spatial and temporal redundancy in the image. In some embodiments, the example image may be encoded and decoded using optical and computer vision methods and techniques. In some embodiments, the example image may be separated from multiple overlapping images or sources. In some embodiments, the example image may preserve camera depth resolution.

[0049] Figure 7 An image of a projection for collision avoidance according to an illustrative embodiment is shown. In the embodiment, image 700 is about... Figures 1-4B One of the encoded images used by the described drone. In this example, the projected image is a QR code including a pattern 705 with embedded information. The QR code can be of any size or color. Based on the pattern 705, redundant information is embedded. Specifically, the QR code may include square elements 710 and other redundancies throughout the image. The redundant information is used to ensure that the required information can be shared even if only a portion of image 700 is within the field of view of the detection device.

[0050] Furthermore, as the characteristics of the drone (such as speed, trajectory, flight plan, etc.) change, the pattern arrangement, color, size, etc., are altered to present an updated image 700. Additionally, in some embodiments, the collision avoidance system detects the color of the projection area (such as the ground, buildings, ceiling, etc.) and changes the image's color and / or other characteristics to make the image more easily detectable by other drones. Similarly, in some embodiments, the collision avoidance system detects overlapping images and changes their color, pattern, etc., to avoid confusion and increase the image's detectability.

[0051] Figure 8The illustration shows images projected by two drones for a cooperative collision avoidance technology according to some embodiments. In this example, a first drone and a second drone project an overlapping first encoded image 800A and a second encoded image 800B. Each image 800A and 800B originates from a separate drone and includes embedded information for detection and utilization by the other drone. In this embodiment, the encoded image 800 is a CCTag reference formed by LED illumination included in the projection device. Each image 800A and 800B includes a series of concentric circles 805 of varying widths, wherein the central circle 810 is pure white at the center of the projection. In one example, red stripes 815 mark the boundaries of image 800A. A central region 820 is formed by blocks 825, which in one example are two different shades of green. Similarly, stripes 815 may include different shades of red to embed tag codes and identify the outer edges of tags.

[0052] like Figure 8 As shown, two images 800A and 800B are projected onto an object 830 (such as a wall). When these images 800A and 800B overlap, the images are processed in the LAB color space in order to decode the encoded images 800A and 800B.

[0053] The LAB color space is a three-dimensional representation of projected colors with coordinates (L*, a*, b*), where L* defines the lightness or brightness axis, a* defines the red-green axis, and b* defines the yellow-blue axis. The value of L* ranges from 0 (black) to 100 (fully diffused white). Therefore, the detection device receives the colors within the image from the L, a, and b channels for processing.

[0054] The tristimulus values ​​X, Y, Z of a given color in the projection are compared to the tristimulus value X of the white reference. n Y n Z n Standardize to obtain X / X n Y / Y n Z / Z n The sensing compression response is determined by the applied transformation (X / X). n ) 1 / 3 (Y / Y) n ) 1 / 3 (Z / Z) n ) 1 / 3 Representation. Numerical scaling is applied to (Y / Y). n ) 1 / 3 To obtain the coordinates L*, and other numerical scaling is applied to the difference (X / X). n ) 1 / 3 –(Y / Y n )1 / 3 and (Y / Y) n ) 1 / 3 –(Z / Z n ) 1 / 3 To obtain coordinates a* and b*.

[0055] Therefore, by utilizing the LAB color space, it is possible to convert the chromaticity C* ab Defined as (a* 2 +b* 2 ) 1 / 2 and hue h ab Defined as angle tan -1 (b* / a*) (in degrees) are used to obtain some color appearance attributes from L*, a*, and b* to form a cylindrical coordinate system, where chromaticity represents the detected relative color. Therefore, saturation is the color of the region proportional to luminance. Thus, the color difference ΔE* between the first stimulus 1 and the second stimulus 2 is... ab Defined as [(L*1-L*2)] 2 +(a*1-a*2) 2 +(b*1-b*2) 2 ] 1 / 2 .

[0056] As an example, the brightness channel is first determined. Binarization of the brightness image is provided through double thresholding. The first step is to decode the two CCTags observed in the image to obtain the center position of the corresponding target. The next step is to use temporal information from past detections to associate the projection with the tracked object, while taking motion constraints into account.

[0057] The a-channel is processed to extract the boundaries of each projection 800A, 800B, which is why stripe 815 is typically red. If no boundary is found within a certain distance of the pure white central region 810 within concentric circles 805, the corresponding projection 800A or 800B is discarded. Similarly, the central region 820 is typically provided in green. In such embodiments, if no green pixels are found in the central region 820 between stripe 815 and central region 810, the projection is again discarded.

[0058] In this embodiment, the embedded code in the central green region 820 is obtained by processing the b-channel. The green pixels are clustered such that the decision boundary is the midpoint between the maximum and minimum values, so that the clustering represents the 1 and 0 values ​​that define the label signature. In this example, a predetermined hash table containing all signatures is used to identify the decoded signature as a match. In this way, the signature can represent a unique identifier for the drone.

[0059] Additionally, the drone may attempt to locate areas in the shadow of the projection to allow for greater lumens in those areas. Each drone can apply real-time keystone correction to make its projected surface appear undistorted or corrected. The drone can also examine the projected surface and apply color compensation to image 800A or 800B based on surface color, texture, gloss, etc.

[0060] Figure 9 This is a flowchart for proximity localization according to some embodiments. In this example, the flowchart illustrates method 900 for avoiding collisions between two drones. At 905, a first drone projects an encoded image onto at least one surface. The encoded image includes, but is not limited to, QR codes, spatial patterns, color patterns, barcodes, known shapes, combinations of any of these encoded images, etc. Specifically, in this example, the image includes... Figures 6-8 The example projection described herein. This image may include redundant data encoded throughout the image. The projected image has redundant patterns, colors, shapes, etc. The image can be projected using any type of image projection device, including but not limited to, 3D cameras (such as Kinect). TM RealSense TM Structural sensors, Asus Xtion TM wait).

[0061] At point 910, the second drone utilizes an image detection device to monitor the field of view. This detection device may include, but is not limited to, a 3D camera (such as Kinect). TM RealSense TM Structural sensors, Asus Xtion TM (etc.). The detection device can provide a field of view upwards, downwards, at a predetermined angle, or along any given direction. At 915, the field of view of the detection device at least partially overlaps with the projected coded image. Optionally, at 920, a second UAV aligns the field of view with the coded image.

[0062] At point 925, the second UAV decodes the encoded image of the projection to receive data related to the first UAV and / or another adjacent UAV. Specifically, if an adjacent UAV is in the area providing the encoded projection, and that encoded projection is not in the field of view of the second UAV but is in the field of view of the first UAV, then data related to that adjacent UAV can be encoded in the encoded image of the first UAV's projection, so that the second UAV can still receive data even though its field of view does not overlap with the encoded image of the adjacent UAV's projection.

[0063] At point 930, the second drone alters its flight path based on decoded data to avoid colliding with another drone. The flight path can be changed to avoid collisions with the first drone or adjacent drones. Any method can be used to avoid collisions. This includes, but is not limited to, using collision avoidance algorithms, utilizing virtual force technology, etc.

[0064] Figure 10 A block diagram of an example machine 1000 on which any one or more of the techniques (e.g., methods) discussed herein may be performed. In alternative embodiments, machine 1000 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, machine 1000 may operate as a server machine, a client machine, or both in a server-client network environment. In the example, machine 1000 may act as a peer-to-peer (P2P) (or other distributed) network environment. Further, although only a single machine is illustrated, the term "machine" should also be considered to include any collection of machines that individually or jointly execute a set (or more) of instructions to perform any or more of the methods discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations.

[0065] Examples as described herein may include logic or multiple components, engines, or mechanisms, or may be operated by logic or multiple components, engines, or mechanisms. A circuit system is a collection of circuits implemented in a tangible entity including hardware (e.g., simple circuits, gates, logic, etc.). The members of a circuit system can be flexible over time and due to the variability of the underlying hardware. A circuit system includes members that can perform a specified operation individually or in combination during operation. In the examples, the hardware of the circuit system may be immutably designed to perform a specific operation (e.g., hardwired). In the examples, the hardware of the circuit system may include physical components (e.g., execution units, transistors, simple circuits, etc.) that are variably connected, including computer-readable media that are physically modified (e.g., magnetic, electrical, movable arrangement, etc. of invariant aggregate particles) to encode instructions for a specific operation. When connecting physical components, the underlying electrical properties of the hardware components change, for example, from insulator to conductor, or vice versa. These instructions enable embedded hardware (e.g., execution units or loading mechanisms) to create members of a circuit system in the hardware via variable connections to perform multiple parts of a specific operation during operation. Accordingly, while the device is operating, the computer-readable medium is communicatively coupled to other components of the circuit system. In the example, any one of the physical components may be used in more than one member of more than one circuit system. For example, under operation, an execution unit may be used at one moment in a first circuit of a first circuit system and reused at a different time by a second circuit of the first circuit system, or by a third circuit of the second circuit system.

[0066] Machine (e.g., computer system) 1000 may include a hardware processor 1002 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), main memory 1004, and static memory 1006, some or all of which may communicate with each other via an interconnect (e.g., a bus) 1008. Machine 1000 may further include a display unit 1010, an alphanumeric input device 1012 (e.g., a keyboard), and a user interface (UI) navigation device 1014 (e.g., a mouse). In the example, display unit 1010, input device 1012, and UI navigation device 1014 may be a touchscreen display. Machine 1000 may additionally include a storage device (e.g., a drive unit) 1016, a signal generation device 1018 (e.g., a speaker), a network interface device 1020, and one or more sensors 1021 (such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors). Machine 1000 may include an output controller 1028 for connecting or controlling one or more peripheral devices (e.g., printers, card readers, etc.), such as serial (e.g., Universal Serial Bus (USB)), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC) etc.) connections.

[0067] Storage device 1016 may include a machine-readable medium 1022 on which one or more sets of data structures or instructions 1024 (e.g., software) are stored, which embody or are utilized by any one or more of the techniques or functions described herein. Instructions 1024 may also reside wholly or at least partially in main memory 1004, in static memory 1006, or in hardware processor 1002 during execution by machine 1000. In this example, one or any combination of hardware processor 1002, main memory 1004, static memory 1006, or storage device 1016 may constitute a machine-readable medium.

[0068] Although machine-readable medium 1022 is illustrated as a single medium, the term "machine-readable medium" can include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 1024.

[0069] The term "machine-readable medium" can include any medium capable of storing, encoding, or carrying instructions for execution by machine 1000 and causing machine 1000 to perform any one or more of the technologies disclosed herein, or capable of storing, encoding, or carrying data structures used by or associated with such instructions. Non-limiting examples of machine-readable media can include solid-state memory as well as optical and magnetic media. In examples, mass-capacity machine-readable media includes machine-readable media having a plurality of particles having an invariant (e.g., stationary) mass. Accordingly, mass-capacity machine-readable media are not transient propagating signals. Specific examples of mass-capacity machine-readable media can include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0070] Instruction 1024 can be further transmitted or received via network interface device 1020 using a transmission medium through communication network 1026, utilizing any of a variety of transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), conventional telephone (POTS) networks, and wireless data networks (e.g., wireless networks called wireless communication networks). The Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard family, known as The IEEE 802.16 family of standards, the IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, etc. In the example, network interface device 1020 may include one or more physical jacks (e.g., Ethernet jacks, coaxial jacks, or telephone jacks) or one or more antennas for connecting to communication network 1026. In the example, network interface device 1020 may include multiple antennas for wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be understood to include any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 1000, and "transmission medium" includes digital or analog communication signals or other intangible media used to facilitate communication of such software.

[0071] Additional notes and examples:

[0072] Example 1 is a device for drone collision avoidance, the device comprising: processing circuitry for the drone on its flight path, the processing circuitry being configured to: extract information from an encoded image captured by a detection device having a field of view overlapping the encoded image; determine, based on the extracted information, whether a collision with an external source will occur on the flight path; and modify the drone's flight path to avoid a collision.

[0073] In Example 2, the subject of Example 1 may optionally include: where the drone is a first drone and the external source is a second drone.

[0074] In Example 3, one or more of the topics from Examples 1-2 may be optionally included, wherein the encoded image is a QR code.

[0075] In Example 4, one or more of the topics from Examples 1-3 are optionally included, wherein the encoded image is CCTag.

[0076] In Example 5, one or more of the topics from Examples 1-4 are optionally included, wherein information is extracted from the encoded image based on the color of the encoded image.

[0077] In Example 6, one or more of the topics from Examples 1-5 are optionally included, wherein information is extracted from the encoded image based on patterns within the encoded image.

[0078] In Example 7, one or more of the subjects in Examples 1-6 are optionally included, wherein the processing circuitry is further configured to: encode an image having information related to the drone; and project the encoded image onto a surface using a camera.

[0079] In Example 8, the subject of Example 7 is optionally included, wherein the information related to the drone includes one of the following: drone speed, drone trajectory, and drone flight path.

[0080] In Example 9, one or more of the topics in Examples 7-8 are optionally included, wherein encoding an image with information includes encoding redundant information into the image.

[0081] In Example 10, one or more of the subjects from Examples 7-9 are optionally included, wherein the surface is a ceiling.

[0082] In Example 11, one or more of the subjects in Examples 7-10 are optionally included, wherein the processing circuitry is further configured to: determine the color of the surface; and modify the color of the projected encoded image based on the color of the surface.

[0083] In Example 12, the subject matter of any one or more of Examples 7-11 is optionally included, wherein the processing circuitry is further configured to: detect when the coded image of the projection overlaps with the coded image of the projection of the second UAV; and modify the coded image of the projection based on the coded image of the projection of the second UAV.

[0084] In Example 13, the subject matter of any one or more of Examples 1-12 is optionally included, wherein the processing circuitry is further configured to: determine the relative velocity of the external source based on the encoded image provided by the external source; determine the virtual force based on the relative velocity of the external source; and change the acceleration of the UAV based on the virtual force to change the flight trajectory.

[0085] In Example 14, the subject of Example 13 is optionally included, wherein, in order to determine whether a collision with an external source will occur, the processing circuitry is configured to: determine the position of the UAV based on the current flight trajectory at a given time point; determine the position of the external source based on an encoded image of the external source at that time point; and determine whether the position of the UAV at that time point is the same as the position of the external source.

[0086] In Example 15, one or more of the subjects in Examples 1-14 are optionally included, wherein the processing circuitry is further configured to: align the drone's camera with the encoded image before extracting information from the encoded image.

[0087] In Example 16, the subject matter of any one or more of Examples 1-15 may optionally be included, wherein extracting information from an encoded image includes decoding the encoded image.

[0088] In Example 17, the subject of Example 16 is optionally included, wherein decoding the encoded image includes processing the image in the LAB color space.

[0089] In Example 18, the subject of Example 17 optionally includes, wherein decoding the encoded image further includes: determining a tag signature based on processing the image in the LAB color space; and comparing the tag signature with a predetermined tag signature in a hash table to determine a unique identifier for the drone.

[0090] Example 19 is a method for a machine implementation of drone collision avoidance, the method comprising: capturing an encoded image projected by a second drone using a camera of a first drone; extracting a trajectory of the second drone from the encoded image using processing circuitry of the first drone; comparing the trajectory of the first drone with the trajectory of the second drone; and modifying the trajectory of the first drone based on the comparison of the trajectory of the first drone with the trajectory of the second drone.

[0091] In Example 20, the subject matter of Example 19 optionally includes the following: extracting the trajectory of a second drone from an encoded image using the processing circuitry of a first drone includes: detecting the encoded image in the field of view of the camera of the first drone; aligning the field of view with the encoded image; and decoding the encoded image.

[0092] In Example 21, the subject matter of any one or more of Examples 19-20 optionally includes, wherein extracting the trajectory of the second UAV from the encoded image using the processing circuitry of the first UAV includes: detecting a portion of the encoded image in the camera field of view of the first UAV; and decoding the encoded message based on the portion of the encoded image detected by the camera.

[0093] In Example 22, the subject matter of any one or more of Examples 19-21 is optionally included, wherein extracting the trajectory of the second UAV from the encoded image using the processing circuitry of the first UAV includes: detecting the encoded image in the field of view of the camera of the first UAV; and decoding the encoded message based on the color pattern of the encoded image.

[0094] In Example 23, the subject matter of any one or more of Examples 19-22 optionally includes: wherein changing the trajectory of the first drone based on a comparison of the trajectory of the first drone with the trajectory of the second drone includes: determining the virtual force of the first drone based on the trajectory of the second drone; and accelerating the first drone based on the virtual force.

[0095] Example 24 is at least one non-transitory computer-readable medium for drone collision avoidance, including instructions that, when executed by a machine, cause the machine to perform the following operations: detect an encoded image associated with a second drone in the field of view of a detection device of a first drone; decode the image to receive information related to the trajectory of the second drone; and modify the trajectory of the first drone based on the information related to the trajectory of the second drone.

[0096] In Example 25, the subject of Example 24 is optionally included, wherein the encoded image is one of a QR code, a CCTag, or a colored pattern.

[0097] Example 26 is at least one machine-readable medium including instructions that, when executed by a machine, cause the machine to perform any of the operations described in Examples 19-23.

[0098] Example 27 is an apparatus that includes means for performing any of the operations described in Examples 19-23.

[0099] Example 28 is at least one machine-readable medium comprising instructions that, when executed by processing circuitry, cause the processing circuitry to perform operations to implement any of the examples 1-25.

[0100] Example 29 is a device that includes means for implementing any one of Examples 1-25.

[0101] Example 30 is a system for implementing any one of Examples 1-25.

[0102] Example 31 is a method for implementing any of Examples 1-25.

[0103] The above detailed description includes references to the accompanying drawings, which form part of the detailed description. The drawings illustrate specific embodiments that can be implemented by way of illustration. These embodiments are also referred to herein as "examples". Such examples may include elements other than those shown or described. However, examples including the elements shown or described are also contemplated. Moreover, examples of any combination or arrangement of those elements (or one or more aspects thereof) shown or described are also contemplated, or with reference to a particular example (or one or more aspects thereof) shown or described herein, or with reference to other examples (or one or more aspects thereof) shown or described herein.

[0104] Publications, patents, and patent documents cited in this document are incorporated herein as if they were incorporated individually by reference. In the event of inconsistencies in usage between this document and those documents incorporated by reference, the usage in the incorporated reference(s)(s) is supplementary to the usage in this document; in the case of irreconcilable inconsistencies, the usage in this document prevails.

[0105] In this document, as is common in patent documents, the term “a (or an)” is used to include one or more, independent of any other instance or usage of “at least one” or “one or more.” In this document, unless otherwise indicated, the term “or” is used to refer to a non-exclusive “or,” such that “A or B” includes “A but not B,” “B but not A,” and “A and B.” In the appended claims, the terms “including” and “in which” are used as their common English equivalents to the corresponding terms “comprising” and “wherein.” Furthermore, in the appended claims, the terms “comprising” and “including” are open-ended, meaning that a system, apparatus, article of manufacture, or process that includes elements other than those listed following such terms in a claim is still considered to fall within the scope of that claim. Additionally, in the appended claims, the terms “first,” “second,” and “third,” etc., are used merely as designations and are not intended to indicate a numerical order of their objects.

[0106] The above embodiments can be implemented in various hardware configurations, which may include a processor for executing instructions implementing the above techniques. Such instructions may be contained in a machine-readable medium (such as a suitable storage medium or memory) or other processor-executable medium.

[0107] The above description is intended to be illustrative and not restrictive. For example, the examples described above (or one or more aspects thereof) may be used in conjunction with other examples. Other embodiments may be used by those skilled in the art after careful reading of the above description. The abstract allows the reader to quickly determine the nature of this technical disclosure. This abstract is submitted with the understanding that it is not intended to interpret or limit the scope or meaning of the claims. Moreover, in the above detailed description, various features may be grouped together to simplify the disclosure. However, the claims may not state every feature disclosed herein, as embodiments may characterize a subset of said features. Furthermore, embodiments may include fewer features than those disclosed in particular examples. Therefore, the appended claims are thus incorporated into the detailed description, wherein one claim stands alone as a separate embodiment. The scope of the embodiments disclosed herein should be determined by reference to the full scope of the appended claims together with the equivalents of such claims.

Claims

1. A device for drone collision avoidance, the device comprising: The processing circuitry of the first unmanned aerial vehicle (UAV) on its flight path, the processing circuitry being used for: Information is extracted from a first encoded image, which is projected onto a surface by a second UAV and captured by a detection device having a field of view that overlaps with the first encoded image; Based on the extracted information, determine whether a collision with the second UAV will occur on the flight path; and The flight path of the first drone was altered to avoid the collision.

2. The apparatus of claim 1, wherein, The first encoded image captured by the detection device is a fast response code.

3. The apparatus of claim 1, wherein, The first encoded image captured by the detection device is a concentric circle marker CCTag.

4. The apparatus of claim 1, wherein, Information is extracted from the encoded image captured by the detection device based on the color of the first encoded image.

5. The apparatus of claim 1, wherein, Information is extracted from the first encoded image captured by the detection device based on patterns within the first encoded image captured by the detection device.

6. The apparatus of claim 1, wherein, The processing circuit is further configured to: Encode the second image containing information related to the first UAV to produce a second encoded image; and The second encoded image, containing information related to the first drone, is projected onto the surface using a camera.

7. The apparatus of claim 6, wherein, The information associated with the first drone includes one of the following: drone speed, drone trajectory, or drone flight path.

8. The apparatus of claim 6, wherein, Encoding the second encoded image, which contains information related to the first UAV, includes encoding redundant information into the second encoded image.

9. The device as described in claim 6, characterized in that, The surface in question is the ceiling.

10. The apparatus of claim 6, wherein, The processing circuit is further configured to: Determine the color of the surface; and Based on the color of the surface, the color of the second encoded image, which contains information related to the first drone, is modified.

11. The device as claimed in claim 6, characterized in that, The processing circuit is further configured to: Detect when the second encoded image, which contains information related to the first drone, overlaps with the first encoded image captured by the detection device; and Based on the first encoded image captured by the detection device, the second encoded image containing information related to the first UAV is modified.

12. The apparatus of claim 1, wherein, The second drone projects the first encoded image captured by the detection device, and the processing circuit is configured to: The relative speed of the second drone is determined based on the first encoded image captured by the detection device; The virtual force is determined based on the relative velocity of the second UAV; and The acceleration of the first drone is changed based on the virtual force to alter its flight trajectory.

13. The apparatus of claim 12, wherein, To determine whether a collision with the second drone will occur, the processing circuitry is configured to: The position of the first UAV is determined based on the flight trajectory at a given point in time. The location of the second drone is determined based on the first encoded image captured by the detection device at the specified time point; and Determine whether the position of the first drone at that time point is the same as the position of the second drone.

14. The apparatus of claim 1, wherein, The processing circuit is further configured to: Before extracting information from the first encoded image captured by the detection device, the camera of the first drone is aligned with the first encoded image captured by the detection device.

15. The apparatus of claim 1, wherein, Extracting information from the first encoded image captured by the detection device includes: decoding the first encoded image captured by the detection device.

16. The apparatus of claim 15, wherein, Decoding the first encoded image captured by the detection device includes processing the first encoded image captured by the detection device in the luminance-red / green-blue / yellow value LAB color space.

17. The apparatus of claim 16, wherein, Decoding the first encoded image captured by the detection device further includes: The tag signature is determined based on processing the first encoded image captured by the detection device in the LAB color space; and The tag signature is compared with a predetermined tag signature in a hash table to determine the unique identifier of the first drone.

18. A machine implementation method for collision avoidance of unmanned aerial vehicles, the method comprising: The camera of the first drone captures a first encoded image projected onto the surface by the second drone; The processing circuitry of the first UAV is used to extract the trajectory of the second UAV from the first encoded image; Compare the trajectory of the first UAV with the trajectory of the second UAV; as well as The trajectory of the first drone is modified based on a comparison of the trajectory of the first drone with the trajectory of the second drone to avoid a collision.

19. The method of claim 18, wherein, Extracting the trajectory of the second drone from the first encoded image using the processing circuitry of the first drone includes: Detect the first encoded image in the field of view of the camera of the first UAV; Align the field of view with the first encoded image; and The first encoded image is decoded.

20. The method of claim 19, wherein, Extracting the trajectory of the second drone from the first encoded image using the processing circuitry of the first drone includes: Detecting a portion of the first encoded image within the field of view of the camera of the first UAV; and The first encoded image is decoded based on the portion of the first encoded image detected by the camera.

21. The method as described in claim 18, characterized in that, Extracting the trajectory of the second drone from the first encoded image using the processing circuitry of the first drone includes: Detect the first encoded image within the field of view of the camera of the first UAV; and The first encoded message is decoded based on the color pattern of the encoded image.

22. The method of claim 18, wherein, Changing the trajectory of the first drone based on a comparison of the trajectory of the first drone and the trajectory of the second drone includes: The virtual force of the first drone is determined based on the trajectory of the second drone; and The first drone is accelerated based on the virtual force.

23. A machine-readable medium comprising instructions that, when executed by a machine, cause the machine to perform any one of the operations described in claims 18-22.

24. An apparatus comprising: A means for capturing a first encoded image projected onto a surface by a second drone using a camera of a first drone; A means for extracting the trajectory of the second drone from the first encoded image using the processing circuitry of the first drone; A device for comparing the trajectory of the first UAV with the trajectory of the second UAV; as well as A means for changing the trajectory of the first drone based on a comparison of the trajectory of the first drone with the trajectory of the second drone to avoid a collision.

Images