Machine-to-machine targeting to maintain positive identification

Abstract

The present application relates to machine-to-machine targeting that maintains positive identification, which involves: capturing a first video of a scene about potential targeting coordinates by a first video sensor (102) on a first aircraft (100); sending the first video (232) and associated potential targeting coordinates by the first aircraft; receiving the first video on a first display that communicates with a processor, the processor also receiving the potential targeting coordinates; in response to observing the first video on the first display, selecting the potential targeting coordinates as actual targeting coordinates (226) for a second aircraft (116); and directing the second aircraft toward the actual targeting coordinates; wherein positive identification of a target (114) corresponding to the actual targeting coordinates is maintained starting from the selection of the actual targeting coordinates.

Description

[0001] This application is a divisional application of an application filed on March 24, 2016, with application number 201680018100.4 and invention name “Machine-to-machine targeting with maintained positive identification”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 138,305, filed on March 25, 2015, the contents of which are hereby incorporated by reference for all purposes. Technical Field

[0004] The present invention relates to unmanned aerial systems (UAS), and more particularly to UAS identification of targets. background

[0005] Unmanned aerial systems (UAS) include unmanned aerial vehicles (UAVs) that may be remotely controlled by one or more operators at one or more ground control systems (GCSs). Restrictive rules of engagement may require that positive identification (PID) of the target be maintained from identification to final contact.

[0006] Overview

[0007] Some embodiments of the present invention may include a targeting method, which may include: capturing a first video of a scene about potential targeting coordinates via a first video sensor on a first aircraft; transmitting the first video and associated potential targeting coordinates via the first aircraft; receiving the first video on a first display in communication with a processor, the processor also receiving the potential targeting coordinates; in response to observing the first video on the first display, selecting the potential targeting coordinates as actual targeting coordinates for a second aircraft; and directing the second aircraft toward the actual targeting coordinates; wherein positive identification of the target corresponding to the actual targeting coordinates is maintained from the selection of the actual targeting coordinates. Additional exemplary method embodiments may include launching a second aircraft having a second video sensor.

[0008] Other exemplary method embodiments may include causing the first aircraft to orbit around the actual sighting coordinates to maintain the field of view (FOV) of the first video sensor around the actual sighting coordinates. Other exemplary method embodiments may include capturing a second video of the scene by a second video sensor, and receiving the second video on a first display when the second aircraft approaches the actual sighting coordinates. In other exemplary method embodiments, maintaining positive identification of the target may also include comparing the first video of the received scene on the first display and the second video of the received scene by a user of the processor to confirm that the second aircraft is approaching the actual sighting coordinates. Other exemplary method embodiments may include selecting a terminal homing mode of the target aimed at by the second aircraft in response to observing the first video and the second video on the first display. In other exemplary method embodiments, the terminal homing mode may also include at least one of the following: impacting the target by the second aircraft, impacting the target by a load deployed by the second aircraft, and impacting the target by a marker deployed by the second aircraft. Other exemplary method embodiments may include confirming the condition of the target after the impact in response to observing the first video on the first display.

[0009] Additional exemplary method embodiments may include: capturing a first video of a new scene about new potential targeting coordinates via a first video sensor on a first aircraft; transmitting the first video and associated new potential targeting coordinates via the first aircraft; receiving the first video on a first display in communication with a processor, the processor also receiving the new potential targeting coordinates; selecting the new potential targeting coordinates as new actual targeting coordinates for a third aircraft in response to observing the first video on the first display; and directing the third aircraft toward the new actual targeting coordinates; wherein positive identification of the new target corresponding to the new actual targeting coordinates is maintained from the selection of the new actual targeting coordinates. In additional exemplary method embodiments, the selected potential targeting coordinates may be geographic coordinates corresponding to a center field of view (CFOV) of the captured first video and / or geographic coordinates corresponding to pixels in a field of view (FOV) of the captured first video.

[0010] Another exemplary method embodiment may include: receiving, by a ground control system (GCS), a first video stream from a first aircraft and potential targeting coordinates associated with the first video stream; selecting, by the GCS, the potential targeting coordinates corresponding to a target as actual targeting coordinates; receiving, by the GCS, a second video stream from a second aircraft; and guiding, by the GCS, the second aircraft toward the actual targeting coordinates; wherein positive identification of the target is maintained by the GCS starting from the selection of the actual targeting coordinates.

[0011] Another exemplary method embodiment may include selecting, by the GCS, a terminal homing mode in which the second aircraft is aimed at a target. In another exemplary method embodiment, the terminal homing mode may include at least one of the following: the second aircraft impacting the target, the payload deployed by the second aircraft impacting the target, and the marker deployed by the second aircraft impacting the target. Another exemplary method embodiment may include: confirming the condition of the target after the impact via the GCS via the first video stream received from the first aircraft. Another exemplary method embodiment may include: selecting the new potential targeting coordinates corresponding to the new target as the new actual targeting coordinates by the GCS; receiving the third video stream from the third aircraft by the GCS; and guiding the third aircraft to the new actual targeting coordinates by the GCS, wherein the positive identification of the new target corresponding to the new actual targeting coordinates is maintained from the selection of the new actual targeting coordinates. In another exemplary method embodiment, the selected potential targeting coordinates are geographic coordinates corresponding to the center field of view (CFOV) of the received first video stream and / or geographic coordinates corresponding to pixels in the field of view (FOV) of the received first video stream.

[0012] An exemplary system embodiment may include: a first aircraft, the first aircraft comprising a first processor and configured to send a first video stream and potential targeting coordinates associated with the first video stream; a second aircraft, the second aircraft comprising a second processor and configured to send a second video stream; a first ground control system (GCS), the first GCS comprising a third processor, and wherein the first GCS is configured to guide the first aircraft; and a second ground control system (GCS), the second GCS comprising a fourth processor, and wherein the second GCS can: receive the first video stream, the potential targeting coordinates associated with the first video stream, and the second video stream; select the potential targeting coordinates corresponding to a target as actual targeting coordinates; and guide the second aircraft toward the actual targeting coordinates; wherein positive identification of the target is maintained by the second GCS starting from the selection of the actual targeting coordinates.

[0013] In another exemplary system embodiment, the first GCS may receive the first video stream. In another exemplary system embodiment, the first GCS may include a first digital data link (DDL) to guide the first aircraft and receive the first video stream. In another exemplary system embodiment, the second GCS may include a second DDL to receive the first video stream, potential targeting coordinates associated with the first video stream, and the second video stream, and guide the second aircraft to the actual targeting coordinates. In another exemplary system embodiment, the selected potential targeting coordinates may be geographic coordinates corresponding to at least one of the following: the center field of view (CFOV) of the first video stream, and the pixels in the field of view (FOV) of the first video stream. In another exemplary system embodiment, the second GCS also includes a graphical user interface (GUI) that communicates with the fourth processor, and wherein the GUI is configured to display the first video stream and the second video stream. In another exemplary system embodiment, the GUI of the second GCS may display a map having the location of the first aircraft, the location of the second aircraft, the CFOV of the first video stream of the first aircraft, and the CFOV of the second video stream of the second aircraft. In another exemplary system embodiment, the second GCS may select a terminal homing mode of a target aimed at by the second aircraft, wherein the terminal homing mode may include at least one of: impacting the target by the second aircraft, impacting the target with a payload deployed by the second aircraft, and impacting the target with a marker deployed by the second aircraft; and confirming the condition of the target after the impact via a first video stream received from the first aircraft.

[0014] Another exemplary method embodiment may include: receiving, by a ground control system (GCS), a first video stream from a first aircraft and potential targeting coordinates associated with the first video stream; selecting, by the GCS, the potential targeting coordinates corresponding to a target as actual targeting coordinates; receiving, by the GCS, a second video stream from a second aircraft among a plurality of aircraft, wherein the second aircraft among the plurality of aircraft is closest to the target; guiding, by the GCS, the second aircraft among the plurality of aircraft toward the actual targeting coordinates; wherein positive identification of the target is maintained by the GCS starting from the selection of the actual targeting coordinates.

[0015] Other exemplary method embodiments may include selecting a terminal homing mode in which a second aircraft is aligned with a target through a GCS. In other exemplary method embodiments, the terminal homing mode may include at least one of the following: a second aircraft impacting a target, a load deployed by the second aircraft impacting a target, and a marker deployed by the second aircraft impacting a target. Other exemplary method embodiments may include: confirming the condition of the target after the impact via a first video stream received from the first aircraft via the GCS. Other exemplary method embodiments may include: guiding a third aircraft among multiple aircraft to the actual aiming coordinates through the GCS. In other exemplary method embodiments, multiple aircraft wander around a preset area until the actual aiming coordinates are selected. In other exemplary method embodiments, each aircraft among the multiple aircraft follows the second aircraft guided toward the actual aiming coordinates, and is spaced apart by a set time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The components in the figures are not necessarily drawn to scale, but rather emphasize the principles of the invention. In all the different views, the same reference numerals indicate corresponding parts. Embodiments are shown by way of example and not limitation in the figures of the accompanying drawings, wherein:

[0017] Figure 1A An embodiment of an unmanned aerial system (UAS) having a first aircraft transmitting information to one or more ground control systems (GCS) is depicted;

[0018] Figure 1B An embodiment of a UAS having a second aircraft that launches after selecting a target identified by a first aircraft is depicted;

[0019] Figure 1C An embodiment of a UAS having a second aircraft observing a target identified by the first aircraft while the first aircraft maintains positive identification of the target is depicted;

[0020] Figure 1D An embodiment of a UAS having a second aircraft that enters a "commit" mode and moves toward a target while a first aircraft maintains positive identification of the target is depicted;

[0021] Figure 1E An embodiment is depicted in which a target is destroyed by a second aircraft and a first aircraft sends visual confirmation of the target hit to a UAS of one or more GCSs;

[0022] Figure 2 An exemplary GCS having an embodiment of a user interface for viewing transmitted information and / or video and selecting targeting coordinates based on received information and / or video is depicted;

[0023] Figure 3 An exemplary connection between a first GCS controlling a first aircraft and a second GCS controlling a second aircraft is depicted;

[0024] Figures 4A-4C An exemplary functional block diagram of an embodiment of a method for identifying, eliminating, and confirming a hit on a target using two or more aircraft while maintaining positive identification of the target is depicted;

[0025] Figure 5 An embodiment of a UAS having multiple aircraft that loiter until a target is identified and actual targeting coordinates associated with the target are selected is depicted;

[0026] Figure 6 An embodiment of a UAS having multiple aircraft directed to a single target is depicted, wherein each of the aircraft directed to the single target are automatically spaced apart to allow a single operator to switch to control a subsequent aircraft upon impact; and

[0027] Figure 7 An exemplary top-level functional block diagram of a computing device embodiment of a GCS and / or one or more UASs is shown. Detailed Description

[0028] The present invention allows a first aircraft operated by a first operator at a first ground control system (GCS) to capture information about a scene that may include a video stream and geographic coordinates associated with the captured video stream. The video stream from the first aircraft can be sent to one or more GCSs as encrypted data. A second operator at a second GCS can observe the sent video stream on a display (e.g., a graphical user interface (GUI)) and select the aiming coordinates corresponding to the position of the target on the video stream on the GUI. The position of the target can be any pixel in the center field of view (CFOV) of the video or the field of view (FOV) of the video. The position of the target can be the geographic coordinates of any pixel in the aircraft video stream. The geographic location of any given pixel can be interpolated from the CFOV coordinates and the coordinates of the four corners of the video. A second aircraft can be launched and directed to the aiming coordinates selected by the second operator. The first aircraft can maintain a positive identification (PID) for the target while the second aircraft is directed to the selected aiming coordinates. The second aircraft can be used to impact the target at the selected aiming coordinates. The video stream from the first aircraft can be used to confirm that the target has been hit and confirm the extent of any damage to the target. The first aircraft may continue to loiter looking for other targets.

[0029] Figure 1AAn embodiment of an unmanned aerial system (UAS) having a first aircraft 100 that captures information about a scene is depicted. The information may be a video stream and targeting coordinates associated with the video stream. The first aircraft 100 (e.g., a Puma, Raven, Wasp, or Shrike manufactured by AeroVironment, Inc. of Monrovia, California) may have a first camera 102 having a first video sensor that may be mounted on a gimbal to allow a first operator to direct the camera to various locations on the ground 104. The first aircraft 100 and / or the first camera 102 may be controlled by a first operator 101 at a first GCS 106 via one or more signals sent 105 between the first GCS 106 and the first aircraft 100. The first camera 102 may have a field of view (FOV) 108 with a center field of view (CFOV) 110 in the center of the captured video stream. The position of the first aircraft 100, the position of the first camera 102, and / or additional details may be used to continuously calculate the geographic location of the CFOV 110 and / or the geographic location of any pixel in the FOV 108 of the first camera 102 as potential targeting coordinates. The potential targeting coordinates may be calculated on the aircraft 100 and / or sent to one or more GCSs (106, 112) as video stream metadata. In some embodiments, data from the aircraft 100 may be sent to one or more GCSs (106, 112), and the one or more GCSs (106, 112) may calculate the potential targeting coordinates. The video stream and the potential targeting coordinates may be sent (105, 111) to one or more GCSs, such as the first GCS 106 and the second GCS 112. A second operator 113 at the second GCS 112 may monitor the video stream 111 sent by the first aircraft 100 until the target 114 is identified. Once the target 114 is identified, the second operator 113 may select potential targeting coordinates associated with the target 114 via an input at the second GCS 112 (e.g., a button on a GUI having a monitor displaying a video stream from the first aircraft 100). The potential targeting coordinates may be any pixel in the CFOV 110 or the FOV of the video stream. Once the potential targeting coordinates are selected, the potential targeting coordinates become actual targeting coordinates associated with the target 114. The target 114 may be stationary or moving. If the target is moving, new potential targeting coordinates may be selected to become new actual targeting coordinates corresponding to the new position of the moving target 114, the new CFOV 110 of the first camera 102, and / or the new position of the target 114 in the FOV 108 of the camera 102. In some embodiments, the targeting coordinates may move adaptively with the moving target 114 and / or the moving CFOV 110 of the first camera 102.

[0030] Figure 1B Depicted Figure 1A 1, wherein in response to selection of a targeting coordinate by a second operator 113 at a second GCS 112, a launcher 120 launches 118 a second aircraft 116 (e.g., manufactured by AeroVironment, Inc. of Monrovia, California) ). In some embodiments, the second aircraft 116 may be manually launched by the second operator 113 or other person. Once the actual targeting coordinates are available, the first aircraft 100 may maintain the PID of the target 114 (e.g., by keeping the target 114 within the FOV 108 of the first camera 102) while the second aircraft 116 is guided to the actual targeting coordinates associated with the target 114. The first aircraft 100 may be guided on autopilot and / or set to orbit, hover, and / or remain in a set area so as to maintain the FOV 108 of the first camera 102 aimed at the target 114. In some embodiments, the second aircraft 116 may already be in flight when the targeting coordinates are selected. In other embodiments, the second aircraft 116 may be launched from the first aircraft 100.

[0031] Figure 1C Depicted Figure 1A-1B 108 and 114. An embodiment of a UAS in which a second aircraft 116 is flying toward a target 114 while the first aircraft 100 maintains the PID of the target 114. A second operator 113 can monitor a video stream 111 transmitted by the first aircraft 100 via a second GCS 112 while the second aircraft 116 is directed toward the target 114. The second operator 113 can also monitor a video stream transmitted by the second aircraft 116 (e.g., by a second camera) via one or more signals 122 transmitted between the second aircraft 116 and the second GCS 112. The second operator 113 can simultaneously observe the video streams transmitted by both the first aircraft 100 and the second aircraft 116 at the second GCS 112. When the target 114 is in the FOV (108, 124) of both cameras, a PID transfer occurs, and the second operator 113 can focus on the FOV 124 of the second camera to maintain the PID throughout the completion of the mission. Once target 114 is within FOV 124 of the second camera of second aircraft 116 , second operator 113 may place second aircraft 116 into a “commissioned” mode.

[0032] Figure 1D Depicted Figures 1A-1C12 is an embodiment of a UAS, wherein, via one or more signals sent 122 between the second aircraft 116 and the second GCS 112, the second aircraft 116 is placed in a "commissioned" mode, wherein the second aircraft 116 enters terminal homing toward the actual sighting coordinates of the target 114. In the "commissioned" mode, the second operator 113 guides the second aircraft 116 toward the target 114. The PID of the target 114 is maintained by the first aircraft 100. In some embodiments, the second aircraft 116 may not contact the target 114, but may deploy a payload and / or marker (e.g., paint) that may contact the target 114.

[0033] Figure 1E Depicted Figures 1A-1D In an embodiment of a UAS, the target 115 has been destroyed (e.g., by contact with a second aircraft carrying a payload of explosives). The first aircraft 100 selects the targeting coordinates (see Figure 1A ) maintain the PID of the target 115 from the first identification to the destruction and / or PID transfer of the target 115. The first aircraft 100 may observe the scene in the FOV 108 of the first camera 102 and send (105, 111) the video stream to one or more GCSs, such as the first GCS 106 and the second GCS 112. The destruction of the target 115 may be verified by the first operator 101 and / or the second operator 113 via their respective GCSs (106, 112). The first aircraft 100 may then continue to send (105, 111) the video stream and potential targeting coordinates to one or more GCSs (106, 112) until a new target 124 is identified. The new target 124 may be within the FOV 108 of the first camera 102 of the first aircraft. The new target can also be located outside the FOV 108 of the first camera 102 of the first aircraft 100, and the first operator 101 can control the first aircraft 100 and / or the first camera 102 to locate the new target. Once the new target 124 is identified, the third aircraft can be launched via the second GCS 112 and / or the third aircraft can be directed to the new effective aiming coordinates. This process can continue until there is no extra aircraft and / or all targets have been eliminated. In some embodiments, extra GCS and / or aircraft can be used for identifying and / or eliminating targets. In other embodiments, a single GCS can be used for controlling multiple aircraft. The first aircraft can be used for identifying potential aiming coordinates, and selects potential aiming coordinates as actual aiming coordinates. Then, the first aircraft can maintain the FOV of the actual aiming coordinates, (for example, via automatic flight circling or otherwise hovering above the target), and the operator makes the second aircraft fly to the actual aiming coordinates simultaneously, so that the PID is maintained to the final collision with the target from the initial selection of the potential aiming coordinates.

[0034] Figure 2 A GCS embodiment 200 is depicted having a processor and an addressable memory (see Figure 7 ) on a laptop or tablet computer for viewing information transmitted by one or more aircraft (e.g., first aircraft 206 and second aircraft 208) and selecting a targeting coordinate based on the received transmitted information. First aircraft 206 may be provided by a first GCS (see Figure 1A , 106), and the first aircraft may send 210 information about the scene, such as a video stream and geographic coordinates associated with the captured video stream. The information sent 210 from the first aircraft 206 may be received by a first digital data link (DDL) 212 (e.g., a pocket DDL). The first DDL 212 may be connected 214 to the computing device 204, for example, via a USB and / or Ethernet cable.

[0035] The computing device 204 may have a display (such as the GUI 202) that may display a map 216 showing a position 218 of the first aircraft 206, a position 220 of the second aircraft, a CFOV 222 of the first aircraft 206 (displayed as a "+"), a CFOV 224 of the second aircraft (displayed as an "x"), an actual targeting coordinate 226, an entry point 228 of the second aircraft 208, and a direction 230 of the second aircraft from the entry point 228 to the actual targeting coordinate 226. In some embodiments, the map 216 may also display the FOV of each aircraft. In the presence of a wind that is significant for the second aircraft to operate in a terminal state (e.g., greater than 5 knots), the second aircraft may select an entry point 228 that may be an operator-selectable minimum distance from the actual targeting coordinate 226 and downwind (e.g., 1000 meters downwind from the target). In some embodiments, the entry point may be automatically selected by the computing device 204 without requiring additional operator input. Approaching actual targeting coordinates 226 from entry point 228 may increase the accuracy and / or maneuverability of second aircraft 208 relative to actual targeting coordinates 226 .

[0036] GUI 202 may also display a first video stream 232 from video 210 transmitted by first aircraft 206. An operator may observe first video stream 232 until a desired target appears in first video stream 232. The operator may then select, for example, via one or more options on toolbar 234 (e.g., FalconView Toolbar (FVTB) from AeroVironment, Inc. of Monrovia, California), via a user interface, and / or via a touch interface to use any pixel in the CFOV and / or FOV of first video stream 232 as a targeting coordinate. In some embodiments, the operator may select any portion of the video stream as a targeting coordinate. When the operator selects a potential targeting coordinate, it becomes an actual targeting coordinate 226. Information transmitted 210 by first aircraft 206 and received by first DDL 212 includes geographic information associated with any pixel in the CFOV and / or FOV of first video stream 232 that the operator is observing. Once actual targeting coordinate 226 is valid, the operator may launch second aircraft 208 (see FIG. 2 ). Figure 1B , 118) and / or control the second aircraft 208 that is already in flight. One or more signals sent 234 between the second DDL 236 and the second aircraft 208 may be used to control the second aircraft 208 and receive a video stream from the second aircraft 208. The second DDL 236 may be mounted on a tower 238 to ensure a line of sight with the second aircraft 208. The second DDL 236 may be connected to a hub 240, which may be connected to the computing device 204 and / or a second aircraft controller 242.

[0037] The GUI can display a second video stream 244 from the video sent 234 by the second aircraft 208. Once the actual targeting coordinates 226 are valid and the second aircraft 208 is in flight, the operator can view the video streams (232, 244) of both the first aircraft 206 and the second aircraft 208. By using two video streams (232, 244), the operator directing the second aircraft 208 toward the actual targeting coordinates 226 can use the second aircraft controller 242 to maintain the PID of the target from the initial selection of the potential targeting coordinates to the final impact with the target. In some embodiments, the video streams (232, 244) can be presented close to each other on a single screen and device and / or multiple screens and / or devices.

[0038] Figure 3An exemplary GCS embodiment 300 is depicted between a first GCS 302 controlling a first aircraft 304 and a second GCS 306 controlling a second aircraft 308. In some embodiments, multiple GCSs may be geographically spaced such that a direct connection between them (e.g., via Ethernet and / or USB cables) is not feasible (see Figure 2 ). Utilizing multiple GCSs can help reduce the time from identifying a target to ultimately impacting the target. In other embodiments, multiple GCSs (302, 306) can be geographically spaced close enough that they can be directly connected, and the (310, 312) information sent by the first aircraft 304 and / or the second aircraft 308 can be shared without the need for additional DDLs (such as packet DDLs (see Figure 2 , 212)). In some embodiments, a single DDL may be used to receive and / or send data between two or more aircraft, such as a video stream, potential targeting coordinates associated with the video stream, and / or one or more aircraft controls.

[0039] Information (e.g., a video stream and geographic coordinates associated with the captured video stream) may be sent 310 by the first aircraft to a first DDL 314 of the first GCS 302. The first DDL 314 may send the received information to a first computing device 316, such as a laptop or tablet computer. The first computing device 316 may be connected to a first adapter box 318 via a cable (e.g., a USB cable). The first adapter box 318 may be connected to a second adapter box 320 via an Ethernet cable 322 (e.g., an Ethernet cable up to 100 meters long). The second adapter box 320 may be connected to a second computing device 324 via a cable (e.g., a USB cable). The adapter boxes (318, 320) may be used to convert USB to Ethernet and vice versa. An operator at the second computing device 324 may view the video stream captured by the first aircraft 304 and select potential targeting coordinates based on the video stream (see Figure 2 The operator at second computing device 324 may also receive a video stream from second aircraft 308 , which is sent 312 to second DDL 326 of second GCS 306 and then to second computing device 324 .

[0040] Figures 4A-4C An exemplary functional block diagram of an embodiment of a method for using two aircraft to identify, eliminate and confirm any damage to a target while maintaining positive identification of the target from initial identification to final confirmation (400, 401, 403). Reference characters A and B are used to show Figure 4A-4C4. The first operator of the first aircraft captures a video of a potential target via a first camera disposed on the first aircraft via a first ground control system (GCS) (step 402). The operator maintains the first camera of the first aircraft at the target via the first GCS to maintain the PID (step 404). The first aircraft sends a video stream from the first camera and potential aiming coordinates associated with the first video stream (step 406). The potential aiming coordinates can be embedded in the video stream as metadata and can be continuously updated as the CFOV changes in the scene. The second operator receives the first video stream from the first aircraft and the potential aiming coordinates associated with the first video stream via the second GCS (step 408). The second operator selects the potential aiming coordinates as the actual aiming coordinates via the second GCS (step 410). In some embodiments, if the second operator identifies a new target (e.g., a target with higher importance than the originally selected target), the second operator can select the new actual aiming coordinates. A second aircraft with a second camera is launched (step 412). The second operator can launch the second aircraft, or another individual can perform a pre-launch checklist and launch the second aircraft. In some embodiments, the second aircraft may already be in flight, and the second operator can switch to and / or continue to control the second aircraft. The second aircraft can be launched via the first GCS and / or the second GCS. The second aircraft sends the second video stream from the second camera (step 414). The second operator receives the second video stream and the first video stream via the second GCS (step 416). The second operator guides the second aircraft to the actual aiming coordinates via the second GCS (step 418). The second operator confirms via the second GCS that the target identified in the first video stream is identical to the target located at the actual aiming coordinates (step 419). The second operator selects the terminal homing mode (step 420) of the second aircraft to aim at the target via the second GCS. In some embodiments, the target can be mobile and can have a position changed from its initial aiming coordinates. If the target has moved, the second operator can adjust the path of the second aircraft via the second GCS to enter the terminal homing mode to the new target position. In some embodiments, as long as the PID has been maintained by the first video stream, the second operator can select new aiming coordinates from the first video stream via the second GCS based on the new target position. The second aircraft impacts the target (step 422). In some embodiments, the second aircraft may deploy a payload aimed at the target, such as an explosive or a marker (e.g., paint). The first operator and / or the second operator confirms the condition of the target after impact via the first video stream from the first aircraft via the first GCS and / or the second GCS (step 424). In some embodiments, the second operator may also confirm the condition of the target after impact via the first video stream from the first aircraft via the second GCS.Then, the second operator can decide whether further action is needed via the second GCS, for example, selecting new aiming coordinates and / or launching a third aircraft. In some embodiments, the process can be repeated until the desired effect (e.g., eliminating the target) has been achieved (step 426).

[0041] Figure 5 An embodiment of a UAS is depicted having a first operator 500 controlling a first aircraft 502 via a first GCS 501 and a second operator 504 who may control one of a plurality of aircraft (506, 508, 510, 512, 514) in flight via a second GCS 505. The plurality of aircraft (506, 508, 510, 512, 514) may hover around a preset area and / or pattern until desired by the second operator 504. The first operator 502 may identify a target 516 within a FOV 518 of a camera 520 on the first aircraft 502 via the first GCS 501. The second operator 504 may observe the target 516 via a video stream sent by the first aircraft 502 to the second GCS 505 via the second GCS 505. The second operator 504 may select a potential targeting coordinate associated with the target 516 as a valid targeting coordinate (see FIG. 5 ). Figure 2 ). The second operator 504 can select a second aircraft 506 of the plurality of aircraft (506, 508, 510, 512, 514) that is hovering via the second GCS 505 to be directed 522 toward the target 516. The second operator 504 maintains the PID of the target 516 during the selection of the second aircraft 506 via the video stream sent by the first aircraft 502 to the second GCS 505 via the second GCS 505. The remaining plurality of aircraft (508, 510, 512, 514) remain hovering until they need to be directed toward the target 516 and / or a new target by the second operator 504.

[0042] Figure 6 An embodiment of a UAS is depicted with a first operator 600 controlling a first aircraft 602 via a first GCS 601 and a second operator 604 controlling a plurality of aircraft (606, 608, 610, 612, 614) that are being directed toward a target 616 at set times per interval via a second GCS 605. The first operator 600 can identify a target 616 in a FOV 618 of a first camera 620 on the first aircraft 602 via the first GCS 601. The second operator 604 can select a valid targeting coordinate based on the target 616 via the second GCS 605 (see FIG. 1 ). Figure 2). Once a valid targeting coordinate is selected by the second operator 604, multiple aircraft (606, 608, 610, 612, 614) can be lined up to follow and be guided to the target 616, so that each of the multiple aircraft (606, 608, 610, 612, 614) is separated by a set time. The second operator 604 can observe the video feed from the first aircraft 602 and the aircraft 606 closest to the target 616 via the second GCS 605 to maintain the PID of the target 616. The second operator 604 can also control the aircraft 606 closest to the target 616, while the remaining aircraft (608, 610, 612, 614) can continue to rely on the autopilot. Once the aircraft 606 contacts the target, the second operator 604 can continue to observe the video feed from the first aircraft 602 via the second GCS 605. The second operator 604 can then control the aircraft 608 (which is now the closest aircraft to the target 616) via the second GCS 605 and observe the video feed from the third aircraft 608. As each aircraft (606, 608, 610, 612, 614) contacts the target, switching control and / or observing the video feed from the subsequent aircraft (608, 610, 612, 614) can be automatically controlled by the computing device of the second GCS 605 (see Figure 2 and Figure 7 ). This process may continue until target 616 is eliminated and / or there are no remaining aircraft for second operator 604 to control.

[0043] Figure 7 An exemplary top-level functional block diagram of a computing device embodiment of a GCS and / or one or more UAS 700 is shown. The exemplary embodiment 700 is shown with a processor 724 (such as a central processing unit (CPU)), addressable memory 727, external device interfaces 726 (e.g., an optional universal serial bus port and associated processing and / or an Ethernet port and associated processing), and an optional user interface 729 (see Figure 2 ) (e.g., an array of status lights and one or more toggle switches, and / or a display, and / or a keyboard, and / or a pointer-mouse system, and / or a touch screen). Optionally, the addressable memory 727 can be, for example: flash memory, eprom, and / or a disk drive or other hard disk drive. These elements can communicate with each other via a data bus 728. The processor 724 can have an operating system 725 (such as, an operating system supporting a web browser 723 and / or applications 722), which can be configured to perform process steps according to the exemplary embodiments described herein.

[0044] It is contemplated that various combinations and / or sub-combinations of the specific features and aspects of the above-described embodiments may be performed and still fall within the scope of the present invention. Therefore, it should be understood that the various features and aspects of the disclosed embodiments may be combined or substituted with each other to form a variation of the disclosed invention. In addition, the scope of the present invention, which is intended to be disclosed herein by way of example, should not be limited to the above-described specific disclosed embodiments.

Claims

1. A aiming method, comprising: capturing, via a first video sensor on the first aircraft, a first video of a scene surrounding the potential targeting coordinates; selecting the potential targeting coordinates as actual targeting coordinates for a second aircraft based on the first video; guiding and launching the second aircraft toward the actual targeting coordinates; capturing a second video of the scene via a second video sensor on the second aircraft after the second aircraft is launched; When the second aircraft approaches the actual aiming coordinates, displaying the second video close to the first video; wherein, based on the target being identified in both the first video and the second video, positive identification is performed; as well as The second aircraft is controlled to impact the target while maintaining the positive identification.

2. The method according to claim 1, wherein: The selected actual aiming coordinates are geographic coordinates corresponding to pixels in the field of view FOV of the captured first video.

3. The method according to claim 1, further comprising: sending the first video and associated potential targeting coordinates; as well as The first video is received on a first display in communication with a second processor that also receives the potential targeting coordinates.

4. The method according to claim 1, further comprising: launching the second aircraft, the second aircraft having the second video sensor; as well as The first aircraft is guided about the actual targeting coordinates to maintain a field of view FOV of the first video sensor about the actual targeting coordinates.

5. The method according to claim 1, further comprising: For the second aircraft, an entry point to the actual targeting coordinates is set based on a predetermined distance from the target and a wind speed.

6. The method according to claim 1, wherein: The maintaining the positive identification further comprises: The received first video of the scene and the received second video of the scene on a first display are compared to confirm that the second aircraft is approaching the actual targeting coordinates.

7. The method according to claim 1, further comprising: In response to observing the first video and the second video on the first display, a terminal homing mode is selected in which the second aircraft is aimed at the target, wherein the terminal homing mode also includes at least one of the following items: the second aircraft impacting the target, the payload deployed by the second aircraft impacting the target, and the marker deployed by the second aircraft impacting the target.

8. The method according to claim 7, further comprising: In response to observing the first video on the first display, a condition of the target after impact is confirmed.

9. The method according to claim 7, further comprising: capturing, by the first video sensor on the first aircraft, a new first video of a new scene around new potential targeting coordinates; transmitting, via the first aircraft, the new first video and the associated new potential targeting coordinates; receiving the new first video on the first display and receiving the new potential targeting coordinates; selecting the new potential targeting coordinates as new actual targeting coordinates for a third aircraft in response to observing the new first video on the first display; as well as directing the third aircraft toward the new actual targeting coordinates; wherein positive identification of the new target corresponding to the new actual targeting coordinates is maintained from the selection of the new actual targeting coordinates; and Therein, the selected potential aiming coordinates are geographic coordinates corresponding to the central field of view CFOV of the captured new first video.

10. A method comprising: receiving, by a ground control system GCS, a first video stream and potential targeting coordinates associated with the first video stream from a first aircraft, the first aircraft being configured to transmit the first video stream; selecting the potential aiming coordinates corresponding to the target as actual aiming coordinates by the GCS; receiving, by the GCS, a second video stream from a second aircraft, the second aircraft being configured to transmit the second video stream; as well as guiding the second aircraft to the actual targeting coordinates via the GCS; wherein the GCS performs positive identification based on the target being identified in both the field of view FOV of the received first video stream and the FOV of the received second video stream; as well as The second aircraft is controlled, via the GCS, to impact the target while maintaining the positive identification.

11. The method according to claim 10, wherein: The selected actual aiming coordinates are geographic coordinates corresponding to pixels in the field of view FOV of the received first video stream.

12. The method according to claim 10, further comprising: A terminal homing mode is selected via the GCS for aligning the second aircraft with the target.

13. The method according to claim 12, wherein: The terminal homing mode includes at least one of: impacting the target by the second aircraft, impacting the target by a payload deployed by the second aircraft, and impacting the target by a marker deployed by the second aircraft.

14. The method according to claim 13, further comprising: confirming, via the GCS, a condition of the target after impact via the received first video stream from the first aircraft; selecting, by the GCS, a new potential targeting coordinate corresponding to the new target as a new actual targeting coordinate; receiving, by the GCS, a third video stream from a third aircraft, the third aircraft being configured to transmit the third video stream; as well as guiding the third aircraft to the new actual targeting coordinates via the GCS; Therein, the positive identification of the new target corresponding to the new actual targeting coordinates is maintained starting from the selection of the new actual targeting coordinates.

15. The method according to claim 10, wherein: The selected potential targeting coordinates are geographical coordinates corresponding to a central field of view CFOV of the received first video stream.

16. A system comprising: a first aircraft configured to transmit a first video stream and potential targeting coordinates associated with the first video stream; a second aircraft configured to transmit a second video stream; A ground control system GCS, the GCS being configured to: receiving the first video stream and the potential targeting coordinates associated with the first video stream; selecting the potential aiming coordinates corresponding to the target as actual aiming coordinates; receiving the second video stream; directing the second aircraft toward the actual targeting coordinates; wherein positive identification is performed based on the target being identified in both the first video stream and the second video stream; as well as The second aircraft is controlled to impact the target while maintaining the positive identification.

17. The system of claim 16, wherein: The GCS further includes a first digital data link DDL to guide the first aircraft and receive the first video stream.

18. The system of claim 16, wherein: The selected potential aiming coordinates are geographical coordinates corresponding to the central field of view CFOV of the first video stream.

19. The system of claim 16, wherein: The GCS is also configured to: A terminal homing mode is selected in which the second aircraft is directed toward the target.

20. The system of claim 19, wherein: The terminal homing mode includes at least one of: impacting the target by the second aircraft, impacting the target by a payload deployed by the second aircraft, and impacting the target by a marker deployed by the second aircraft.

Images