DRONE-BASED TRACKING

Abstract

Computer (110) programmed to: Determining a trajectory of a moving target object (210) based on data from one or more vehicle sensors (120); and Dispatching a flying drone (135) from the vehicle 100 to track the moving target object (210) based on the determined trajectory,

Description

GENERAL STATE OF THE ART

[0001] Police vehicles or the like may be equipped with electronic devices, such as sensors and computers, that provide assistance in tracking a suspect vehicle. For example, a first vehicle may include camera sensors that can detect a suspect second vehicle and provide information regarding the second vehicle's direction of movement, speed, etc. However, a person and / or object may exit the second vehicle and move to an area outside of the detection range of the sensors included in the first vehicle.

[0002] US 2015 / 0268 338 A1 discloses a computer in a vehicle that collects data relating to a person outside the vehicle. The collected data is used to create a virtual map of one or more targets, including at least the person, in the vicinity of the vehicle. Based on the virtual map, a display is created that contains information about the person's location.

[0003] The document US 2015 / 0 321 758 A1 discloses the deployment of a drone and a corresponding control system. The drone launches from a vehicle or a stationary object such as a building to track a fleeing person.

[0004] The document "DJI developer challenge official rules." Version 1.0. Nanshan, 2016. 8 pages. discloses a scenario for the use of drones in search and rescue missions. This involves drones taking off from and landing on vehicles.

[0005] US Patent No. 9,471,059 B1 discloses miniaturized drones and an assistance system for controlling them. A use case for assisting police officers during traffic checks is also disclosed.

[0006] The document US 2016 / 0 304 217 A1 discloses an apparatus, a system, and a method for drones. The drones have takeoff and landing points on people's clothing or equipment.

[0007] The document US 2016 / 0 332 748 A1 discloses systems and methods relating to a vehicle that interacts with a drone.

[0008] The document WO 2016 / 012 867 A2 discloses a method for using a drone to track a target.

[0009] US Pat. No. 7,299,130 B2 discloses methods and apparatus for monitoring a convoy. This involves the use of a drone that orbits the convoy. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The Fig. 1A is a block diagram of an example tracking system and an example vehicle with a trunk door in an open position. Fig. 1B shows the vehicle from Fig. 1A with the trunk door in a closed position and the drone placed in a vehicle trunk. Fig. Figure 2 is a diagram showing vehicles and a person exiting one of the vehicles. Fig. Figure 3 is a diagram showing vehicles, drones, and a person moving within a structure. The Fig. 4A-4C is a flowchart of an example process for a host vehicle controlling a drone that is tracking a target object. Fig. 5 is a flowchart of an example process for a client vehicle receiving control commands from a host vehicle. DETAILED DESCRIPTION INTRODUCTION

[0011] Disclosed herein is a method comprising determining a trajectory of a moving target object based on data from one or more vehicle sensors and dispatching an aerial drone from the vehicle to track the moving target object based on the determined trajectory.

[0012] The method may further include determining a current location of the target object based on the data from the one or more vehicle sensors and sending an instruction to the aerial drone to fly to a target location based on the determined current location.

[0013] The method may further include determining a location based on the determined trajectory and operating the drone to fly to the determined location.

[0014] The method may further include tracking the target object by determining a current location of the target object based on sensor data of the aerial drone and actuating the drone to fly within a certain distance from the target object while the target object is moving.

[0015] The method may further include tracking the target object based on a current trajectory of the target object determined based on the sensor data of the aerial drone.

[0016] The method may further include determining that the target object has entered an area inaccessible to the drone and communicating a last determined location and trajectory of the moving target object to a remote computer.

[0017] The method may further include detecting a second vehicle based on the data from the one or more vehicle sensors and detecting the moving target object when the moving target object exits the second vehicle. Detecting the moving target object may be performed only after determining that a door of the second vehicle is open.

[0018] The method may further include actuating a drone actuator to spray ink onto the moving target object.

[0019] The method may further include tracking the moving target object, which further includes detecting a particular ink on the moving target object.

[0020] Further disclosed is a computing device programmed to perform any of the above method steps. Further disclosed is an aerial drone comprising the computing device. Further disclosed is a vehicle comprising the computing device.

[0021] Furthermore, a computer program product is disclosed that includes a computer-readable medium storing instructions executable by a computer processor to perform any of the above method steps. EXAMPLE SYSTEM ELEMENTS

[0022] The Fig. 1A-1B illustrate a vehicle 100. The vehicle 100, sometimes referred to as a first or host vehicle 100 for convenience, may be powered in a variety of known ways, such as using an electric motor and / or an internal combustion engine. The vehicle 100 may include any motor vehicle, such as a car, a truck, an SUV, a van, a minivan, etc. The vehicle 100 may include a computer 110, actuator(s) 115, sensor(s) 120, and a human machine interface (HMI) 125. In some possible approaches, as discussed below, the vehicle 100 is an autonomous vehicle 100 configured to operate in an autonomous (e.g., driverless) mode, a partially autonomous mode, and / or a non-autonomous mode.

[0023] Computer 110 includes a processor and memory, as known in the art. Memory includes one or more forms of computer-readable media and stores instructions executable by computer 110 to perform various operations, including those disclosed herein.

[0024] The computer 110 can operate the vehicle 100 in an autonomous mode, a semi-autonomous mode, or a non-autonomous mode. For the purposes of this disclosure, an autonomous mode is defined as one in which each of the propulsion, braking, and steering of the vehicle 100 is controlled by the computer 110; in a semi-autonomous mode, the computer controls one or two of the propulsion, braking, and steering of the vehicle 100; in a non-autonomous mode, an operator of the vehicle 100 controls the propulsion, braking, and steering of the vehicle.

[0025] Computer 110 may include programming to operate one or more of the braking, propulsion (e.g., controlling acceleration in vehicle 100 by controlling one or more of an internal combustion engine, electric motor, hybrid engine, etc.), steering, climate control, interior and / or exterior lighting, etc., of land vehicle 100, as well as to determine if and when computer 110, as opposed to a human operator, should control such operations. Additionally, computer 110 may be programmed to determine if and when a human operator should control such operations.

[0026] The computer 110 may include or be communicatively coupled to more than one processor, e.g., controllers or the like included in the vehicle 100 for monitoring and / or controlling various vehicle controls, e.g., powertrain control, braking control, steering control, etc., e.g., via a network of the vehicle 100, e.g., including a communication bus, as described in more detail below. The computer 110 is generally arranged for communication within a communication network of the vehicle 100, which may include a bus within the vehicle 100, such as a Controller Area Network (CAN) or the like, and / or other wired and / or wireless mechanisms.

[0027] Via the vehicle 100 communications network, the computer 110 may transmit messages to and / or receive messages from various devices within the vehicle 100, e.g., an actuator 115, an HMI 125, etc. Alternatively or additionally, in cases where the computer 110 actually includes multiple devices, the vehicle communications network may be used for communications between devices, which are depicted in this disclosure as the computer 110.

[0028] The actuators 115 of the vehicle 100 are implemented via circuits, chips, or other electronic and / or mechanical components that can actuate various subsystems of the vehicle 100 according to suitable control signals, as known. The actuators 115 can be used to control vehicle systems, such as braking, acceleration, and / or steering of the vehicle 100.

[0029] The sensors 120 of the vehicle 100 may include a variety of devices known for providing data over the vehicle communication bus. For example, the sensors 120 may include one or more camera, radar, infrared, and / or LIDAR (Light Detection And Ranging) sensors 120 disposed within and / or on the vehicle 100 and providing data covering at least a portion of the exterior of the vehicle. The data may be received by the computer 110 via a suitable interface, as are known. A LIDAR sensor 120, for example, disposed on a top surface of the vehicle 100, may provide object data including relative locations, sizes, and shapes of objects, such as other vehicles, surrounding the vehicle 100.A computer 110 may receive the object data and operate the vehicle 100 in an autonomous and / or semi-autonomous mode based at least in part on the received object data.

[0030] The vehicle 100 may include a global positioning system (GPS) sensor 120 configured to determine coordinates of a current location of the vehicle 100. The navigation system 110 may be further configured to identify a route from the current location to a selected destination and display a map and driving directions to the selected destination, e.g., via the HMI 125.

[0031] Additionally, the computer 110 may be configured to communicate through a vehicle-to-infrastructure (V-to-I) interface with other vehicles 100, drones 135, and / or a remote computer 155 via a network 160. The network 160 represents one or more mechanisms by which the computer 110 and the remote computer 155 may communicate with each other, and may be one or more of various wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable or fiber optic) and / or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or topologies if multiple communication mechanisms are used). Example communication networks 160 include wireless communication networks (e.g.,using one or more of cellular, Bluetooth, IEEE 802.11, etc.), dedicated short range communication (DSRC), local area networks (LAN) and / or wide area networks (WAN), including the Internet, that provide data communication services.

[0032] The HMI 125 presents information to and receives information from an occupant of the vehicle 100. The HMI 125 may be located, for example, on an instrument panel in a passenger cabin of the vehicle 100 or one or more other locations accessible to the occupant. The HMI 125 includes dials, digital displays, screens, such as a touchscreen, speakers, and so forth for providing information to the occupant. The HMI 125 may include buttons, knobs, keypads, a microphone, and so forth for receiving information from the occupant. In this example, the computer 110 of the vehicle 100 may be programmed to control a second vehicle 200 (see Fig. 2) select for tracking based on the information received via the HMI 125.

[0033] For example, the computer 110 may be programmed to display image data on the HMI 125, including multiple vehicles 200. The computer 110 may be further programmed to select a vehicle 200 for tracking after touching a screen of an HMI 125 at an area of the HMI screen where the vehicle 200 is displayed. As discussed below, the computer 110 of the vehicle 100 may be programmed to output information on the HMI 125, including a particular location of the selected vehicle 200 and / or a location of its occupants if one or more of the selected vehicle 200 exits the selected vehicle 200.

[0034] The drone 135 is an unmanned aerial vehicle (UAV) and includes a series of circuits, chips, or other electronic and / or electromechanical components that can control various operations of the drone 135. For example, the drone 135 can fly according to control signals output to its propeller actuators 140. The drone 135 can be equipped with a navigation system so that it can fly to and hover at a specific location. Fig. 1A illustrates the drone 135 as it is located in a trunk 165 of the vehicle 100. Fig. 1B illustrates the drone 135 hovering over the host vehicle 100.

[0035] The drone 135 may include one or more camera sensors 145 that can capture images of an area near the drone 135. The camera sensors 145 of the drone 135 may be mounted on the same housing as the lights, and the drone 135 may be programmed to activate the camera sensors 145 of the drones 135 to capture images of an area below the drone 135. Thus, when hovering above the vehicle 100, the camera sensor 145 of the drone 135 may capture images of the vehicle 100 and possibly the area surrounding the vehicle 100. Additionally or alternatively, the drone 135 may include other types of object detection sensors 145, such as radar, LIDAR, cameras, etc.

[0036] The drone 135 may include a GPS sensor 145 that provides GPS location coordinates of the drone 135 to, e.g., a computer 110 of the vehicle 100, a computer of the drone 135, etc. For example, the computer of the drone 135 may navigate the drone 135 based on the received GPS location coordinates, a predetermined route, etc.

[0037] The drone 135 may include one or more lights. For example, a computer 110 may be programmed to activate the light 150 of the drone 135 to illuminate an area surrounding the drone 135.

[0038] The drone computer 135, the vehicle computer 100, etc., may communicate with each other and with a remote computer 155 via a network 160 that includes one or more telecommunications protocols, e.g., cellular technologies such as 3G, 4G, Long Term Evolution (LTE), etc., Bluetooth®, Bluetooth Low Energy®, Wi-Fi, etc.

[0039] As in Fig. 2, the computer 110 of the vehicle 100 may be programmed to determine a trajectory m1 of a moving target object 210 based on data received from the sensors 120 of the vehicle 100. The computer 110 may be further programmed to dispatch the drone 135 from the vehicle 100 to track the moving target object 210 based on the determined trajectory m1.

[0040] A trajectory, as used in the present disclosure, refers to an expected movement path of a target object that starts immediately from a current location of the target object 210. The trajectory m1 of a moving human target object 210 may include a curvature or a straight line on a ground surface that identifies an expected path of the human target object 210. For example, the computer 110 may be programmed to determine the trajectory m1 by repeatedly (i.e., two or more times) identifying a location of the target object 210, e.g., based on data received from the LIDAR sensor 120, and adjusting a curvature between the identified locations using known adjustment techniques. In one example, the computer 110 may be programmed to readjust a predetermined trajectory m1 based on a newly determined location of the target object 210.For example, upon determining that a current location of the target object 210 is not on the predetermined trajectory m1 of the target object 210, the computer 110 may be programmed to adjust a new curvature (i.e., a set trajectory) between the current location and the previously determined locations of the target object 210. Additionally or alternatively, the computer 110 may be programmed to determine the trajectory m1 of a human target object 210 based on various characteristics of the human target object 210, e.g., a speed, a direction of a face of a target object 210, a posture, etc. For example, the computer 110 may be programmed to determine the trajectory m1 based on a virtual line that is perpendicular to a chest of the human target object 210 and directed toward a forward direction of the target object 210.

[0041] As an example, the computer 110 may be programmed to dispatch the drone 135 because the target object 210 is not accessible by the vehicle 100. For example, a user in the host vehicle 100 may select a second vehicle 200 to track. In other words, the computer 110 may display image data via the HMI 125, including multiple vehicles 200 within a detection range 250 of the sensors 120 of the vehicle 100. For example, the computer 110 may be programmed to select the vehicle 200 based on input via the HMI 125, e.g., the touchscreen. The computer 110 may be programmed to display information on the HMI 125 indicating a current location and / or trajectory of the selected vehicle 200 relative to the host vehicle 100. In one example, the vehicle 100 may control the operation of the vehicle 100, e.g., accelerating, steering, braking, etc.to track (or track) the selected vehicle 200. The computer 110 may be programmed to operate the host vehicle 100 in an autonomous or semi-autonomous mode. Thus, the computer 110 may be programmed to navigate the vehicle 100 based on the current location and / or trajectory of the selected second vehicle 200.

[0042] The selected vehicle 200 may stop, e.g., due to a road blockage. In one example, a target object 210, e.g., a human suspect, may exit the selected vehicle 200. The computer 110 may be programmed to detect the target object 210 upon determining that a door 240 of the selected vehicle 200 is open, e.g., based on data received from the sensors 120 of the vehicle 100. The computer 110 may be programmed to dispatch the drone 135 to track the target 210.

[0043] The computer 110 may be programmed to deploy the drone 135 from the trunk 165 of the vehicle 100, e.g., upon determining that the vehicle 100 is unable to track the target object 210. To deploy the drone 135 from the host vehicle 100, the computer 110 may actuate a trunk opening actuator 115 to open a door 170 of the trunk 165, e.g., from a closed position (see Fig. 1B) into an open position (see Fig. 1A). The computer 110 may be further programmed to actuate a dispatching device 175 of the vehicle 100 to release the drone 135 from any type of in-vehicle locks that otherwise secure the drone 135 when not in use. Once released from the locks, the drone 135 actuation commands may include an actuation command for the drone 135 to exit the host vehicle 100 (i.e., fly out of the trunk 165) and over the host vehicle 100 (see Fig. 1A) to float.

[0044] The dispatching devices 175 are implemented via electromechanical actuators, such as electromagnets or other types of electromechanical devices that convert electrical signals into motion. One or more actuators 115 may be used to lock the drone 135 in or to the host vehicle 100 when the dispatchable drone 135 is not in use. That is, these dispatching devices 175 may be in a locked state while the dispatchable drone 135 is stored, for example, in a trunk 165 of the host vehicle 100, and may transition to an unlocked state in response to receiving a control signal output from, for example, the computer 110. For example, the dispatching device 175 may include one or more electromagnets that, when actuated, maintain the drone 135 in the locked position, for example, when the drone 135 is in the locked position. B. by securing a body of the drone 135.For example, the deployment device 175 may include a clamp mechanically coupled to an electromagnet. Upon activation of the electromagnet by the computer 110, the clamp may hold the drone 135, and upon deactivation of the electromagnet, the clamp may release the drone 135. Upon receiving a release signal from the computer 110, the deployment device 175 may move to the unlocked position and release the drone 135. Thus, in response to control signals output by the computer 110, the deployment devices 175 may transition to the unlocked state so that the drone 135 can be released from, for example, the trunk 165. The trunk-opening actuator 115 may be used to open the door 170 of the trunk 165 or another door that allows the drone 135 to exit the host vehicle 100.The trunk opening actuator 115 may include an electromagnet, and the computer 110 may be programmed to activate the trunk opening actuator 115 to open the door 170 of the trunk 165.

[0045] The computer 110 may be programmed to determine a current location of the target object 210 based on the data from the sensor 120 of the vehicle 100 and send an instruction to the drone 135 to fly to a destination based on the determined location of the target object 210. In one example, the computer 110 may determine the current location of the target object 210 upon detecting the target object 210 leaving the vehicle 100, e.g., based on data received from the sensors 120 of the vehicle 100. The computer 110 may output the instructions, including a location (e.g., according to latitude and longitude geocoordinates) of the target object 210. For example, the computer 110 may be programmed to determine the location of the target object 210 based on the relative location of the target object 210 to the vehicle 100, e.g., using image processing techniques, and the location of the vehicle 100, e.g.,received by the GPS sensor 120 of the vehicle 100.

[0046] Therefore, operating the drone 135 to fly to the location of the target object 210 based on data from the sensor 120 of the vehicle 100 can advantageously prevent the risk of losing the target object 210. For example, in a crowded area, the target object 210 may quickly move, e.g., in a few seconds, outside the detection range 250 of the sensors 120 of the vehicle 100 after leaving the selected vehicle 200. Therefore, operating the drone 135 to fly to the location of the target object 210 based on data from the sensor 120 of the vehicle 100 can provide uninterrupted tracking of the target object 210. Continuous tracking as used herein refers to a transition from tracking the target object 210 based on data from the sensors 120 of the vehicle 100 to tracking the target object 210 based on the sensors 145 of the drone 135.

[0047] Upon dispatching the drone 135 from the vehicle 100, the computer 110 may, in one example, be programmed to determine a location 220 based on the determined trajectory m1 and to operate the drone 135 to fly to the determined location 220, e.g., instead of the current location of the target object 210, as in Fig. 2. For example, the computer 110 may be programmed to determine a location 220 where adding a distance d1 from the current location of the target object 210 and a traveled distance d2 of the target object 210 is longer than a flight distance d3 from the vehicle 100 to the location 220. The computer 110 may be programmed to determine an area 230 around the location 220 and actuate the drone 135 to fly to the area 230. In one example, the area 230 may be a circle centered at the location 220 with a diameter of 10 meters.

[0048] The computer 110 may be programmed to track the target object 210, e.g., after detecting the target object 210 within the area 230, by determining a current location of the target object 210 based on data from the sensor 145 of the drone 135. The computer 110 and / or the computer of the drone 135 may be programmed to determine, e.g., periodically every 50 ms, an updated trajectory m2 of the target object 210 and to fly the drone 135 within a certain distance, e.g., 5 meters, from the target object 210 while the target object 210 is moving. The computer 110 may be programmed to track the target object 210 based on a current trajectory m2 of the target object 210 determined based on the data from the sensor 145 of the drone 135.

[0049] The computer 110 may be programmed to receive data, including the location and / or trajectory of the drone 135 and / or the target object 210, and output image and / or text data to the HMI 125 of the vehicle 100 based on the received data. For example, the computer 110 may display a schematic image of the target object 210 on a map via the HMI 125. The computer 110 may be programmed to output data, e.g., image, text, and / or audio data, to a user's mobile device.

[0050] The computer 110 may be programmed to actuate a spray actuator of the drone 135 to spray ink onto the target object 210. For example, the computer 110 may be programmed to actuate the drone 135 to approach the target object 210, e.g., within 3 meters, and spray ink, e.g., a photoluminescence-based ink that emits light under low-light conditions. Thus, the drone 135 may advantageously have improved detection of the target object 210, e.g., under low-light conditions. For example, the computer 110 may be programmed to detect the target object 210 (or to distinguish the target object 210 from non-sprayed objects) based on an expected wavelength of light emitted by the sprayed ink. Additionally or alternatively, the computer 110 may be programmed to spray ink onto a surface, e.g., a surface. B.a landmark, an edge of a building 310, etc., within a predetermined distance, such as 5 meters, from the target object 210. Therefore, another person and / or drone 135 tracking the target object 210 can advantageously determine that the target 210 was within the predetermined distance of the sprayed surface based on the expected wavelength of light emitted by the sprayed surface.

[0051] In an example, illustrated in Fig. 3, the computer 110 of the host vehicle 100 dispatches a drone 135 to track the target object 210. The target object 210 may enter an area inaccessible to the drone 135, e.g., a hallway 320 of a building 310, a tunnel, a crowd, etc. The computer 110 of the vehicle 100 may be programmed to determine that the target object 210 has entered the building 310 and may transmit a last determined location, e.g., entrance location 330 of the building 310 and / or a last determined trajectory m1 of the target object 210 to a remote computer 155.

[0052] In one example, the remote computer 155 and / or the computer 110 of the vehicle 100 may be programmed to estimate an exit location 340 of the building 310 based on the entry location 330, the last trajectory m1, and an interior floor plan of the building 310. For example, the remote computer 155 may be programmed to estimate the exit location 340 based on a floor plan including the hallway 320. The remote computer 155 may be further programmed to identify a third vehicle 101 (referred to herein as a third vehicle 101 for simplicity and to distinguish it from the vehicles 100, 200) based on the estimated exit location 340 and the location coordinates of the third vehicle 101.The computer 110 of the third vehicle 101, upon receiving an instruction from the remote computer 155, may operate a second drone 136 from the third vehicle 101 to fly to the expected exit location 340. Additionally or alternatively, the remote computer 155 may be programmed to identify multiple possible exit locations and operate multiple drones 135 (e.g., from multiple vehicles 100) to fly to the identified exit locations. The host and second vehicles 100, 101 may share common elements, including a computer 110, actuators 115, sensors 120, a trunk 165, and / or an HMI 130, as discussed above. The drone 135 associated with the host vehicle 100 and the second drone 136 associated with the third vehicle 101 may have common elements, including propeller actuators 140, sensors 145, etc., as discussed above. PROCESSING

[0053] The Fig. 4A-4C illustrate an exemplary process 400 for tracking a target object 210. For example, the computer 110 of the vehicle 100 may be programmed to execute the blocks of process 400. Additionally or alternatively, the computer 110 of the vehicle 100, the computer of the drone 135, the remote computer 155, and / or a combination thereof may be programmed to execute blocks of process 400.

[0054] The process 400 begins at a block 405, in which the computer 110 outputs image data including a surrounding area, e.g., the detection area 250, of the host vehicle 100. In one example, the computer 110 may be programmed to receive data from various sensors 120 of the vehicle 100, e.g., LIDAR, radar, camera sensors 120, and output images based on the received sensor data to a display of the HMI 125 of the host vehicle 100.

[0055] Next, in a decision block 410, the computer 110 determines whether a second vehicle 200 is selected. Further, the computer 110 may be programmed to select operation of the vehicle 200 or any other object based on input via the HMI 125. For example, a user in the host vehicle 100 may touch a screen of the HMI 125 to select a vehicle 200 shown on the screen of the HMI 125. If the computer 110 determines that a vehicle 200 is selected, the process 400 proceeds to a block 415; otherwise, the process 400 returns to decision block 405.

[0056] In block 415, the computer 110 navigates the vehicle 100 to track the selected vehicle 200. For example, the computer 110 may be programmed to track the selected vehicle 200 by operating the vehicle 100 in an autonomous mode such that a predetermined distance, e.g., 20 meters, is maintained between the host vehicle 100 and the selected vehicle 200. As another example, the computer 110 may be programmed to output navigation instructions to the HMI 125 of the vehicle 100, and a user of the vehicle 100 may control the vehicle 100 in a non-autonomous mode based on the output navigation instructions.

[0057] Next, in a decision block 420, the computer 110 determines whether a door 240 of the selected vehicle 200 is open. For example, the computer 110 may be programmed to determine whether a door 240 of the vehicle 200 is open based on data received, for example, from the LIDAR, radar, or camera sensors 120 of the vehicle 100. Additionally or alternatively, the computer 110 may be programmed to determine whether the selected vehicle 200 has an open roof, such as a convertible. In another example, the computer 110 may be programmed to determine whether a door of a trailer attached to the selected vehicle 200 is open. If the computer 110 determines that a door 240 of the selected vehicle 200 is open, the process 400 proceeds to a decision block 425; otherwise, process 400 returns to block 415.

[0058] At decision block 425, the computer 110 determines whether the target object 210 has left the selected vehicle 200, e.g., using known image processing techniques. If the computer 110 determines that the target object(s) 210 have left the selected vehicle 200, the process 400 proceeds to a block 430; otherwise, the process 400 returns to decision block 425.

[0059] In block 430, the computer 110 determines a trajectory and / or location of the target object 210. For example, as in Fig. 2, the computer 110 may be programmed to determine a location and / or trajectory m1 of the target object 210. Additionally or alternatively, the computer 110 may be programmed to determine locations and / or trajectories of multiple target objects exiting the selected vehicle 200. Additionally or alternatively, the computer 110 may be programmed to select the target object 210 from among multiple detected individuals exiting the selected vehicle 200, e.g., based on facial recognition.

[0060] Now with reference to Fig. 4B, in a block 435, the computer 110 dispatches the drone 135, e.g., by actuating the drone 135 in the trunk 165 of the host vehicle 100 to fly out of the vehicle 100. The computer 110 may be programmed to dispatch the drone 135 by actuating an actuator 115 of the vehicle 100 to unlock the drone 135, actuating an actuator 115 of the vehicle 100 to open the door 170 of the trunk 165, and actuating the drone 135 to fly. The computer 110 may be programmed to transmit data to the drone 135 including the determined location and / or trajectory of the target object 210.

[0061] Next, in a block 440, the computer 110 navigates the drone 135 based on the determined location and / or trajectory. For example, as in Fig. 2, the computer 110 may be programmed to navigate the drone 135 based on the trajectory m1 of the target object 210, the location of the target object 210, the location of the host vehicle 100, etc.

[0062] Next, in a block 445, the computer 110 actuates the drone 135 to track the target object 210. The computer 110 may be programmed to track the target object 210 based on the location and / or trajectory of the target object 210. For example, the computer 110 may be programmed to determine a current location of the target object 210 based on the data from the sensor 145 of the drone 135 and actuate the drone 135 to fly within a certain distance, e.g., within a range 230, of the target object 210 while the target object 210 is moving. In one example, the computer 110 may be programmed to determine the trajectory m1 of the target object 210 by repeatedly identifying a location of the target object 210, e.g.,based on data received from the sensors 145 of the drone 135, and adjusting a curvature between the identified locations using known adjustment techniques. Additionally, the computer 110 may be further programmed to actuate an actuator of the drone 135 to spray ink onto a target object 210 and / or a surface within a predetermined distance from the target object 210 while the drone 135 tracks the target object 210.

[0063] Next, in a decision block 450, the computer 110 determines whether a tracking session of the target object 210 is complete, e.g., based on receiving input via the HMI 125 of the vehicle 100. If the computer 110 determines that the tracking session is complete, the process 400 proceeds to a block 452; otherwise, the process 400 proceeds to a decision block 455.

[0064] At block 452, the computer 110 operates the drone 135 to navigate to a current location of the host vehicle 100, e.g., based on location coordinates received from the GPS sensor 120 of the vehicle 100. The computer 110 may be further programmed to operate the drone 135 to land in a designated area, e.g., the trunk 165 of the host vehicle 100. For example, the computer 110 may be programmed to open the door 170 of the trunk 165 to operate the drone 135 to land in the trunk 165. The computer 110 may actuate an actuator of the vehicle 100 to lock the drone 135 in place and may then actuate the door 170 of the trunk 165 to close. Following block 452, the process 400 ends or alternatively returns to block 405, although this may be Fig. 4A-4B is not shown.

[0065] In decision block 455, the computer 110 determines whether the target object 210 has entered a non-accessible area, e.g., a building 310, a tunnel, etc. If the computer 110 determines that the target object 210 has entered a non-accessible area, the process 400 proceeds to a block 460 (see Fig. 4C); otherwise, process 400 returns to block 445.

[0066] With reference to Fig. 4C, in block 460, the computer 110 estimates an exit location of the target object 210 from the inaccessible area. In one example, the computer 110 may be programmed to estimate an exit location 340 of the building 310 based on the entry location 330, the last trajectory m1, and an interior floor plan of the building 310 (see Fig. 3). The computer 110 may be programmed to determine location coordinates of the estimated starting location(s).

[0067] Next, in a block 465, the computer 110 dispatches a drone 136 of a third vehicle 101 to the estimated starting location, e.g., by transmitting the location coordinates of the estimated starting location 340 to the remote computer 155.

[0068] For example, the computer 110 and / or the remote computer 155 may be programmed to identify the third vehicle 101 based on the identified location coordinates of the origin 340 and the location coordinates of the third vehicle 101. In one example, the computer 110 may be programmed to dispatch multiple drones 135 from one or more second vehicles 101, e.g., if multiple origins for the destination 210 are estimated within the accessible area.

[0069] Next, in a decision block 470, the computer 110 determines whether the computer 110 has received location coordinates of the target object 210 from the dispatched third vehicle 101. As described below with respect to Fig. 5, the computer 110 of the third vehicle 101 may be programmed to navigate to the starting location 340, detect the target object 210, and identify location coordinates and / or the trajectory of the target object 210. If the computer 110 determines that the location coordinates and / or the trajectory of the target object 210 have been received from the third vehicle 101, the process 400 proceeds to a block 475; otherwise, the process 400 proceeds to a block 480.

[0070] In block 475, the computer 110 navigates the drone 135 based on the received location coordinates of the target object 210. For example, the computer 110 may actuate the drone 135 to fly to a current location of the target object 210 based on the location coordinates and / or trajectory of the target object 210 received, e.g., from the drone 136 of the third vehicle 101, the remote computer 155, etc. Referring to Fig. 3, the computer 110 may be programmed to plan a route around the building 310 to navigate the drone 135 to the location of the target object 210.

[0071] In block 480, the computer 110 operates the drone 135 to hover over the inaccessible area entry point 330. Alternatively, the computer 110 may operate the drone 135 to return to the host vehicle 100. Following block 480, the process 400 proceeds to block 440 (see Fig. 4B).

[0072] Fig. 5 shows an example process 500 for a third vehicle 101 receiving commands from the host (or first) vehicle 100. For example, a computer 110 of the third vehicle 101 may be programmed to execute blocks of process 500.

[0073] The process 500 begins in a decision block 505 in which the computer 110 of the vehicle 101 determines whether, for example, coordinates of an estimated starting location 340 of a target object 210 have been received, e.g., from a computer 110 of the host vehicle 100, the remote computer 155, etc. If the computer 110 of the third vehicle 101 determines that the coordinates of the estimated starting location have been received, the process 500 proceeds to a block 510; otherwise, the process 500 returns to decision block 505.

[0074] In block 510, the computer 110 of the third vehicle 101 dispatches a second drone 136, e.g., from the trunk 165 of the third vehicle 101.

[0075] Next, in a block 515, the computer 110 of the third vehicle 101 navigates the second drone 136 to the existing location 340. The computer 110 of the third vehicle 101 may be programmed to navigate the second drone 136 based on the received location coordinates of the estimated starting location 340.

[0076] Next, in a decision block 520, the computer 110 of the third vehicle 101 determines whether the target object 210 is detected, e.g., within a predetermined distance of the starting location 340. For example, the computer 110 of the third vehicle 101 may be programmed to detect the target object 210 using known image processing techniques, e.g., based on information received from the host vehicle 100. The computer 110 of the third vehicle 101 may be programmed to detect the target object 210 based on received information, including an image of the target object 210, a wavelength of emitted light from an ink sprayed onto the target 210 by the drone 135 of the host vehicle 100, etc.If the computer 110 of the third vehicle 101 determines that the target 210 has been detected, the process 500 proceeds to a block 525; otherwise, the process 500 returns to decision block 520.

[0077] In block 525, the computer 110 of the third vehicle 101 transmits location coordinates and / or the trajectory of the detected target object 210, e.g., to the host vehicle 100, the remote computer 155, etc. The computer 110 may be programmed to track the detected target object 210, e.g., to fly within a predetermined distance of the target object 210. Additionally or alternatively, the computer 110 of the third vehicle 101 may be programmed to periodically, e.g., every second, transmit the current location coordinates and / or trajectory of the target object 210 while flying within a predetermined distance of the target object 210.

[0078] Next, in a decision block 530, the computer 110 of the third vehicle 101 determines whether a tracking session of the target object 210 is complete, e.g., upon determining that the drone 135 of the host vehicle 100 has arrived within a predetermined distance, such as 10 meters, of the target object 210. Additionally or alternatively, the computer 110 of the third vehicle 101 may be programmed to determine that a tracking session is complete upon receiving a control command from the drone 135 of the first vehicle 100 including a request to stop tracking the target object 210. If the computer 110 of the third vehicle 101 determines that the tracking session is complete, the process 500 proceeds to a block 535; otherwise, the process 500 proceeds to block 525.

[0079] At block 535, the computer 110 of the third vehicle 101 actuates the second drone 136 to return to the third vehicle 101. For example, the computer 110 actuates the second drone 136 to land in the trunk 165 of the third vehicle 101. Additionally, the computer 110 of the third vehicle 101 may be programmed to open the door 170 of the trunk 165 before the second drone 136 lands and actuates the door 170 of the trunk 165 to close after the second drone 136 lands.

[0080] Following block 535, process 500 ends.

[0081] The article "a" or "an" when modifying a noun should be understood to mean one or more, unless otherwise stated or the context requires otherwise. The phrase "based on" includes "based on" or "partially" or "fully" or "in part ...

[0082] Computing devices, as discussed in this document, generally each include instructions executable by one or more computing devices, such as those identified above, and for performing blocks or steps of processes described above. Computer-executable instructions may be compiled or interpreted by computer programs created using a variety of programming languages and / or technologies, including, among others, either alone or in combination, Java™, C, C++, Visual Basic, Java Script, Perl, HTML, etc. In general, a processor (e.g., a microprocessor) receives instructions, e.g., from memory, a computer-readable medium, etc., and executes those instructions, thereby performing one or more processes, including one or more of the processes described in this document.Such instructions and other data may be stored and transmitted using a variety of computer-readable media. A file in the computing device is generally a collection of data stored on a computer-readable medium, such as a storage medium, random access memory, etc.

[0083] A computer-readable medium includes any medium involved in providing data (e.g., instructions) that can be read by a computer. Such a medium can take many forms, including, but not limited to, non-volatile media, volatile media, and so on. Non-volatile media includes, for example, optical or magnetic disks and other persistent storage. Volatile media includes dynamic random access memory (DRAM), which is typically main memory.Common forms of computer-readable media include, for example, a floppy disk, a film disk, a hard disk, a magnetic tape, any other magnetic medium, a CD-ROM, a DVD, any other optical medium, punched cards, punched tape, any other physical medium with hole patterns, a RAM, a PROM, an EPROM, a FLASH, an EEPROM, any other memory chip or any other memory cartridge, or any other medium that a computer can read.

[0084] With respect to the media, processes, systems, methods, etc. described herein, it is understood that while the steps of such processes, etc., have been described as occurring according to a particular sequence, such processes could be implemented such that the described steps are performed in a different order than the order described herein. It is further understood that certain steps could be performed concurrently, other steps could be added, or certain steps described herein could be omitted. In other words, the descriptions of systems and / or processes herein are provided for the purpose of illustrating particular embodiments and should in no way be construed to limit the disclosed subject matter.

[0085] Accordingly, it is to be understood that the present disclosure, including the foregoing description and the appended figures and following claims, is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided will become apparent to those skilled in the art upon reading the foregoing description. The scope of the invention should not be determined by reference to the foregoing description, but instead by reference to claims appended hereto and / or included in a non-provisional patent application based hereon, along with the full scope of equivalents to which such claims are entitled.It is anticipated and intended that there will be future developments in the art discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. It is understood that the disclosed subject matter is susceptible to modification and variation.

Claims

A computer (110) programmed to: determine a trajectory of a moving target object (210) based on data from one or more vehicle sensors (120); and dispatch an aerial drone (135) from the vehicle 100 to track the moving target object (210) based on the determined trajectory, The computer (110) of claim 1, further programmed to: determine a current location of the target object (210) based on the data from the one or more vehicle sensors (120); and send an instruction to the aerial drone (135) to fly to a target location based on the determined current location. The computer (110) of claim 1, further programmed to:determine a location based on the determined trajectory; andactuate the drone (135) to fly to the determined location. The computer (110) of claim 1, further programmed to track the target object (210) by: determining a current location of the target object (210) based on sensor data from the aerial drone (135); and operating the drone (135) to fly within a specified distance of the target object (210) while the target object (210) is moving. The computer (110) of claim 4, further programmed to track the target object (210) based on a current trajectory of the target object (210) determined based on the sensor data of the aerial drone (135). The computer (110) of claim 1, further programmed to:determine that the target object (210) has entered an area inaccessible to the drone (135); andcommunicate a last determined location and trajectory of the moving target object (210) to a remote computer (155). The computer (110) of claim 1, further programmed to:detect a second vehicle (200) based on the data from the one or more vehicle sensors (120); anddetect the moving target object (210) when the moving target object (210) exits the second vehicle (200). The computer (110) of claim 7, further programmed to detect the moving target object (210) upon determining that a door (240) of the second vehicle (200) is open. The computer (110) of claim 1, further programmed to actuate a drone actuator to spray ink onto the moving target object (210). The computer (110) of claim 1, further programmed to track the moving target object (210) by detecting a particular ink on the moving target object (210). A method comprising: determining a trajectory of a moving target object (210) based on data from one or more vehicle sensors (120); and dispatching an aerial drone (135) from the vehicle (100) to track the moving target object (210) based on the determined trajectory. The method of claim 11, further comprising: determining a current location of the target object (210) based on the data from the one or more vehicle sensors (120); and sending an instruction to the aerial drone (135) to fly to a target location based on the determined current location. The method of claim 11, further comprising:determining a location based on the determined trajectory; andactuating the drone (135) to fly to the determined location. The method of claim 11, further comprising tracking the target object (210) by: determining a current location of the target object (210) based on sensor data of the aerial drone (145); and actuating the drone (135) to fly within a predetermined distance of the target object (210) while the target object (210) is moving. The method of claim 11, comprising tracking the target object (210) based on a current trajectory of the target object (210) determined based on the sensor data of the aerial drone (145). The method of claim 11, further comprising: determining that the target object (210) has entered an area inaccessible to the drone (135); and communicating a last determined location and trajectory of the moving target object (210) to a remote computer (155). The method of claim 11, further comprising:detecting a second vehicle (200) based on the data from the one or more vehicle sensors (120); anddetecting the moving target object (210) when the moving target object (210) exits the second vehicle (200). The method of claim 17, wherein detecting the moving target object (210) is performed only after determining that a door (240) of the second vehicle (200) is open. The method of claim 11, further comprising actuating a drone actuator to spray ink onto the moving target object (210). The method of claim 11, wherein tracking the moving target object (210) further includes detecting a particular ink on the moving target object (210).

Images