LOCATING A VEHICLE USING A DRONE

Abstract

System (10), comprising: a computer (12) comprising a transceiver (34), a processor (30), and a memory (32) storing instructions executable by the processor (30), the instructions comprising: Receiving image data of a vehicle parking area (20) from an unmanned aerial vehicle (UAV) (14) using the transceiver (34); Processing the data by identifying an actuation of a light source (84) of a UAV display device (26) that occurs within a predetermined time interval of a feedback response from a vehicle (16) in the area (20); and Determining a location of the vehicle (16) within the area (20) based on the identification; and the UAV (14), wherein the UAV (14) is programmed to record the image data and repeatedly emit a wireless command to trigger the feedback response of the vehicle (16) while moving within the area (20) from waypoint (48) to waypoint (48), wherein the light source (84) of the UAV display device (26) is located within a field of view of a sensor (22) on board the UAV (14) and the transmission of the wireless command occurs simultaneously with the actuation of the light source (84); wherein the command comprises a radio key code of the vehicle (16), and wherein the UAV (14) is programmed to operate in a factory mode that prevents the UAV (14) from following the rolling code.

Description

GENERAL STATE OF THE ARTLocating a particular vehicle in a large parking structure may require a person to move from vehicle to vehicle to identify the particular vehicle. In some cases, the person may examine the license plate of each vehicle to determine whether it is the searched vehicle. And when a large number of vehicles are in the parking structure, this can be a time consuming underrun.There are already various approaches to this in the prior art. The publication US 2009 / 0 015 401 A1, for example, discloses a system in which a sought vehicle recognizes a radio signal transmitted from a vehicle locating station by means of input, for example via a vehicle key, and amplified by a plurality of signal amplifiers across the parking space, the vehicle transmits a confirmation signal, for example flashing headlights or a horn signal, which is received by a plurality of detection units distributed to the parking space. The location is then determined based on the acknowledgment signal from the vehicle location station.U.S. Pat. No. 9,471,059 B1 discloses the use of an unmanned aircraft for locating a vehicle in a parking space, while U.S. Pat. No. 7,920,794 B1 discloses systems for optical communication in free space, in particular between a radio beacon, i.e. a stationary ground communication device, and an unmanned aircraft.BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 illustrates a vehicle localization system including a computer and an unmanned aerial vehicle (UAV). FIG. 2 schematically illustrates a top view of a vehicle parking area. FIG. 3 is a schematic illustration of an example vehicle that may be in the parking range. FIG. 4 is a schematic illustration of a sensor carried by the UAV. FIG. 5 is an example field of view of the sensor. FIG. 6 is a schematic illustration of another example of a sensor carried by the UAV.FIGS. 7-9 are flowcharts illustrating sets of instructions executable by the computer and / or the UAV.DETAILED DESCRIPTIONA vehicle localization system is described that includes a computer and an unmanned aerial vehicle (UAV). According to an illustrative example, the computer may include comprising a processor and a memory storing instructions executable by the processor, the instructions comprising: receiving image data of a vehicle parking area from a UAV; processing the data by identifying an actuation of a UAV display device that occurs within a predetermined time interval of a feedback response of a vehicle in the area; and determining a location of the vehicle within the area based on the identification.According to the at least one example set forth above, the response includes a light pattern emitted by the vehicle in response to a trigger from the UAV.According to the at least one example set forth above, the display device includes a light source located within a field of view of a sensor onboard the UAV.According to the at least one example set forth above, the system further includes the UAV and the UAV is programmed to record the image data and repeatedly emit a wireless command to trigger the response as it moves within the range of waypoint to waypoint.According to the at least one example set forth above, the command includes a keyfob code of the vehicle, wherein the UAV is programmed to operate in a factory mode that inhibits the UAV from following the rolling code.According to the at least one example set forth above, the system further includes a plurality of vehicles in the area including the vehicle providing the response, each of the plurality comprising an on-board computer programmed to execute the response when the UAV provides a wireless command corresponding to a keyfood code stored in the respective on-board computer.According to another illustrative example, an unmanned aerial vehicle (UAV) is disclosed, comprising: a computer comprising a processor and a memory storing instructions executable by the processor; a sensor; and a key fob transmitter, the instructions comprising: receiving image data of a vehicle parking area via the sensor; transmitting a wireless command via the transmitter; and receiving a visual feedback response from a vehicle in the area via the sensor and recording the response based on the transmission.According to the at least one example set forth above, the sensor is a digital camera.According to the at least one example set forth above, the image data is video data.According to the at least one example set forth above, the response includes a light pattern emitted by a headlamp of the vehicle.According to the at least one example set forth above, the UAV further includes a display device located within a field of view of the sensor.According to the at least one example set forth above, the display device is coupled adjacent an opening of the sensor.According to the at least one example set forth above, the display device comprises a light source, wherein the instructions further comprise: transmitting the command and simultaneously actuating the source.According to the at least one example set forth above, the instructions further include: repeatedly transmitting the command at a plurality of waypoints in the area, thereby recording image data, the data including the response and actuation of the display device; and subsequently transmitting the data to a second computer for video image processing, the processing including identifying the vehicle based on both the actuation and the response occurring within a predetermined time interval.According to the at least one example set forth above, the instructions further include: determining location data associated with the vehicle providing the response.According to the at least one example set forth above, the instructions further include: processing the image data at the UAV; determining location data of the vehicle based on the processing; and providing the location data to a second computer.According to the at least one example set forth above, a system includes the UAV and the second computer programmed to maintain a database of keyfood codes for a plurality of vehicles, wherein the command transmitted by the UAV includes one of the codes.According to another illustrative example, a method includes: receiving image data of a vehicle parking area from an unmanned aerial vehicle (UAV); processing the data by identifying actuation of a UAV display device that occurs within a predetermined time interval of a feedback response of a vehicle in the area; and determining a location of the vehicle based on the identification.According to the at least one example set forth above, the response is triggered by the UAV emitting a keyfob code towards the vehicle.According to the at least one example set forth above, the display device includes a light source located within a field of view of the image data, and the response includes a light pattern emitted by the vehicle.According to the at least one example, a computer programmed to execute any combination of the examples set forth above is disclosed.According to the at least one example, a computer programmed to perform any combination of the examples set forth above for the method / s is disclosed.According to the at least one example, a computer program product is disclosed that includes a computer readable medium storing instructions executable by a computer processor, the instructions including any combination of the instruction examples set forth above. According to the at least one example, a computer program product is disclosed that includes a computer readable medium storing instructions executable by a computer processor, the instructions including any combination of the examples set forth above for the method / s.Referring now to the Figures, wherein like reference numerals designate like parts throughout the several views, a vehicle localization system 10 is shown that includes a factory computer 12 and an unmanned aerial vehicle (UAV) 14. In vehicle factory environments, service personnel often need to locate or locate a specific vehicle 16 - e.g., for inspection purposes, software update purposes, etc. The system 10 may assist such personnel in locating the vehicle 16 among potentially hundreds or thousands of vehicles 18 - e.g., located in a vehicle parking area 20 of the factory. A vehicle plant may include a production or manufacturing facility, a storage facility for vehicles, a maintenance facility, or the like. Further, the following description is illustrated in the context of a vehicle plant; however, this is merely an example. Other suitable locations and applications, such as Value Park services or the like, are also possible.According to at least some examples, the UAV 14 may receive from the computer 12 a unique vehicle identifier, such as a key fob code for the specific vehicle 16 that may be located in the parking area 20. Thereafter, the UAV 14 may use an on-board sensor 22 to locate the vehicle 16-e.g., by capturing image data of the parking area 20 while re-transmitting the code through the area 20 (e.g., as it moves through a series of predetermined waypoints). In at least one example, UAV 14 may include an on-board display device 26 (such as a light source) that emits a signal each time code is emitted, and these emissions may be detected in image data (e.g., by locating the display device within a field-of-view (FOV) of sensor 22). When the specific vehicle 16 receives the key fob code (e.g., when the UAV 14 is in proximity thereto), the vehicle 16 may emit a response (such as blinking the headlights). At this time, the vehicle 16 may or may not be within a FOV of the UAV sensor 22; however, when the vehicle 16 is within the FOV, the UAV 14 may capture image data that includes both the signal (e.g., lighting of the light source) of the display device 26 and a response (e.g., flashing headlights) of the vehicle 16. Thus, the vehicle 16 may be partially located by identifying the signal and response within a predetermined time interval. As described below, in one example, the UAV 14 may have sufficient memory to store image processing algorithms (and / or software) and have sufficient processing speed and capacity to execute such algorithms in so-called real-time - thus, in one example, the UAV 14 may perform the identification. In other examples, the UAV 14 cannot perform this identification. Instead, the UAV 14 may provide the image data to the computer 12 and the computer 12 may determine the location of the vehicle 16 - e.g., by processing the image data, identifying the vehicle 16, and also determining location data for it.It will therefore be appreciated that once an identification of the signal and corresponding response is determined, that identification may be associated with an indication of where the identification has been made in the parking area 20 (e.g., so that service personnel may locate the vehicle 16). According to one example, the image data captured by the UAV 14 may be time stamped and the UAV may record a time stamped data log of its movements in the parking area 20; these two data sets may be correlated to determine location data for the vehicle 16. In one example, the UAV 14 moves according to the series of waypoints - e.g., it captures image data at each waypoints; thus, a time stamp (or time period) may also be stored by the UAV 14 for each waypoints. In other examples, other identifying information (e.g., markings on or near the parking area 20) may be captured in the image data-and the UAV 14 and / or the computer 12 may identify the location data of the vehicle 16 using this identifying information. Finally, the location data of the vehicle, when determined, may be provided to service personnel, who may then perform any suitable service on the vehicle 16.The computer 12 may include a single computer or multiple interconnected computing devices. For example, in at least one case, computer 12 is a production or manufacturing plant computer connected to an intranet of other computers, plant systems, and / or plant sub-systems. The computer 12 may include at least one processor 30 coupled to a memory 32. For example, processor 30 may be any type of device capable of processing electronic instructions, non-limiting examples of which include a microprocessor, microcontroller or controller, application specific integrated circuit (ASIC), etc., to name a few. In general, the computer 12 may be programmed to execute digitally stored instructions that may be stored in the memory 32 and may enable the computer 12 to, among other things: receive image data of the vehicle parking area 20 from the UAV 14; process the image data by identifying an actuation of the UAV display device 26 that occurs within a predetermined time interval of a response of a specific vehicle 16 that may be located within the area 20; and determine a location of the vehicle 16 based on the identification and the image data. These and other instructions are discussed in more detail below.The memory 32 may include any non-transitory computer usable or readable medium that may include one / n or more storage devices or articles. Exemplary non-transitory computer usable storage devices include random access memory (RAM), read only memory (ROM), EPROM (erasable programmable ROM), EEPROM (electrically erasable programmable ROM) from conventional computer systems, as well as any other volatile or non-volatile media. Non-volatile media includes, for example, optical or magnetic disks and other persistent storage. Volatile media includes dynamic random access memory (DRAM) which typically constitutes main memory. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip or cartridge, or any other medium from which a computer can read. As discussed above, one or more computer program products, which may be implemented as software, firmware, or the like, may be stored in the memory 32.The computer 12 may further include a transceiver 34 that enables communication between it and the UAV 14. The transceiver 34 may include a docking connector for mating with the UAV 14 (e.g., in wired implementations); however, in at least one example, the transceiver 34 enables wireless communication. For example, the transceiver 34 includes an antenna (not shown) and a short-range wireless chipset (not shown), according to one example; the chipset may enable communications over one or more protocols such as Wi-Fi, Wi-Fi Direct, Bluetooth, Bluetooth Low Energy (BLE), or the like. In other examples, transceiver 34 may enable long-range wireless communication (e.g., comprising an LTE, CDMA, GSM, etc. chipset or the like); other communication techniques (e.g., including satellite communication) are also possible. In at least one example, computer 12 communicates with UAV 14 via a BLE protocol-e.g., requiring less power of non-connected UAV 14.In at least one example, computer 12 also includes at least one database 36. database 36 may include any suitable memory or securely connected server that stores, among other things, unique vehicle identifiers for the plurality of vehicles 18 (e.g., including vehicle 16). According to one example, the database 36 maintains a record for each vehicle 18 and each record includes one or more identifiers. For example, each record may include a key fob code, a vehicle identification number (so-called VIN), or the like. In some examples, the dataset could also include an identifier of the license plate; however, it should be appreciated that such vehicles are typically not yet permitted (therefore, they do not have a license plate identifier) in a vehicle-plant environment.According to at least one example, the computer 12 may be located in a vehicle work including the parking area 20. Due to the dimensions of a typical vehicle structure, short-range wireless communication may be enabled via a series of wireless signal amplifiers 38 spaced apart from each other and disposed between the computer transceiver 34 and the parking area 20. Additionally, the parking area 20 itself may include a plurality of amplifiers 38 that are also spaced apart (e.g., around and within a perimeter of the area 20). The spacing of such amplifiers 38 may be based on an expected attenuation range of the wireless signals-e.g., based in part on the wireless protocol used. According to an additional example of location data, the UAV 14 may determine its location data in the parking area 20 (and thus that of the identified vehicle 16) using triangulation techniques and the signals from the amplifiers 38.The vehicle parking area 20 may be graveled or asphalted parking spaces (e.g., overroofed or not overroofed), a parking garage, or the like. Additionally, in at least some examples, the area 20 includes a plurality of location identifiers 40. non-limiting examples include alphanumeric markings painted or adhered to a facing 42 of the area 20, labels with alphanumeric markings, unique illumination, or other wireless signals indicating an area, etc. In FIG. 2, the identifiers 40 include row and column markings on the facing 42 (e.g., rows AA, BB, CC, DD, etc., and columns 1, 2,..., 16,...); of course, this is merely one example. Other suitable identifiers 40 include a label and / or unique marking of each parking space 44 in the area 20.As will be described in more detail below, the region 20 may be graphically depicted using a plurality of waypoints 48. Each waypoints 48 may have a unique set of GPS coordinates (e.g., with 5-7 decimal point precision) and / or Internet Protocol version 6(IPV6) addresses. Further, the distances and locations of the waypoints 48 may be associated with a field of view (FOV) of the sensor 22 on the UAV 14 (e.g., which space it can capture at a respective time), correspond to groups 50 of parking spaces 44 (or vehicles 18), correspond to a combination of both, or the like.The vehicles 18 (including the specific vehicle 16) are each shown as passenger cars; however, the vehicles 18 could also be trucks, SUVs, recreational vehicles, buses, train cars, watercraft, aircraft, or the like. According to one example (see FIG. 3 ), each vehicle 18 may include an electronic control module 52 (such as a so-called body control module) that responds to wireless commands (including a key fob code) from a transmitter of a keyless entry device (such as a vehicle key fob). For example, module 52 may include a computer including one or more processors, memory, and a wireless receiver - wherein the receiver responds to commands from an associated key fob regardless of whether the vehicle ignition is ON or OFF. In this way, when a user of the vehicle 18 presses a switch on the key fob, the module 52 may receive the command, verify the key fob code, and perform a vehicle function (also named in the command) -- such as locking or unlocking vehicle doors, starting vehicle ignition, actuating a panic function (e.g., triggering the vehicle horn and / or flashing the vehicle lights), etc. In some examples of performing the vehicle function, the module provides a feedback response to the user (e.g., so that the user knows that the function has been performed). For example, when the module 52 receives a command to lock the doors, the module 52 may both actuate the vehicle door latches 54 and blink vehicle lights 56, trigger a vehicle horn 58, and / or the like. In this way, the user-who may be far-away-knows that the vehicle 18 has executed the wireless command. In at least one example, at least with some wireless commands, headlights 56 emit a predetermined light pattern. The light pattern may include one or more blinks, each blink having a predetermined turn-on duration (and when multiple blinks are used, each non-blink duration between blinks may have a predetermined turn-off duration). As will be discussed in more detail below, the UAV 14 may broadcast a wireless command that includes the key fob code, and a feedback response (such as the light pattern) may be received by the UAV 14 and used to identify the specific vehicle 16 among other vehicles 18.FIGS. 1 and 4-6 illustrate aspects of unmanned aerial vehicle (UAV) 14 (e.g., a so-called drone). The UAV 14 may be any suitable aircraft that is at least partially, if not fully, operated and controlled by a computer 70 onboard the UAV 14 itself, which enables the UAV to operate in a fully autonomous mode in which no human intervention or control is required. In at least one example, UAV 14 may float using one or more propeller assemblies 72; this is merely an example (and other UAV flight techniques may be employed instead or additionally).Generally, the UAV 14 includes the computer 70, the sensor 22, a wireless transceiver 74, and a key fob transmitter 76. the computer 70 includes at least one processor 78, and a memory 80 storing instructions executable by the at least one processor 78. Processor 78 may be similar to or identical to processor 30 (described above with respect to computer 12) and memory 80 may be similar to or identical to memory 32 (also described above with respect to computer 12); therefore, this hardware will not be re-described herein. However, the memory 80 stores instructions different from those stored in the memory 32; the instructions stored in the memory 80 may be executed by the processor 78 and specifically configure the computer 70 to perform different tasks. For example, the computer 70 may be programmed with instructions including, but not limited to: receiving and / or storing a key fob code from the computer 12; controlling UAV flight; controlling UAV steering; controlling UAV stability; controlling UAV navigation; moving the UAV 14 through a series of waypoints 48; receiving and / or recording image data of the vehicle parking area 20 via the sensor 22; storing the image data in the memory 80; transmitting a video stream of the image data to the computer 12; providing a recorded video clip of the vehicles 18 as the UAV 14 moves in the parking area 20; transmitting a wireless command (including a key fob code) to the vehicles 18 via the transmitter 76 in an effort to identify (and repeat this transmission); preventing the computer 70 from following the rolling or skip key fob code; receiving and / or recording a response from the vehicle 16 (via the sensor 22); actuating the display device 26 at the UAV 14 when the transmitter 76 transmits the command; processing image data using image processing algorithms, instructions, and / or any other suitable technique; and executing any combination of the example instructions listed herein.According to one example, image processing is performed by the processor 78 and may include: analyzing multiple video images of the image data; determining which images include both the response of the vehicle 16 (e.g., pattern light emission of the headlights 56) and a display of the display device 26 (e.g., lighting); identifying the vehicle 16 based on the response and display occurring within a predetermined time interval therebetween; and determining location data of the vehicle 16 using any suitable technique. As will be discussed below, determining the location data may include storing GPS coordinates of the vehicle 16 (once the display and corresponding response have been identified), triangulating the position using wireless signals from the amplifiers 38, and / or using location identifiers 40 in or near the parking area 20. The image processing instructions may also be executable individually or in any suitable combination with each other. In addition, all instructions discussed above are merely examples; other instructions may also be stored by the memory 80 and executed by the processor 78.As used herein, the sensor 22 on board the UAV 14 includes an input device configured to detect a feedback response generated by the vehicle 16 when the key fob transmitter 76 sends a wireless command (including a matching key fob). For example, the sensor 22 may include a digital camera (e.g., a complementary metal oxide semiconductor (CMOS) device, a charge-coupled device (CCD), or the like), an image intensifier (a so-called I-square device), or the like, just to name a few non-limiting examples. The sensor 22 may include an aperture 82 (e.g., in which a cover or lens is typically disposed) spaced from a detector or image plane (not shown) - the aperture 82 letting light fall on the detector.The display device 26 may be any suitable device that, when actuated by the computer 70, provides an indication of its actuation. In at least one example, the display is a visual display. For example (FIG. 4 ), the display device 26 may include a light source 84 that is within a field of view (FOV) of the detector and lights upon initiation by the computer 70. The light source 84 may be any suitable light emitting device (such as a light emitting diode or LED). In at least one example, display device 26 includes an arm 86 coupled to a body 88 of sensor 22, arm 86 being disposed adjacent opening 82 such that light from light source 84 is detected in the FOV of the detector.There are other examples of display devices. For example, Figure 6 illustrates a sensor 22' having a body 88', an aperture 82', and a display 26'. Here, the display device 26' is a reticle. For example, the reticle may be illuminated when triggered by the computer 70. In another display device example, one or more predefined pixels or pixel arrays could be incorporated into the video data by the computer 70, and the presence of these pixels in the images of the video data could indicate the transmission of the keyfob code. There are still further examples.The wireless transceiver 74 may enable communication between the UAV 14 and the computer 12, among other things. The transceiver 74 may include an antenna (not shown) and a short-range wireless chipset (not shown); the chipset may enable communications via any suitable protocol such as Wi-Fi, Wi-Fi Direct, Bluetooth, Bluetooth Low Energy (BLE), or the like. In other examples, transceiver 74 may enable long-range wireless communication (e.g., comprising an LTE, CDMA, GSM, etc. chipset or the like); other communication techniques (e.g., including satellite communication) are also possible. Transceiver 34 (computer 12) and transceiver 74 (UAV 14) may be paired-e.g., to enable communication therebetween. And in at least one example, wireless transceiver 74 includes a BLE chipset and uses a BLE protocol.The key transmitter 76 of the UAV 14 may include a microprocessor (not shown) coupled to a wireless chipset (not shown) operating in a frequency range suitable for performing one or more remote keyless entry functions (in one example, at a frequency of about 315 MHz). Unlike hand held radio keys, transmitter 76 may not be coupled to user actuatable pushbuttons (but instead may be actuated by computer 70 onboard the UAV). For example, the transmitter chipset may transmit a wireless command to the vehicles 18 when the processor 78 actuates the transmitter 76. As discussed above, the command may include an instruction to perform a vehicle function coupled to a keyfob code. When the associated vehicle 16 receives the command (and verifies the code), the vehicle 16 may perform the command-similar to when the vehicle 16 receives the command from a handheld wireless key. Continuing with the above example, when the vehicle 16 receives at least one command, the vehicle door latches may be actuated (to a locked state) and the headlights 56 may flash.When traditional handheld radio keys are actuated by a user pressing a push button, the microprocessor in the radio key typically uses a rolling code (or skip code) to avoid replay attacks. One skilled in the art will understand that replay attacks include a malicious listener recording and playing back the transmitted code coupled with an instruction to unlock the doors, start vehicle ignition, etc. (e.g., allowing the listener to enter the vehicle and / or drive away with). To avoid this, hand-held radio keys may be programmed with a rolling code - i.e. a next radio key code is in each case entered in a sequence (as calculated, for example, by an algorithm executed by the microprocessor of the radio key). Thus, when the key fob is re-actuated (by a user pressing a button), the next wireless command uses a different code in a sequence determined by the algorithm. The corresponding vehicle (e.g., in particular, the computer 52) operating according to the handheld wireless key is generally configured using the same algorithm. Thus, when the computer 52 receives the command with another code, the computer 52 is programmed to determine whether the new code corresponds to one of a predetermined number (e.g., 256) of other codes that can be determined using the sequence-determining algorithm. For example, another code may be received at the computer 52 when the keys of the hand held key have been pressed one or more times but the wireless command has not reached the computer 52 (e.g., the key code will transition next each time a conventional key key is pressed - e.g., resulting in a mismatch later). When a match is detected (by computer 52) with any of the predetermined number of other codes, computer 52 updates its memory (according to the last key code used by the hand held key) and also executes the instruction sent by the hand held key. Basically, the computer 52 synchronizes the most recently used key code in its memory. (If no match is found, the hand held radio key is no longer coupled to the computer 52 - e.g., authorized maintenance personnel must perform a special re-pairing process for the hand held radio key to be re-operable).In the present case, the key transmitter 76 may be programmed to operate in a factory mode, where the transmitter 76 does not switch to the next key code each time it sends a wireless command - e.g., while searching the parking area 20 in the factory mode. Thus, because the parking area 20 may be extremely large (and as will be described in more detail below), the UAV 14 may repeatedly broadcast the code outside of a radio range of the specific vehicle 16 without transitioning more than a predetermined number of times to the next key code. In at least one example, this predetermined number may be zero. In this way, the UAV 14 (and its transmitter 76) may not undesirably disable the transmitter 76 -- and may be further capable of receiving the desired feedback response from the specific vehicle 16.In addition to the components listed above, the UAV 14 may further include a positioning unit 94, which may include any suitable computing device that may provide global and / or local location data to the computer 70. For example, the positioning unit 94 may receive location data (such as latitude / longitude (LAT / LONG) coordinate data) from one or more satellites. Non-limiting satellite types and location systems include GPS and GLONASS. Further, the UAV 14 may move through the parking area 20 using the unit 94 according to a predetermined flight plan - the flight plan including a series of predetermined waypoints 48 and pause times at one or more of the waypoints. For example, according to a flight plan, the pause times at the waypoints 48 may each be similar or identical - e.g., allow sufficient time for each of the target vehicles 18 to provide a feedback response within a current field of view of the UAV sensor 22.Referring now to FIGS. 7-9, these flowcharts illustrate processes 700, 800, 900 that may be executed by one or more computing devices of the vehicle localization system 10. These processes are intended to illustrate uses and applications of the system 10 and are not intended to be limited to the examples set forth herein.The process 700 may begin at block 705 where the UAV 14 receives a unique identifier from the work computer 12 (e.g., via the transceiver 74). For example, a service technician or computer 12 may determine by himself that a specific vehicle 16 from a plurality of vehicles 18 needs to be serviced, and vehicle 16 may be located at an unknown location in parking area 20. For illustrative purposes only, the technician or computer 12 may determine that the vehicle 16 needs the latest update telematics software. The identifier provided to the UAV 14, wired or wireless, by the computer 12 may be a keyfob code of the specific vehicle 16, or may include such keyfob code that the UAV 14 may use to locate the vehicle.At block 710, the UAV 14 may navigate to or move to a first (or next) path point 48 in the parking area 20. For example, in at least one example, computer 70 on board UAV 14 may be programmed to navigate along a predetermined series of waypoints 48 using positioning unit 94. Illustratively, these waypoints 48 may include a loop from a docking station (not shown) to the parking area 20 through a series of spaced waypoints 48 in the parking area (e.g., to search for the vehicle 16) and a return path from one or more waypoints back to the docking station. (Later in process 700, block 710 may be repeated and block 710 then proceeds to the "next" path point 48 in the predetermined row.) At block 715, at the path point of block 710, the UAV 14 may instruct the key transmitter 76 to emit a wireless command that may include the identifier (e.g., key code) received from the computer 12. This command may be sent towards a plurality of vehicles 18 in the parking area 20, e.g., at the current waypoint 48, to trigger a feedback response of the sought vehicle 16. According to one example, multiple vehicles 18 may receive the keyfood code and a vehicle instruction (and each vehicle 18 receiving the code may use its on-board computer 52 to determine whether the code matches its stored keyfood code). According to an example set forth above, the vehicle instruction may be configured to actuate the latches of the vehicle doors to the locked state (e.g., even if the latches on the vehicle 16 are already in the locked state). Based on the programming of the vehicle 16, the UAV 14 may assume that the vehicle 16 - when receiving the wireless command - not only executes the instruction (based on the key fob code) but also provides a feedback response, such as blinking the vehicle headlights 56 (e.g., once or twice).At block 720, the UAV 14 may hover over the respective waypoints 48 while receiving image data from the sensor 22 (e.g., image data including all or a portion of the portion of the parking area 20 to which the wireless command is directed). In at least one example, this image data is video data or a stream of video data (e.g., multiple consecutive images). Block 720 may occur at least partially simultaneously with block 715. In some cases, once the UAV 14 reaches the first waypoints 48 in the parking area 20, it may receive, record, and / or store image data (e.g., until the entire parking area 20 is mapped or until the vehicle 16 is identified). Thus, in some examples, image data may be received at least shortly before block 715. In other examples, sections of video data are captured at each waypoints 48 (e.g., to conserve storage space onboard the UAV 14).At block 725, the UAV 14 may execute an on-board trigger via the computer 70. According to one example, the trigger actuates the display device 26; and in at least one example, the trigger actuates the light source 84, thereby lighting it. As described above, the light from the source 84 may be within the field of view of the sensor 22; thus, the image data may include lighting. This lighting may be time limited - e.g., have a predetermined duration (e.g., one or two seconds). Block 725 may also be executed at least partially simultaneously with block 715 and block 720. In some cases, the execution of block 725 may be delayed relative to the execution of blocks 715 and 720; however, this delay may be less than one second (e.g., even less than 500 milliseconds in some examples). The use of the display device 26 (e.g., implemented as actuatable light source 84) may reduce the need for complex software to correlate the transmission of a keystream transmission (block 715) with the video data that may be recorded by the UAV computer 70.Other examples of block 725 include an on-board trigger that causes an internal actuation, such as actuation of an electronic circuit, switch, counter, or the like (e.g., not captured by the image data of sensor 22). In at least one example of this type, the trigger is execution of the wireless command. This implementation may be appropriate when the UAV 14 performs onboard image processing to determine the feedback response from the vehicle 16; in these cases, the UAV 14 may be able to correlate a feedback response from the vehicle 16 with its own internal actuation. In other cases (e.g., where computer 12 analyzes the image data instead of UAV computer 70), a visual actuation detected in the image data may be desirable. For illustrative purposes only, the remainder of the process 700 describes the display device 26 as being external to the sensor 22 (e.g., as the light source 84 described above).At block 730-following execution of blocks 715-725, the computer 70 determines whether a feedback response has been received from at least a portion of the plurality of vehicles 18 that received the wireless command at block 715. For example, in some cases, block 730 may only refer to vehicles 18 within a field of view of sensor 22 (e.g., at the time of execution of blocks 715- 725). When a feedback response (such as flashing of the vehicle headlights 56) is determined in response to the broadcast of the wireless command (e.g., having the vehicle 16 key code), the UAV 14 may determine that it has most likely found the specific vehicle 16. However, further identification of the vehicle 16 may be required and an example of this further identification is described below at block 735 (which may be made subsequently).According to at least one example, process 700 returns and repeats blocks 715, 720, and 725 to result in re-execution of these instructions to a second, similar feedback response from the vehicle. In this case, when the headlights 56 of the same vehicle flash again as a result of the second attempt, the UAV 14 may execute the process 800 to further identify whether it has located the vehicle 16.At block 730, the process 700 may return to block 710 and proceed to the next path point 48 if no feedback response has been received (or if no second feedback response has been received during the attempt to confirm upon re-execution of the blocks 715, 720, 730). Subsequently, the process 700 may proceed as already described above.At block 735, the UAV computer 70 may identify the vehicle that provided the feedback response -- e.g., confirming that the flashing of the headlights was not an accident. In at least one example, this identification may include additional analysis of image data received from sensor 22, analysis of timing of feedback response from vehicle as compared to timing of wireless command, and the like. An example of a set of instructions that may be executed to identify the vehicle 16 is described in more detail below in process 800.At block 740, which occurs subsequently, the computer 70 may identify location data for the identified vehicle 16. For example, the computer 70 may determine where the vehicle is in the parking area 20. According to one example, using the transceiver 74, the UAV 14 may triangulate the position of the vehicle 16 relative to a plurality of wireless amplifiers 38 having known locations in the parking area 20. Or, the UAV 14 may execute optical processing algorithms for identifying optical indicia using the computer 70 that enable the UAV 14 to determine location data of the vehicle 16 using location identifiers 40 in the parking area 20 (such as signs, markings on the pavement 42 of the parking area 20, and the like).In other examples, the UAV 14 may simply record and download location data to the computer 12, which then performs the image processing (and ultimately determines the location of the vehicle 16). For example, the UAV 14 may transmit this data to the computer 12 as it travels through the waypoints 48 in the parking area 20, or may store the image data (which may or may not include this location data) in the memory 80 and provide it to the computer 12 later.Continuing with the example where the UAV 14 identifies the vehicle 16 and its respective location data, at block 745, the UAV 14 may provide a message to the computer 12 (e.g., using the transceiver 74) indicating that it has found the vehicle 16. This message may also include the location data. The service technician can then pick up the vehicle 16 and update the telematics software contained therein.After block 745, process 700 may include UAV 14 determining that parking area 20 is to be searched for another vehicle (e.g., a similar vehicle 16) (see block 750). If, at block 750, the UAV 14 has instructions to search for or receive another vehicle, the process 700 returns to block 710 and repeats the respective instructions as described above. Otherwise, the process 700 may proceed to block 755, where the UAV 14 returns to its docking station and the process ends.Referring now to FIG. 8, the process 800 further identifies the specific vehicle 16 using the image data obtained from the UAV 16 and attempts to remove potential false positive determinations. Recall that the image data contained a feedback response (such as blinking vehicle headlights 56) prior to process 800. Of course, vehicle headlights could blink for other reasons and process 800 attempts to verify that the blinking of the headlights or other response was not accidental.As discussed above, process 800 could be performed using so-called real-time image processing techniques at UAV 14 (e.g., as part of block 730 and / or 735). Or, the UAV 14 may execute process 800, but not in real-time. Or, the process 800 may be executed by the computer 12, e.g., once the UAV 14 provides recorded image data to it. For illustrative purposes only, and not by way of limitation, process 800 is described with respect to UAV 14 executing the instruction blocks.The process 800 may begin at block 810, which includes determining a first time stamp (t 1) of actuation of the on-board trigger (block 725). Recall that this trigger may be actuated simultaneously with or within a predetermined time period after the transmission of the wireless command (block 715). Again for illustrative purposes only, the operation will be described as illuminating the light source 84 of the display device 26. Thus, the time stamp t 1 may correspond to a time at which the image data captures the light source 84 in an illuminated state (and this illuminated state could indicate in the video data when the UAV 14 has sent a wireless command).At block 820, the computer 70 may determine a second time stamp (t 2) corresponding to the time of detecting a vehicle headlight illumination event (the feedback response) in the image data. Using time stamps t1and t2(at block 830), the computer 70 may determine a time interval (Δt), where Δt=|t1-t2|. As will be discussed in more detail below, the time interval (Δt) between the trigger actuation and the headlight illumination may be correlated with the transmission of the wireless command by the key fob transmitter 76. For example, the computer 70 may determine that there is a correlation and there is a higher likelihood that headlight illumination was a result of the wireless command (and not, e.g., a random appearance) when the time interval (Δt) is less than a predetermined time threshold (T 1).At block 840, the computer 70 may also determine a duration of turn-on of the headlights (t HL_ON). The turn-on duration may be measured from the time when the headlights emit light to the time when they stop emitting light. For example, this switch-on duration can be less than 1 or 2 seconds in the case of a flashing of the headlights. In addition, each duration may be measured when the image data includes blinking the vehicle a plurality of times. As discussed below, the vehicle door lock on duration(s) may be an indication of the sought vehicle 16. For example, the vehicles 18 of some trademarks and models may have different durations of turn, different blinking or lighting patterns, etc. On the other hand, if the image data includes headlights 56 being turned on but remaining turned on (e.g., for any suitable reason, such as the vehicle is driving straight away), parameter t HL_ON may be used to indicate a lack of correlation with the sought vehicle 16, as discussed below.At block 850, which occurs subsequently, the time interval (Δt) may be compared to the first predetermined threshold (T 1) and the turn-on duration of the headlights (t HL_ON) may be compared to a second predetermined threshold (T 2). According to one example, the process 800 proceeds to block 860, where the computer 70 determines that the specific vehicle 16 has been identified (e.g., with a high probability of a correct determination) when Δt<T 1 (i.e., when the time between the wireless command and the headlight flash is less than a threshold) and t HL_ON< T 2 (i.e., the time during which the headlights were actually turned on is less than a threshold). On the other hand, if the computer 70 determines at block 850 that Δt is not less than T1or t HL_ON is not less than T2, the process 800 proceeds to block 870, where the computer 70 determines that the specific vehicle 16 has not been identified.In the context where the computer 70 on board the UAV 14 executes the process 800, this process may be repeated each time the UAV 14 detects headlight light emissions within its field of view. In this way, processing power and capabilities on board the UAV 14 are saved, e.g., since the UAV does not execute this instruction set at each wireless transmission.In the context where the computer 12 executes the process 800, the computer 12 may parse the image data and identify segments of image data that include headlight light emissions, and then identify the vehicle 16 using the interval Δt, duration t HL_ON, threshold T 1, threshold T 2, etc., as well as other correlation data provided by the UAV 14. In this manner, the computer 12 also saves computational capabilities by not executing the process 800 for each waypoints 48 in the parking area 20.Referring now to process 900, a process is illustrated in which the UAV 14 captures image data of the parking area 20, e.g., from viewpoint to viewpoint 48, and provides the image data to the factory computer 12 for image processing used to identify the vehicle 16 and its location data. As discussed below, in at least one example, the image data may be provided to computer 12 after UAV 14 has traversed all waypoints 48 of parking area 20.The process 900 begins at blocks 905, 910, 915, 920, and 925. These blocks may be identical or similar to blocks 705, 710, 715, 720, and 725 described above, respectively (process 700), therefore, they will not be described again herein.At block 970, which occurs subsequent to block 925, the UAV 14 may determine whether it has executed blocks 910- 925 for each of the waypoints in a predetermined or preprogrammed series of waypoints 48 associated with the parking area 20 (e.g., has it captured and received image data at all waypoints 48). If not, process 900 may jump back and repeat blocks 910- 925 (e.g., for each waypoints 48). If all waypoints 48 in the parking area 20 have been detected, the process 900 may proceed to block 975.At block 975, the UAV 14 provides the image data to the computer 12. As discussed above, this may be via wired transmission (e.g., via the docking station) or via wireless transmission (e.g., using transceiver 74). Moreover, it may be via streaming data, via a collection of fragmentary transmissions, or via a single download or transmission. Regardless of the type of transmission, the process proceeds to block 980 after block 975.Block 980 may be similar to block 730. At block 730, the image data of the parking area 20 was fragmentarily parsed. Here, in at least one example, all image data may be available to the computer 12. Thus, in at least one example of block 980, the computer 12 parses the image data to find possible feedback responses from one or more vehicles 18 in the parking area 20. Continuing with the example set forth above, this may include parsing the video data recorded by the UAV sensor 22 to find moments with headlight light emissions.After these moments are determined, the process 900 may proceed to block 985 and attempt to identify the vehicle 16 from the video data. According to at least one example of block 985, this may include executing process 800 discussed above. Therefore, the instructions that may be executed to determine the identification are not discussed again herein.At block 990, which occurs subsequently, after identifying the vehicle 16, the computer 12 may identify location data associated with the particular vehicle 16. This instruction may be similar or identical to block 740; therefore, it is not repeated here either.After the vehicle 16 and its location data are identified, the location data may be provided to the service technician, who may then pick up the vehicle 16 as described above. After block 990, process 900 may execute block 995, which may be similar or identical to block 750 (e.g., whether another vehicle is to be located). If another vehicle is to be located, the process 900 may proceed to block 905 where the computer 12 may provide (and the UAV 14 may receive) another unique identifier. Otherwise, the process 900 ends.There are other examples of the vehicle 16 as well, for example, the computer 52 may be programmed to respond to key-like commands, e.g., using similar protocols and / or similar frequencies. In at least one example, computer 52 may be programmed with special instructions that do not require that the next key fob code always be transitioned to at UAV 14 or vehicle 16.Thus, a vehicle localization system has been described that includes a computer and an unmanned aerial vehicle (UAV). The computer and / or the UAV are programmed to receive and / or analyze image data of a vehicle parking area, the image data including a feedback response from a specific vehicle based on a wireless command sent by the UAV. The wireless command includes a unique identifier that causes a response of the specific vehicle.In general, the computing systems and / or devices described may employ any of a number of computer operating systems, including, but not limited to, versions and / or varieties of the SYNC® application of Ford, AppLink / Smart Device Link Middleware, the operating systems Microsoft® Automotive, Microsoft Windows® Unix (e.g., the operating system Solaris® sold by Oracle Corporation of Redwood Shores, California), AIX UNIX sold by International Business Machines of Armonk, New York, Linux, Mac OSX, and iOS sold by Apple Inc. of Cupertino, California, BlackBerry OS, sold by Blackberry Ltd. of Waterloo, Canada, and Android, developed by Google Inc. and the Open Handset Alliance, or the platform QNX® CAR for Infotainment, offered by QNX Software Systems. Examples of computing devices include, but are not limited to, an on-board vehicle computer, a workstation computer, a server, a desktop, notebook, laptop, or handheld computer, or other computing system and / or device.Computing devices generally include computer-executable instructions, wherein the instructions may be executed by one or more computing devices such as those listed above. Computer-executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages and / or technologies, including, without limitation, and either alone or in combination, Java™ C, C++, Visual Basic, Java Script, Perl, etc. Some of these applications may be compiled and executed on a virtual machine, such as the Java Virtual Machine, the Dalvik Virtual Machine, or the like. Generally, a processor (e.g., a microprocessor) receives instructions, e.g., from a memory, a computer readable medium, etc., and executes these instructions, thereby performing one or more processes, including one or more of the processes described herein. Such instructions and other data may be stored and transmitted using a variety of computer readable media.A computer-readable medium (also referred to as a processor-readable medium) includes any non-transitory (e.g., tangible) medium that participates in providing data (e.g., instructions) that may be read by a computer (e.g., by a processor of a computer). Such a medium may take many forms, including, but not limited to, non-volatile media and volatile media. Non-volatile media may include, for example, optical and magnetic disks and other persistent storage. Volatile media may include, for example, dynamic random access memory (DRAM), which typically represents main memory. Such instructions may be transmitted by one or more transmission media including coaxial cables, copper wire, and fiber optics, including the wires comprising a system bus coupled to a processor of a computer. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip or cartridge, or any other medium from which a computer can read.Databases, data records, or other data stores described herein may include various types of mechanisms for storing, accessing, and retrieving various types of data, including a hierarchical database, a set of files in a file system, an application database in a native format, a relational database management system (RDBMS), etc. Each of these data stores is generally included within a computing device that uses a computer operating system, such as one of those mentioned above, and is accessed via a network in one or more of a variety of ways. A file system may be accessed by a computer operating system and may include files stored in various formats. An RDBMS generally employs the computer language structured query language (SQL) in addition to a language for creating, storing, editing, and executing stored procedures, such as the PL / SQL language mentioned above.In some examples, system elements may be implemented as computer readable instructions (e.g., software) on one or more computing devices (e.g., servers, personal computers, etc.) stored on computer readable storage media in association therewith (e.g., disks, memories, etc.). A computer program product may include such instructions stored on computer readable media for performing the functions described herein.The processor is implemented via circuits, chips, or other electronic component and may include one or more microcontrollers, one or more field programmable gate arrays (FPGA), one or more application specific circuits (ASIC), one or more digital signal processors (DSP), one or more custom integrated circuits, etc. The processor may be programmed to process sensor data. Processing the data may include processing the video input or other data stream detected by the sensors to determine the roadway lane of the host vehicle and the presence of any target vehicles. As described below, the processor instructs vehicle components to be actuated according to the sensor data. The processor may be incorporated into a controller, e.g., an autonomous mode controller.The memory (or data storage device) is implemented via circuits, chips, or other electronic components and may include one or more of read only memory (ROM), random access memory (RAM), flash memory, electrically programmable read only memory (EPROM), electrically programmable and erasable read only memory (EEPROM), embedded multimedia cards (eMMC), a hard disk, or any volatile or non-volatile media, etc. The memory may store data collected by the sensors.The disclosure has been described in an illustrative manner, and it is to be understood that the terminology used is intended to be in the nature of description rather than of limitation. Many modifications and variations of the present disclosure are possible in light of the above teachings, and the disclosure may be practiced otherwise than as specifically described.

Claims

A system (10) comprising: a computer (12) comprising a transceiver (34), a processor (30), and a memory (32) storing instructions executable by the processor (30), the instructions comprising: receiving image data of a vehicle parking area (20) from an unmanned aerial vehicle (UAV) (14) using the transceiver (34); processing the data by identifying actuation of a light source (84) of a UAV display device (26) occurring within a predetermined time interval of a feedback response of a vehicle (16) in the area (20); and determining a location of the vehicle (16) within the area (20) based on the identification; The UAV (14), wherein the UAV (14) is programmed to record the image data and to repeatedly emit a wireless command to trigger the feedback response of the vehicle (16) as it moves within the range (20) from the viewpoint (48) to the viewpoint (48), wherein the light source (84) of the UAV display device (26) is within a field of view of a sensor (22) onboard the UAV (14), and transmission of the wireless command occurs simultaneously with actuation of the light source (84); wherein the command comprises a key fob code of the vehicle (16), and wherein the UAV (14) is programmed for operation in a factory mode that inhibits the UAV (14) from following the scroll code.The system of claim 1, wherein the response comprises a light pattern emitted by the vehicle (16) in response to a trigger from the UAV (14).The system of claim 1, further comprising a plurality of vehicles (18) in the area (20) including the vehicle (16) providing the response, each of the plurality of vehicles (18) comprising an on-board computer (52) programmed to execute the response when the UAV (14) provides a wireless command corresponding to a key code stored in the respective on-board computer (52).An unmanned aerial vehicle (UAV) (14) comprising: a computer (70) comprising a processor (78) and a memory (80) storing instructions executable by the processor (78); a sensor (22); and a key fob transmitter (76), a display device (26) located within a field of view of the sensor (22), the display device (26) comprising a light source (84); wherein the instructions comprise: receiving image data of a vehicle parking area (20) via the sensor (22); transmitting a wireless command via the transmitter (76), and simultaneously actuating the light source (84); and receiving a visual feedback response from a vehicle (16) in the area via the sensor (22) and recording the response based on the transmission; repeatedly transmitting the command at a plurality of waypoints (48) in the area (20), thereby recording image data, the data comprising the response and actuation of the display device (26); and subsequently transmitting the data to an external second computer (12) for video image processing, the processing including identifying the vehicle (16) based on both the actuation and the response occurring within a predetermined time interval.The UAV (14) of claim 4, wherein the sensor (22) is a digital camera.The UAV (14) of claim 4, wherein the image data is video data.The UAV (14) of claim 4, wherein the response comprises a light pattern emitted by a headlamp (56) of the vehicle (16).The UAV (14) of claim 4, wherein the display device (26) is coupled adjacent an opening (82) of the sensor (22).The UAV (14) of claim 4, wherein the instructions further comprise: determining location data associated with the vehicle (16) providing the response.The UAV (14) of claim 4, wherein the instructions further comprise: processing the image data at the UAV (14); determining location data of the vehicle (16) based on the processing; and providing the location data to the external second computer (12) for provision to a user, wherein the processing includes identifying the vehicle (16) based on both the actuation and the response occurring within a predetermined time interval.The UAV (14) of claim 10, wherein the second computer (12) comprises a transceiver (34) for communicating with the UAV (14) and is programmed to maintain a database (36) of keyfood codes for a plurality of vehicles (16, 18) and provide the UAV (14) with a keyfood code associated with a searched vehicle (16), wherein the command transmitted by the UAV (14) includes one of the codes.A method comprising: receiving image data of a vehicle parking area (20) from an unmanned aerial vehicle (UAV) (14); processing the data by identifying actuation of a UAV display device (26) occurring within a predetermined time interval of a feedback response of a vehicle (16) in the area (20); and determining a location of the vehicle (16) based on the identification; wherein the UAV (14) records the image data and repeatedly emits a wireless command to trigger the response as it moves within the range (20) from the viewpoint (48) to the viewpoint (48), the command comprising a key fob code of the vehicle (16), the UAV (14) being programmed to operate in a factory mode that inhibits the UAV (14) from following the scroll code.The method of claim 12, wherein the response is triggered by the UAV (14) emitting a keyfob code toward the vehicle (16).The method of claim 12, wherein the display device (26) comprises a light source (84) located within a field of view of the image data, and the response comprises a light pattern emitted by the vehicle (16).

Images