Using an unmanned aerial vehicle to test autonomous vehicles

Abstract

Procedure, wherein the procedure comprises the following: Recognizing the termination of a paid journey offered to one or more occupants (236); Determine that an autonomous vehicle (201) should be inspected according to inspection rules; Releasing an unmanned aerial vehicle (222) from a protected area (211) within the autonomous vehicle (201) for the purpose of checking the autonomous vehicle (201); Receiving images (271, 272) of one or more of: vehicle interior components (233) and vehicle exterior components (213) of the autonomous vehicle (201) from the unmanned aerial vehicle (222) that is checking the autonomous vehicle (201); Forwarding the images (271, 272) for image analysis; and Receiving instructions (273) that instruct the autonomous vehicle (201) on how to proceed based on the analysis of the images (271, 272).

Description

GENERAL STATE OF THE ART1. Field of the invention

[0001] This invention relates generally to the field of vehicle inspection and in particular to the use of unmanned aerial vehicles (UAVs) for inspecting autonomous vehicles. 2. Related technology

[0002] Autonomous vehicles have no driver. Therefore, if an autonomous vehicle is used to provide paid rides for third parties, it can be difficult to check the vehicle for cleanliness, maintenance issues, and so on between trips. Cameras can be mounted on an autonomous vehicle to inspect certain areas between trips. However, over time, camera lenses can become damaged or covered with dirt, limiting their usefulness. Dirt and grime are significant problems for externally mounted cameras, especially in positions exposed to harsh weather. For example, it can be difficult to keep chassis cameras free of dirt and snow in locations exposed to winter weather.

[0003] Systems and methods for automatic vehicle imaging are known from US patent 2016 / 0 173 740 A1. Exemplary embodiments may include cameras configured to capture images and / or videos. The cameras may be coupled to booms and / or unmanned vehicles, such as unmanned aerial vehicles and unmanned ground vehicles. Exemplary methods may include determining a camera path for a vehicle, wherein the camera path is a path that a camera is to follow while capturing images or videos of the vehicle, positioning the camera at the starting position, and capturing an initial image or video segment of the vehicle using the camera.The example procedure may include matching the first image or video segment to a first description, moving the camera to a second position along the camera path, capturing a second image or video segment of the vehicle, and matching the second image or video segment to a second description.

[0004] The object of the present invention is to further develop the prior art in such a way that a more extensively automated verification of an autonomous vehicle at the end of a paid journey is provided, which also includes the interior. This object is achieved by methods according to claims 1 and 13 and by an autonomous vehicle according to claim 3. Advantageous embodiments of the invention are described in the dependent claims.

[0005] Also known from US 2016 / 0 129 999 A1 are drone systems to assist drivers of large wheeled vehicles in reversing and to carry out inspections of the wheeled vehicle before driving, and from US 9 390 489 B1 is the assessment of damage to infrastructure using aerial images taken by an unmanned aerial vehicle.

[0006] US 2016 / 0129999A1 refers to the use of a UAV to assist drivers of large wheeled vehicles in reversing and to conduct pre-departure inspections of the wheeled vehicle. The UAV may be a multirotor drone that utilizes simultaneous localization and mapping technology to maneuver autonomously.

[0007] US 9,390,489 B1 provides a method and system for assessing damage to infrastructure using aerial imagery. The aerial imagery is acquired by an unmanned aerial vehicle (UAV), a manned aerial vehicle (MAV), or a satellite device. Specifically, an infrastructure element can be identified for damage assessment. The UAV, MAV, or satellite device can then acquire aerial imagery in an area surrounding the identified infrastructure element. Subsequently, the aerial imagery can be analyzed to determine the condition, extent, and / or severity of the damage to the infrastructure element. Furthermore, the aerial imagery, along with information about the extent of the damage, can be displayed on a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The specific features, aspects and advantages of the present invention can be better understood with reference to the following description and the accompanying drawings, in which the following applies: Fig. Figure 1 illustrates an example block diagram of a computing device. Fig. Figure 2 illustrates an environment that facilitates the use of an unmanned aerial vehicle (UAV) to verify an autonomous vehicle. Fig. Figure 3 illustrates a flowchart of an exemplary procedure for using an unmanned aerial vehicle (UAV) to verify an autonomous vehicle. The Fig. 4A and Fig. 4B illustrates examples of an unmanned aerial vehicle (UAV) being released from a protected area within an autonomous vehicle. Fig. Figure 5 illustrates an example of an unmanned aerial vehicle (UAV) flying around an autonomous vehicle. Fig. Figure 6 illustrates an example of an autonomous vehicle passing an unmanned aerial vehicle (UAV). Fig. Figure 7 illustrates an example of an autonomous vehicle driving over an unmanned aerial vehicle (UAV). DETAILED DESCRIPTION

[0009] The present invention relates to methods, systems and computer program products for using unmanned aerial vehicles (UAVs) for checking autonomous vehicles.

[0010] A Transportation as a Service (Taas) fleet includes one or more autonomous vehicles. Each of these autonomous vehicles can provide rides to third parties for a fee. Each autonomous vehicle carries a UAV (or "drone") in a secure area, for example, in a glove compartment, trunk, etc. Between rides, the vehicle can use the UAV to inspect the vehicle. During the inspection, the UAV remains electronically tethered to the vehicle. Once an inspection is complete, the UAV can return to the secure area.

[0011] During an inspection, the UAV can check both the interior and exterior of an autonomous vehicle. The UAV can check the interior for cleanliness and shine. The UAV can inspect the exterior for fluid leaks, tire damage, body damage, and the integrity of the brake pads, as well as ensure that the lights and turn signals are functioning, etc.

[0012] The UAV's flight path can be pre-programmed. To conduct an inspection, the UAV follows the flight path, pausing at designated positions to take images of specific parts of the vehicle. A UAV can fly around and underneath an autonomous vehicle during an external inspection. A UAV can also land on the ground and allow an autonomous vehicle to drive past and / or over the top of the UAV during an external inspection.

[0013] During an inspection, a UAV can send still images and / or video to the autonomous vehicle. The autonomous vehicle, in turn, can forward the still image and / or video to cloud computing resources for analysis. Specifically, the still images and / or video are sent to a neural network running on the cloud computing resources. The neural network analyzes the still images and / or video for anomalies, such as fluid puddles, reduced tire tread, tire sidewall damage, brake pad wear, etc. If certain anomalies are detected, the inspection fails.

[0014] If a check is passed, the autonomous vehicle can begin a new journey. If a check is failed, the autonomous vehicle contacts a transportation service to request a replacement autonomous vehicle. Additionally, depending on the detected anomaly(s) that caused the failed check, the autonomous vehicle can autonomously return to a repair facility or electronically request a tow truck.

[0015] The protected area can include a docking station for the UAV. The docking station can resemble a charging port for an electric vehicle (EV), the fuel filler neck, or a cavity in the trunk. Any cavity in the vehicle with a door that opens and closes automatically can be used.

[0016] The docking station can include a landing pad, multiple permanent hooks, one or more electromagnetic pins (or electromagnets), and components for wireless power transfer (WPT) via inductive coupling. The multiple permanent hooks can be used to anchor the UAV in several directions, such as front, rear, and right. The electromagnetic pins can be used to anchor the UAV in a different direction, such as to the left. The WPT components can be used to charge the UAV's batteries between checks.

[0017] In one aspect, the electromagnetic pins are normally closed (anchored) so that the electromagnets draw power from the vehicle while the UAV performs its inspection. Keeping the electromagnetic pins normally closed also balances the vehicle's power requirements for battery recharging. Furthermore, power contacts can be used instead of inductive / wireless power transfer. During landing, using a downward-facing camera, the UAV can position itself under the hooks and signal the vehicle to deactivate the locking pin's electromagnetic mechanism.

[0018] The capabilities of a UAV can extend beyond simply conducting vehicle inspections. For example, after an accident, a UAV can survey the accident scene, all vehicles involved, affected infrastructure, vehicle occupants and belongings, and witnesses (taking statements), and provide security until the relevant authorities arrive. The UAV can electronically summon the relevant authorities. In this respect, the UAV can be used to prevent theft and accident tampering, which can be helpful for an investigation. A recorded video can also serve as the basis for an insurance claim.

[0019] In one specific aspect, a UAV is used to inspect the interior of an autonomous vehicle between TaaS (Technology as a Service) trips. Cameras on the UAV can complement inward-facing cameras mounted on the vehicle. The UAV can request the autonomous vehicle to lower a window (in good weather), fly into areas that the inward-facing cameras cannot cover, exit the vehicle through an open window, and then request the vehicle to raise the window.

[0020] The complexity of an inspection can be tailored to a specific service level and / or the battery capacity of a UAV. Vehicle services in the higher segment may involve more frequent and / or more thorough inspections. Vehicle services in the lower segment may involve less frequent and / or less thorough inspections.

[0021] The frequency and accuracy of inspections can be enhanced using a variety of techniques. An inspection can involve multiple flight / recharge cycles per vehicle stop. A UAV can land beneath an autonomous vehicle and guide the autonomous vehicle back and forth while the UAV remains stationary. A UAV can land alongside an autonomous vehicle and guide the autonomous vehicle back and forth while the UAV remains stationary. A UAV can utilize other landing sites, such as walls, and guide the autonomous vehicle back and forth while the UAV remains stationary. Cooperative flight control can be used to guide a UAV using sensors on the autonomous vehicle (e.g., cameras, LiDAR, ultrasound, GPS, radar, etc.) instead of having the UAV carry the sensors.

[0022] A UAV can also perform diagnostics, alignment, and calibration of the autonomous vehicle's sensors (e.g., LiDAR, radar, camera, ultrasound, etc.). For example, the UAV can carry a metal object for radar, a reflective object or itself for LiDAR, itself for a camera, a visible pattern for calibration, etc.

[0023] A UAV can also perform maintenance on an autonomous vehicle. For example, a UAV can carry a damp sponge. The UAV can use the damp sponge (e.g., by rotating it) to clean sensor covers after bad weather or to remove dirt in other ways.

[0024] Aspects of the invention can be implemented in a wide range of different types of computing devices. Fig. Figure 1 illustrates an example block diagram of a computing device 100. The computing device 100 can be used to perform various operations, such as those discussed in this document. The computing device 100 can function as a server, a client, or any other computing unit. The computing device 100 can perform various communication and data transmission functions, as described in this document, and can execute one or more application programs, such as those described in this document. The computing device 100 can be any of a wide range of computing devices, such as a mobile phone or other mobile device, a desktop computer, a notebook computer, a server computer, a portable computer, a tablet computer, and the like.

[0025] The computing device 100 includes one or more processors 102, one or more storage devices 104, one or more interfaces 106, one or more mass storage devices 108, one or more input / output (I / O) devices 110, and a display device 130, all connected to a bus 112. The processor(s) 102 includes one or more processors or controllers that execute instructions stored in the storage device(s) 104 and / or mass storage device(s) 108. The processor(s) 102 may also include various types of computer storage media, such as cache memory.

[0026] The storage device(s) 104 includes / include various computer storage media, such as volatile memory (e.g., random access memory - RAM) 114 and / or non-volatile memory (e.g., read-only memory - ROM) 116). The storage device(s) 104 may also include rewritable ROM, such as flash memory.

[0027] The mass storage device(s) 108 includes / include various computer storage media, such as magnetic tapes, magnetic disks, optical disks, solid-state storage (e.g., flash memory), and so on. As in Fig. As shown in Figure 1, a particular mass storage device is a hard disk drive 124. Furthermore, various drives may be included in the mass storage device(s) 108 to enable reading from and / or writing to the various computer-readable media. The mass storage device(s) 108 includes removable media 126 and / or non-removable media.

[0028] The I / O device(s) 110 includes various devices that enable data and / or other information to be entered into or retrieved from the computing device 100. An example of an I / O device 110 includes cursor control devices, keyboards, keypads, barcode readers, microphones, monitors or other display devices, loudspeakers, printers, network interface cards, modems, cameras, lenses, radars, CCDs or other image acquisition devices, and the like.

[0029] The display device 130 includes any type of device capable of displaying information to one or more users of the computing device 100. Examples of the display device 130 include a monitor, a display terminal, a video projection device, and the like.

[0030] The interface(s) 106 includes various interfaces that enable the computing device 100 to interact with other systems, devices, or computing environments, as well as with people. An example interface 106 may include any number of different network interfaces 120, such as interfaces to personal area networks (PANs), local area networks (LANs), wide area networks (WANs), wireless networks (e.g., near field communication (NFC), Bluetooth, Wi-Fi, etc.), and the internet. Other interfaces include a user interface 118 and a peripheral device interface 122.

[0031] Bus 112 enables the processor(s) 102, the storage device(s) 104, the interface(s) 106, the mass storage device(s) 108, and the I / O device(s) 110 to communicate with each other and with other devices or components connected to bus 112. Bus 112 represents one or more of several types of bus structures, such as a system bus, PCI bus, IEEE 1394 bus, USB bus, and so on.

[0032] Fig. Figure 2 illustrates an environment 200 that facilitates the use of unmanned aerial vehicles (UAVs) 222 for testing autonomous vehicles. The environment 200 includes the vehicle 201, such as an autonomous vehicle, truck, or bus. The vehicle 201 can include one or more occupants, such as occupant 236. The vehicle 201 includes external sensors 202, internal sensors 212, a control system 208, vehicle exterior components 213, and vehicle interior components 233. The vehicle 201 also includes a protected area 211, such as a trunk, glove compartment, etc., for storing the unmanned aerial vehicle (UAV) 222.

[0033] The control system 208 can include control components (processors, controllers, working memory, data storage, network connectivity, global positioning system, etc.) that are used to autonomously control the vehicle 201.

[0034] External sensors 202 include one or more of: LiDAR sensor(s) 203, radar sensor(s) 204, and camera(s) 206. The external sensors 202 may also include other types of sensors (not shown), such as acoustic sensors, ultrasonic sensors, and electromagnetic sensors. In general, the external sensors 202 can detect and / or monitor objects near the vehicle 201. The external sensors 202 can output sensor data that indicates to the control system 208 the position and / or optical flow (i.e., direction and speed) of monitored objects.

[0035] The internal sensors 212 include one or more cameras 216. The cameras 216 can be used to view areas within the vehicle 201, such as seating areas, occupants in the seating areas, etc. In general, internal sensors can detect and / or monitor objects within the vehicle 201. The internal sensors 212 can output sensor data that indicates to the control system 208 the position, status, etc., of monitored objects.

[0036] The vehicle exterior components 213 include components of the vehicle 201, such as tires 241, brake pads 242, lights 243, turn signals 244, etc. In the case of a car or truck, the tires 241 can include four or more tires. Each tire can be attached to a wheel that rotates the tire. The vehicle interior components 233 include the seats 234. One or more occupants, such as occupant 236, can ride in the seats 234 when the autonomous vehicle 201 is in motion. The vehicle components can also include any of a variety of other components found in vehicles, such as doors, windows, an engine, a transmission, an exhaust, a throttle, a battery (and other electrical system components), environmental control components, media system components, etc.

[0037] Based on the sensor data, the control system 208 can control the vehicle's external components 213 and the vehicle's internal components 233 in order to operate the vehicle 201 autonomously.

[0038] Protected area 211 can include a docking / recharging station for the UAV 222. When the vehicle 201 is in motion, the UAV 222 can be moored in protected area 211.

[0039] In one aspect, vehicle 201 is part of a transportation-as-a-service (Taas) fleet. Vehicle 201 can be used to provide autonomous rides to third parties for a fee. Between rides, UAV 222 can be released from protected area 211 to check the interior and / or the exterior of vehicle 201.

[0040] Fig. Figure 3 illustrates a flowchart of an exemplary procedure 300 for using an unmanned aerial vehicle (UAV) to verify an autonomous vehicle. The procedure 300 is described in terms of the components and data in the environment 200.

[0041] Procedure 300 involves recognizing the termination of a paid ride offered to one or more occupants (301). For example, the control system 208 can recognize that a displayed destination for a paid ride offered to an occupant 236 has been reached. Upon reaching the displayed destination, the control system 208 considers the paid ride to be terminated. The occupant 236 may then leave the vehicle 201.

[0042] Procedure 300 involves determining that the autonomous vehicle should be inspected according to the inspection rules (302). For example, the control system 208 may determine that the vehicle 201 should be inspected according to the inspection rules 237. The inspection rules 237 may define when and how the vehicle 201 should be inspected. The frequency and / or accuracy of an inspection may vary based on the level of service provided by the vehicle 201. Vehicle services at the higher level may be assigned more frequent and / or more accurate inspections. Conversely, vehicle services at the lower level may be assigned less frequent and / or less accurate inspections. The frequency and / or accuracy of an inspection may also be based on the battery life of the UAV 222.

[0043] Procedure 300 involves releasing the unmanned aerial vehicle (UAV) from a protected area within the autonomous vehicle in order to inspect the autonomous vehicle (303). For example, the control system 208 can release the UAV 222 from the protected area 211 (e.g., a trunk or glove compartment) to inspect the vehicle 201. Releasing the UAV 222 may involve releasing one or more hooks and / or electromagnetic pins used to recharge the UAV 222 (e.g., via wireless power transfer - WPT).

[0044] Method 300 involves receiving images of one or more of the vehicle's interior and exterior components from the unmanned aerial vehicle (UAV) while the UAV is inspecting the autonomous vehicle (304). The UAV 222 can fly into the interior of the vehicle 201 and take images of the vehicle's interior components 233. For example, the UAV 222 can take an image 271 of the seats 234. The UAV 222 can transmit the image 271 to the control system 208. The control system 208 can receive the image 271 from the UAV 222. The UAV 222 can also fly around the exterior of the vehicle 201 and take images of the vehicle's exterior components 213. For example, the UAV 222 can take an image 272 of the tires 241. UAV 222 can send image 272 to control system 208. Control system 208 can receive image 272 from UAV 222.

[0045] Images of other vehicle interior components 233 and vehicle exterior components 213 can also be sent from the UAV 222 to the control system 208. In one aspect, the UAV 222 records one or more videos of the vehicle interior components 233 and / or vehicle exterior components 213. These one or more videos can also be sent from the UAV 222 to the control system 208.

[0046] Method 300 involves forwarding the images to other computing resources for image analysis (305). For example, the control system 208 can forward images 271 and 272 (along with all other images and / or videos) to cloud resources 291 for image analysis. The images and / or videos can be forwarded via wireless communication.

[0047] In one aspect, the cloud resources 291 include a neural network designed according to a multilayer (or "deep") model. A multilayer neural network model can include an input layer, a variety of hidden layers, and an output layer. A multilayer neural network model can also include a loss layer. For image analysis, values ​​in an image or video (e.g., pixel values) are assigned to input nodes and then fed through the variety of hidden layers of the neural network. The variety of hidden layers can perform a series of nonlinear transformations. At the end of the transformations, an output node produces an inference of possible maintenance problems (damaged tire tread or sidewalls, malfunctioning turn signals, body damage due to a collision, etc.) for the vehicle 201 based on an input image.Multiple different images and / or videos of different parts of vehicle 201 can be analyzed.

[0048] Based on the results of the image analysis, the cloud resources 291 can send instructions 273 back to the vehicle 201. Instructions 273 can be sent back via wireless communication. If no potential maintenance issues are identified (i.e., the vehicle 201 passes the inspection), instructions 273 can instruct the vehicle 201 to pick up passengers for a paid ride. Conversely, if potential maintenance issues are identified (i.e., the vehicle 201 fails the inspection), instructions 273 can instruct the vehicle 201 to return to a repair facility or electronically summon a tow truck.

[0049] The procedure 300 includes instructions from the other computing resources that instruct the autonomous vehicle on how to proceed based on an image analysis of the images (306). For example, the control system 208 can receive instructions 273 from the cloud resources 273. The vehicle 201 can then act according to the instructions 273. For example, the vehicle 201 can autonomously proceed to the nearest pickup location to collect one or more occupants for a paid ride. Alternatively, the vehicle 201 can autonomously proceed to a repair location or electronically summon a tow truck. In combination with proceeding to a repair location or electronically summoning a tow truck, the vehicle 201 can request a replacement vehicle.

[0050] In one aspect, instructions 273 instruct the vehicle 201 to send additional images and / or videos of one or more vehicle components to the cloud resources 291 for further analysis. The control system 208 then instructs the UAV 222 to receive the additional images and / or videos. The UAV sends the additional images and / or videos to the control system 208, which in turn forwards them to the cloud resources 291 for analysis. Based on the analysis of the additional images and / or videos, the cloud resources 291 can generate further instructions for the vehicle 201.

[0051] Procedure 300 involves holding the unmanned aerial vehicle (UAV) in the unprotected area (307). For example, when a verification flight path is completed and / or no additional images and / or videos are requested by the cloud resources 291, the UAV 222 can return to the protected area 211. The UAV 222 may be secured in a charging station within the protected area 211 for recharging. The UAV 222 may be secured with one or more hooks and / or electromagnetic pins for recharging (e.g., wireless power transfer (WPT)) via inductive charging.

[0052] The Fig. 4A and Fig. Figure 4B illustrates an example of an unmanned aerial vehicle (UAV) being released from a protected area within an autonomous vehicle. Fig. 4A releases a UAV 403 from the trunk 402 of an autonomous car 401. Once released, the UAV 403 can fly around the autonomous car 401, recording images of its external components. Alternatively, the UAV 403 can land, and the autonomous car 401 can drive past or over the UAV 403. The UAV 403 can record images of its external components while the car 401 drives past or over it. When an image is captured, the UAV 403 can return to the trunk 402.

[0053] In Fig. 4B releases a UAV 413 from the glove compartment 412 of an autonomous car 411. Once released, the UAV 413 can fly inside the interior of the autonomous car 411, recording images (e.g., for cleanliness or damage). The UAV 413 can fly out through a window to record images of external components of the autonomous car 411. When the image recording is complete, the UAV 413 can return to the glove compartment 412.

[0054] Fig. Figure 5 illustrates an example of an unmanned aerial vehicle (UAV) 502 flying around the autonomous car 501. The UAV 502 can fly between different positions (e.g., according to a pre-programmed flight path) to capture images of external components of the autonomous car 501.

[0055] Fig. Figure 6 illustrates an example of an autonomous car 601 driving past an unmanned aerial vehicle (UAV) 602. The UAV 602 can capture images of external components of the autonomous car 601 as the autonomous car 601 drives past the UAV 602.

[0056] Fig. Figure 7 illustrates an example of an autonomous car 701 driving over the unmanned aerial vehicle (UAV) 702. The UAV 702 can capture images of external components (e.g., the chassis) of an autonomous car 701 as the autonomous car 701 drives over the UAV 702.

[0057] In one aspect, one or more processors are configured to execute instructions (e.g., computer-readable instructions, computer-executable instructions, etc.) to perform any of a variety of described operations. The one or more processors can access information from system memory and / or store information in system memory. The one or more processors can convert information between different formats, such as images, videos, validation rules, instructions, etc.

[0058] System memory can be linked to one or more processors and can store instructions (e.g., machine-readable instructions, machine-executable instructions, etc.) that are executed by that one or more processors. System memory can also be configured to store any of a variety of other types of data generated by the components described, such as images, videos, validation rules, instructions, etc.

[0059] The preceding disclosure refers to the accompanying drawings, which form part thereof and illustrate specific implementations in which the disclosure may be carried out. It is understood that other implementations may be used and structural modifications may be made without deviating from the scope of the present disclosure. References in the description to "an embodiment," "an exemplary embodiment," "an exemplary embodiment," etc., indicate that the described embodiment may include a certain feature, structure, or property, but not every embodiment necessarily includes that certain feature, structure, or property. Furthermore, such formulations do not necessarily refer to the same embodiment.Furthermore, it should be noted that if a particular feature, structure or property is described in connection with an embodiment, it is within the scope of the skilled person's knowledge to implement such a feature, structure or property in connection with other embodiments, whether this is expressly described or not.

[0060] Implementations of the systems, devices, and methods disclosed in this document may include or utilize a specialized or general-purpose computer containing computer hardware, such as one or more processors and system memory, as explained herein. Implementations within the scope of this disclosure may also include physical and other computer-readable media for transporting or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media accessible by a general-purpose or specialized computer system. Computer-readable media on which computer-executable instructions are stored are computer storage media (devices). Computer-readable media that transport computer-executable instructions are transmission media.Thus, implementations of the disclosure may, for example, and without limitation, include at least two distinctly different types of computer-readable media: computer storage media (devices) and transmission media.

[0061] Computer storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), flash memory, phase-change memory (“PCM”), other types of storage, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or specialized computer.

[0062] An implementation of the devices, systems, and methods disclosed in this document can communicate via a computer network. A "network" is defined as one or more data connections that enable the transport of electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted to or provided to a computer via a network or other communication link (either wired, wireless, or a combination of wired and wireless), the computer correctly views the link as a transmission medium. Transmission media can include a network and / or data links that can be used to transport desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or specialized computer.Combinations of the above should also be included in the scope of computer-readable media.

[0063] Computer-executable instructions include, for example, instructions and data that, when executed on a processor, cause a general-purpose computer, a specialized computer, or a specialized processing device to perform a certain function or group of functions. The computer-executable instructions can be, for example, binary files, instructions in an intermediate format such as assembly language, or even source code. Although the subject matter has been described in a language specific to structural features and / or methodological actions, it is understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above. Rather, the described features and actions are disclosed as exemplary implementations of the claims.

[0064] The person skilled in the art will understand that the disclosure can be performed in network computing environments with many types of computer system configurations, including dashboard or other vehicle computers, PCs, desktop computers, laptops, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. The disclosure can also be performed in distributed systems environments where both local and remote computer systems connected by a network (either by wired data links, wireless data links, or a combination of wired and wireless data links) perform tasks.In a distributed systems environment, program modules can reside in both local and remote storage devices.

[0065] Furthermore, the functions described in this document may optionally be implemented in one or more of the following: hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and processes described in this document. Certain terms are used throughout the description and the claims to refer to specific system components. Those skilled in the art will understand that components may be referred to by different designations. This document does not distinguish between components that differ in name but not in function.

[0066] It should be noted that the sensor implementations discussed above may include computer hardware, software, firmware, or any combination thereof to perform at least some of their functions. For example, a sensor may include computer code configured to run on one or more processors and may include hardware logic / electrical circuitry controlled by the computer code. These exemplary devices are provided in this document for illustrative purposes only and are not intended to be limiting.

[0067] Embodiments of the present disclosure can be implemented in other types of devices, as is known to the person skilled in the art.

[0068] At least some embodiments of the disclosure are directed to computer program products comprising such logic (e.g., in the form of software) stored on any computer-usable medium. When executed in one or more data processing devices, such software causes a device to function as described herein.

[0069] Although various embodiments of the present disclosure have been described above, it is understood that these serve only as examples and not as limitations. It is evident to the person skilled in the art that various modifications in form and detail can be made without departing from the spirit and scope of the disclosure. Thus, the breadth and scope of the present disclosure should not be limited by any of the exemplary embodiments described above, but should only be defined in accordance with the following claims and their equivalents. The foregoing description has been set out for illustrative and descriptive purposes. It makes no claim to completeness and is not intended to limit the disclosure to the specific form disclosed. Many modifications and variations are possible in light of the foregoing teachings.Furthermore, it should be noted that any or all of the aforementioned alternative implementations can be used in any desired combination to form additional hybrid implementations of the revelation. REFERENCE MARK LIST 100 calculating devices 102 processor(s) 104 storage device(s) 106 interface(s) 108 mass storage device(s) 110 Input / Output (I / O) Device(s) 112 Bus 114 Random Access Memory (RAM) 116 Read-only memory (ROM) 118 User interface 120 network interfaces 122 Peripheral device interface 124 GB hard disk drive 126 Removable Media 130 Display device 200 surroundings 201 vehicles 202 external sensors 203 LiDAR sensor(s) 204 radar sensor(s) 206 camera(s) (external) 208 Tax system 211 protected area 212 internal sensors 213 Vehicle exterior components 216 camera(s) (internal) 222 Unmanned aerial vehicle (UAV) 233 vehicle interior components 234 seats 236 occupants 241 tires 242 brake pads 243 Lighting 244 turn signals 401, 411 autonomous car 402 trunk 412 Glove compartment 403, 413 Unmanned Aerial Vehicle (UAV) 501, 601, 701 autonomous car 502, 602, 702 Unmanned Aerial Vehicle (UAV)

Claims

[1] A method, the method comprising: Detecting the termination of a journey for payment offered to one or more passengers (236); Determining that an autonomous vehicle (201) should be checked according to checking rules; Releasing an unmanned aerial vehicle (222) from a protected area (211) within the autonomous vehicle (201) to inspect the autonomous vehicle (201); Receiving images (271, 272) of one or more of: vehicle interior components (233) and vehicle exterior components (213) of the autonomous vehicle (201) from the unmanned aerial vehicle (222) inspecting the autonomous vehicle (201); Forwarding the images (271, 272) for image analysis; and Receiving instructions (273) instructing the autonomous vehicle (201) how to proceed based on the analysis of the images (271, 272). [2] The method of claim 1, further comprising retaining the unmanned aircraft (222) in the protected area (211) after the verification is completed. [3] Autonomous vehicle (201), wherein the autonomous vehicle (201) comprises: one or more processors (102); a system memory (104) coupled to the one or more processors (102), the system memory (104) storing instructions executable by the one or more processors (102); an unmanned aerial vehicle (222) stored in a protected area (211) within the autonomous vehicle (201); wherein the one or more processors (102) are configured to execute the instructions stored in the system memory (104) to use the unmanned aerial vehicle (222) to check the autonomous vehicle (201), which includes the following: Detecting the termination of a paid journey offered to one or more passengers (236); Determining that the autonomous vehicle (201) should be checked according to checking rules; Releasing the unmanned aerial vehicle (222) from the protected area (211) to inspect the autonomous vehicle (201). Receiving images (271, 272) of one or more of: vehicle interior components (233) and vehicle exterior components (213) of the autonomous vehicle (201) from the unmanned aerial vehicle (222) while the unmanned aerial vehicle (222) is inspecting the autonomous vehicle (201); forwarding the images (271, 272) to other computing resources (291) for image analysis; Receiving instructions (273) from the other computing resources (291) instructing the autonomous vehicle (201) how to proceed based on the images (271, 272); and Picking up the unmanned aircraft (222) in the protected area (211). [4] The autonomous vehicle (201) of claim 3, further comprising a charging station in the protected area (211) for wirelessly charging the unmanned aerial vehicle (222), and wherein receiving the unmanned aerial vehicle (222) in the protected area (211) comprises anchoring the unmanned aerial vehicle (222) to the charging station. [5] The autonomous vehicle (201) of claim 3, wherein determining that the autonomous vehicle (201) should be inspected according to inspection rules comprises determining that the autonomous vehicle (201) should be inspected based on a service level assigned to the autonomous vehicle (201). [6] The autonomous vehicle (201) of claim 3, wherein receiving images (271, 272) from the unmanned aerial vehicle (222) comprises receiving images (271, 272) indicating the cleanliness of an interior of the autonomous vehicle (201). [7] The autonomous vehicle (201) of claim 3, wherein receiving images (271, 272) of vehicle exterior components (213) comprises receiving images (271, 272) of one or more vehicle exterior components (213) of the autonomous vehicle (201), wherein the one or more vehicle exterior components (213) are selected from: tires (241), brake pads (242), turn signals (244), and headlights (243). [8] The autonomous vehicle (201) of claim 3, wherein forwarding the images (271, 272) to other computing resources (291) for image analysis comprises forwarding the images (271, 272) to a neural network. [9] The autonomous vehicle (201) of claim 3, wherein receiving instructions (273) from the other computing resources (291) comprises receiving instructions (273) from a neural network. [10] The autonomous vehicle (201) of claim 3, wherein receiving instructions (273) from the other computing resources (291) comprises receiving instructions (273) instructing the autonomous vehicle (201) to return to a repair location. [11] The autonomous vehicle (201) of claim 3, wherein receiving instructions (273) comprises receiving instructions (273) instructing the autonomous vehicle (201) to electronically request a tow vehicle. [12] The autonomous vehicle (201) of claim 3, further comprising executing instructions stored in the system memory (104) to instruct the unmanned aerial vehicle (222) to perform a repair operation on the autonomous vehicle (201). [13] A method for use with an autonomous vehicle (201) and for using an unmanned aerial vehicle (222) to check the autonomous vehicle (201), the method comprising: Detecting the termination of a paid journey offered to one or more passengers (236); Determining that the autonomous vehicle (201) should be inspected according to the inspection rules; Releasing the unmanned aerial vehicle (222) from a protected area (211) within the autonomous vehicle (201) to inspect the autonomous vehicle (201); Receiving images (271, 272) of one or more of: vehicle interior components (233) and vehicle exterior components (213) of the autonomous vehicle (201) from the unmanned aerial vehicle (222) while the unmanned aerial vehicle (222) is inspecting the autonomous vehicle (201); forwarding the images (271, 272) to other computing resources (291) for image analysis; Receiving instructions (273) from the other computing resources (291) instructing the autonomous vehicle (201) how to proceed based on the analysis of the images (271, 272); and Holding the unmanned aircraft (222) in the protected area (211). [14] The method of claim 1 or 13, wherein receiving images (271, 272) from the unmanned aerial vehicle (222) comprises receiving images (271, 272) from the unmanned aerial vehicle (222) while the unmanned aerial vehicle (222) is flying around the autonomous vehicle (201). [15] The method of claim 1 or 13, wherein receiving images (271, 272) from the unmanned aerial vehicle (222) comprises receiving images (271, 272) captured by the unmanned aerial vehicle (222) as the autonomous vehicle (201) passes by the unmanned aerial vehicle (222). [16] The method of claim 1 or 13, wherein receiving images (271, 272) from the unmanned aerial vehicle (222) comprises receiving images (271, 272) captured by the unmanned aerial vehicle (222) while the autonomous vehicle (201) is traveling over the unmanned aerial vehicle (222).

Images