Vehicle Supply Chain Damage Tracking System

Abstract

A system for capturing VIN numbers and vehicle images to track vehicle damage through the vehicle supply chain, the system including a mobile software application and / or a robot that autonomously navigates around a parking lot. The mobile application prompts a user to capture VIN and / or vehicle images from a specific view and can collect GPS locations of the VIN and / or vehicle images. The robot includes various cameras and sensors to identify and photograph vehicles. All vehicle images are sent to a central server / storage where the vehicle images can be checked for damage compared to location to determine who owned the vehicle when the damage occurred.

Description

[Technical Field]

[0001] The following relates to a system and method for tracking damage in a vehicle supply chain to determine where / when the damage occurred so that the appropriate responsible party can be determined. [Background technology]

[0002] When you buy a new car from a dealer, it smells new, it's shiny, and it's perfect. However, what most retail vehicle buyers don't realize is that new cars often get damaged during their journey from the factory to the showroom floor. There's an allowable amount of damage a car can sustain before it's still considered new, but while this damage is acceptable, someone has to repair it, and then someone has to pay for it. Typically, each car leaving the factory will have many different people "touch" it or be responsible for certain parts of its journey. Because vehicles often change hands multiple times before reaching a new car dealer, when damage occurs, it's important to know when that damage occurred in the supply chain so you can know who was responsible for causing it, and similarly, who was not responsible for damaging the vehicle at a particular time if it was under the ownership of a particular company. Ultimately, insurance companies are responsible for paying everyone who repaired the car. However, each responsible party for a vehicle may have a different insurance company, and even if they are insured by the same insurance company, the insurance company needs to know who is causing the damage in case a particular responsible party in the supply chain is causing damage more frequently than others, including cases where that party would not be insured without process changes or other protections, or if an increase in premiums is appropriate. Furthermore, in the current environment, in addition to the insurance premium for each vehicle that is covered during transportation, the insured (e.g., manufacturer) pays the insurance company a fee for each vehicle. The insurance company then pays the company to inspect the vehicle, which is a relatively costly process. Currently, insurance company inspection costs are approximately $800 per day per on-site inspector, and the quality of the inspection is often insufficient, resulting in an inability to actually identify where the damage occurred.

[0003] If damage to a car is identified at a new car dealership or its warehouse, without a reliable tracking system, various responsible parties will tend to say "it wasn't me" or "they did it," or simply ask for proof that the damage actually occurred if the car was under the watch of a particular entity. The end result is a lack of inspection processes throughout the supply chain, resulting in a great deal of time being wasted trying to determine who is responsible.

[0004] The current method of tracking damage involves a human inspecting the car, using a piece of paper to indicate the different vehicle panels so that damage can be marked. This is no different from traditional rental car inspections. However, the problem with this human inspection is that inspectors are given a large number of cars to inspect per hour—approximately 200 per day. This leaves them with only about two minutes to inspect each vehicle, including the necessary paperwork to document the damage.

[0005] Given the close spacing of parked cars, human inspectors may be at a disadvantage in that they may not be able to position themselves to get a very good vantage point to visually inspect the damage, and even if they were in a good vantage point, there is human error in that the damage may not be seen or recognized. This may result in the wrong party being identified as responsible, and an errant insurance company paying an insured for damage that was not actually liable.

[0006] Some solutions have been considered, such as drive-through garages / structures with cameras to take multiple photos of each vehicle. However, this solution has significant drawbacks in that it slows the unloading process, often resulting in the trucking company limiting the window of opportunity before the driver is instructed to depart (or charging the unloading company a fee). Therefore, given the time pressure to unload cars from trains (or transport trucks or ships) in a timely and orderly manner, drive-through garages that slow this process are not a practical solution. Furthermore, given the large amount of equipment required by such drive-through garages, the system may become significantly more expensive, requiring numerous display sites and locations to be installed at each location in order to accurately image the condition of vehicles throughout the supply chain. Also, fixed infrastructure such as this drive-through garage solution may not be practical due to its size or footprint, necessitating the use of human inspectors at these locations, which reintroduces the element of human error that the drive-through garage was intended to eliminate.

[0007] Therefore, what is desired is a more cost-effective solution that allows for easy and repeatable image capture with high quality, correct angles and fields of view while reducing and preferably eliminating human error, and accurately and correctly captures the condition of a vehicle as it moves through the supply chain from the factory floor to the showroom floor. It is also desirable that this solution be able to work with existing display stand configurations given the limited space available. Summary of the Invention

[0008] It is therefore an object of the present invention to provide a vehicle damage tracking system that can repeatedly take high quality images of a vehicle at different locations throughout the supply chain and then integrate them into a vehicle record. The images taken may be, for example, still images and / or video.

[0009] Another object of the present invention is to provide a system for automatically and accurately capturing photographs of VIN numbers and their corresponding vehicles, all associated with a GPS location.

[0010] At various stages of the supply chain, large lots often offer 108-inch-wide spaces where vehicles are parked with minimal gaps, preferably in the 6-12 inch range. Each allocated vehicle space is 108 inches by 240 inches (linear parking), with vehicle widths of approximately 60 to 80 inches (excluding mirrors). Vehicle lengths tend to range from 13 to 16 feet. The net result is approximately 28 to 48 inches of inter-vehicle distance, not including mirrors (reducing usable space in certain locations). Herringbone parking lots use roughly the same dimensions as above, except that the distance from the front corner of one vehicle's bumper to the front corner of another vehicle can only be 1 to 2 inches apart. The vertical distance from the front corner bumper of one vehicle to the front corner bumper of the vehicle in front of it may range from 84 inches to 113 inches, although this distance may vary depending on how the human operator parks the vehicle and is not an exact measurement. In many cases, the distance between mirrors may be as little as 14 inches, typically for the largest 80-inch-wide vehicles in the lot. Smaller vehicles offer more space on the premises because they are approximately 60 inches wide. This distance is just enough for a worker to get into the vehicle and drive it into position (because mirrors add about a foot of space). The worker can also get out of the vehicle, allowing other vehicles to be positioned on the premises. The distance between the bumper and the nose is very short, usually 6 to 12 inches, so inspectors cannot easily walk around the front bumper. The tight spacing is designed to fit as many cars as possible on the premises while minimizing other risks (e.g., damage), since inspectors are typically paid a daily wage and handle approximately 200 vehicles per day. However, premises and those involved attempt to pack as many cars as possible into the available space to reduce costs; the more cars that can be transported and displayed at a given location, the more profitable it is for shippers.Therefore, another object of the present invention is to provide a solution that allows images of vehicles to be taken without the need to change the spacing of the vehicles and the configuration of the site, in particular a robot that can move around the site and under the vehicles and position cameras to take pictures of the location at close spacing as described herein.

[0011] One of the more challenging parts of scanning a vehicle is accurately capturing the VIN. Glare, various lighting conditions, reflections, and weather conditions can make it difficult to automatically capture the VIN. Therefore, it is an object of the present invention to provide a method for scanning, verifying, and repositioning and rescanning a vehicle image so that the image can be associated with the correct VIN in order to capture the VIN accurately.

[0012] It would be further desirable to provide a mobile application that collects images of vehicles and associates those images with VIN numbers and vehicle records, either as a standalone solution where the robot alone may not be able to adequately image and scan a particular vehicle and manual intervention may be required, or as a conflict / error resolution system.

[0013] These and other objects are accomplished by providing cloud / web-based software that communicates with one or more robots / mobile devices that receives VIN numbers and images from those robots / devices associated with vehicles. More particularly, the cloud / web software receives images from those robots / devices at different locations associated with the same vehicle as those robots / devices capture and record the locations of those images to most efficiently capture images of the vehicle at a particular parking location.

[0014] In one aspect, a system for autonomous scanning of vehicle VIN numbers is provided, the system including a robot having a camera and a controller executing software that identifies a vehicle positioned within the frame of the camera. The software controls the movement of the robot to position the camera where the VIN number is expected to be visible from outside the vehicle. The software moves the camera to position the VIN number within the frame of the camera and captures an image of the VIN number.

[0015] In other aspects, the software identifies a vehicle positioned within the camera frame by recognizing at least one identifying feature of the vehicle, matches the at least one identifying feature with at least one known vehicle identifying feature from a database of known vehicle identifying features, compares the location of the at least one identifying feature to determine a location where a VIN number is expected to be visible based on an expected VIN location associated with the known vehicle identifying feature, and moves the robot to or near the location where the VIN number is expected to be visible as determined by the software.

[0016] In a further aspect, the database of known vehicle identifying features resides in storage on the robot. In another aspect, the matching further includes selecting from known vehicle identifying features associated with only the manufacturer of at least one vehicle in a group of vehicles for which the robot captures the plurality of images.

[0017] In a further aspect, the software determines whether a VIN number has been imaged and communicates over a network with a computer having a data store containing multiple VIN numbers, where the VIN number imaged in the first image is matched with at least one of the multiple VIN numbers in the data store. The software captures multiple images of vehicles surrounding the vehicle and associates the multiple images in the data store with a vehicle record associated with the VIN number.

[0018] In one aspect, a system for autonomously scanning a vehicle's VIN number includes a robot having a camera and a controller executing software that identifies a vehicle positioned within the camera's frame, the software controlling the robot's movement to position the camera where the VIN number is expected to be visible from outside the vehicle. The software moves the camera to position the VIN number within the camera's frame based on an image captured by the camera. The software captures an initial image, determines whether the VIN number has been captured, and communicates over a network with a computer having a data store containing multiple VIN numbers, where the VIN number captured in the initial image is matched to at least one of the multiple VIN numbers in the data store. The software acquires multiple images of vehicles surrounding the vehicle based on the matched VIN number and associates the multiple images in the data store with a vehicle record associated with the VIN number.

[0019] In one aspect, if the software determines that the VIN number was not captured, the software identifies one or more faults, captures the VIN number based on the image, moves the camera to another location, and captures a new initial image that is used to determine if the VIN number was captured.

[0020] In another aspect, the software determines whether the VIN number has been imaged by performing visual character recognition on the image and comparing it to an expected configuration of the VIN number, and determines whether imaging of the VIN number is complete before matching the imaged VIN number to at least one of the plurality of VIN numbers in the data store.

[0021] In another aspect, the software directs the robot to capture multiple images for multiple vehicles within a defined geographic space. The robot includes a global positioning system (GPS) receiver, and the captured VIN numbers and / or images are associated with one or more locations based on the GPS receiver. Further, the software positions the robot at a next location based on an identification of a next vehicle among the multiple vehicles, such that the location of the next vehicle is at a location different from all previous locations of the captured VIN numbers and / or images.

[0022] In other embodiments, the robot receives instructions over the network identifying VIN numbers and / or previous locations where other robots have taken images. In still other embodiments, the robot takes images of two or more of the plurality of vehicles at one or more of the one or more locations and associates these images with a VIN number other than the one taken. In still other embodiments, the electrical connection is further associated with a semi-rigid support structure comprising multiple linkages that fold and unfold from within the first body. In still other embodiments, the camera is mounted on a gimbal of the drone tethered to a portion of the robot including a source of electrical potential that provides power to the drone.

[0023] In one aspect, the robot includes a drone tethered to a first body of the robot, the first body including a cradle that is wider than the drone such that the maximum width of the drone is less than the width of the first body, the width of the drone and first body being measured perpendicular to the axis of rotation of one or more propellers of the drone when positioned in the cradle.

[0024] In yet another aspect, a robot includes a first body including one or more motors configured to propel the body over a surface and a source of stored electrical potential; and a second body including a camera and one or more air thrusters configured to propel the second body through the air. An electrical connection between the first body and the second body such that the source of electrical potential provides power to the one or more air thrusters via the electrical connection. The second body is configured to be mounted to the first body in a first configuration having a maximum height above the surface of less than 12 inches. The electrical connection is configured to extend to at least three times the maximum ground clearance, optionally to less than 12 times the ground clearance, and preferably all images of the vehicle are captured from a height less than 12 times the ground clearance. The maximum combined thrust of the one or more air thrusters is less than or equal to half the weight of the robot. Software executing on the controller controls movement of the one or more motors and activation of the one or more thrusters and image capture by the camera to position the first body so that the air thrusters can decouple the second body from the first body to capture a plurality of images of each of the plurality of vehicles and to capture a VIN number for each vehicle. A network connectivity device on the first body is configured to transmit the plurality of images captured by the camera to a remote computer over a network, and each of the plurality of images associated with a vehicle record based on the VIN number is further associated with a location determined by a global positioning system (GPS) receiver of the robot.

[0025] In some embodiments, the robot further comprises at least one camera on the first body and at least one proximity sensor on the first body.

[0026] In yet another embodiment, the source of stored potential is at least 1.5 times the weight of the second body, not including the electrical connections to the first body.

[0027] In yet another aspect, the robot comprises a plurality of robots, and the system further comprises, for each robot, the commands generated by software based on identifying one of the plurality of vehicles in proximity to each robot, the commands determined based on a location of one of the plurality of vehicles compared to previous locations of other vehicles of the plurality of vehicles for which images have already been captured by the robot, and the commands further based on data received via network hardware indicating locations of other vehicles of the plurality of vehicles for which other robots of the plurality of robots have already captured images, such that the commands direct each robot to a location for which the robot and the other robots of the plurality of robots have not yet captured images.

[0028] In yet another aspect, at least one of the previous locations is associated with an orientation indicating the direction in which a camera was pointed to capture an image associated with the previous location, and the orientation is determined by the direction indicated by the robot's orientation reference system compared to the position of the camera as rotated relative to the orientation.

[0029] Yet another object is achieved by providing an autonomous image-taking robot including a first body including one or more motors configured to propel the body over a surface and a source of stored electrical potential; and a second body including a camera and one or more air thrusters configured to propel the second body through the air. An electrical connection is between the first body and the second body such that the source of electrical potential provides power to the one or more air thrusters via the electrical connection. The second body is configured to be mounted to the first body in a first configuration having a maximum height above the surface of less than 12 inches. The electrical connection is configured to extend at least three times the maximum ground clearance, and preferably, images are taken at less than 12 times the ground clearance. The maximum combined thrust of the one or more air thrusters is less than half the weight of the robot. Software executing on the controller controls movement of the one or more motors and activation of the one or more thrusters and image capture by the camera to position the first body so that the air thrusters can decouple the second body from the first body to capture a plurality of images of each of the plurality of vehicles and to capture a VIN number for each vehicle. A network connectivity device on the first body is configured to transmit images captured by the camera over a network to a remote computer, each image associated with the VIN number and a location determined by the robot's global positioning system.

[0030] In one embodiment, the source of stored potential is at least 1.5 times the weight of the second body, not including the electrical connection to the first body. An orientation reference system is on the second body configured to measure a magnetic orientation. A camera on the second body rotates about an axis, and each of the images is further associated with an orientation determined by comparing the magnetic orientation to the rotational position of the camera about the axis, the orientation indicating the direction the camera was pointed when capturing each of the images.

[0031] In one embodiment, a network connectivity device allows the two robots to communicate with each other to send and receive the captured VIN number and the location of the captured image.

[0032] Another object is achieved by providing a system for autonomously capturing images of a plurality of vehicles, including a plurality of robots, each of the plurality of robots having one or more motors for moving the robot over a surface, a camera, a controller having software executing on the camera, network hardware for communicating over a network, and a global positioning system (GPS) receiver. The controller controls movement via the one or more motors based on commands from the software. For each robot, a command is generated by the software based on identifying one of the plurality of vehicles in proximity to each robot, the command being determined based on a position of the one of the plurality of vehicles compared to previous locations of other vehicles of the plurality of vehicles for which images have already been captured by the robot, and the command is further based on data received via the network hardware indicating the locations of other vehicles of the plurality of vehicles for which images have already been captured by the other robots, such that the command directs each robot to a location for which the robot and the other robots of the plurality of robots have not yet captured an image.

[0033] In yet another aspect, a vehicle supply chain tracking system using vehicle image capture includes a system computer running software configured to receive image and location data from a plurality of mobile devices, the plurality of mobile devices including first and second mobile devices each having software executing on the first and second mobile devices, the mobile devices further comprising a camera and a global positioning system (GPS) receiver, the system computer software configured to receive from the first mobile device a VIN number of a first vehicle at a first location as captured by the camera and the first mobile device software, as determined by the GPS receiver of the first mobile device, the software further configured to receive a plurality of first sets of images of the first vehicle captured by the first mobile device, and further associate each of the first sets of images with a VIN number in a vehicle record for the first vehicle, the plurality of first sets of images being stored in storage accessible by the system computer. The system computer software is further configured to receive from the second mobile device a VIN number for the first vehicle at a second location that is a different location from the first vehicle in that the first vehicle has been moved relative to the first location when imaged by the camera and software of the second mobile device, the second location being determined by a GPS receiver of the second mobile device, the software is configured to receive a plurality of second sets of images of the first vehicle imaged by the second mobile device, and further associate each of the second sets of images with a VIN number in a vehicle record for the first vehicle, and the plurality of second sets of images are stored in storage. In one aspect, the plurality of mobile devices are robots having a ground movement device configured to move the robot over a ground surface.

[0034] In one aspect, the robot further includes at least one thruster configured to move a portion of the robot through the air to position a camera of the robot to capture multiple images of the first vehicle. In another aspect, the multiple mobile devices are computing devices selected from the group consisting of mobile phones, tablets, laptops, or other mobile devices including a display, a camera, a processor, and a user input device. In one aspect, software running on the computing devices displays one or more prompts to capture the VIN number and the first and second multiple sets of images. In another aspect, the one or more prompts displayed after capturing the VIN number include a target displayed on the screen of each computing device, the target identifying the first vehicle via the camera and positioned and sized based on software of the computing device that determines the position of the camera relative to the vehicle and based on a default view of the first vehicle. In yet another aspect, the screen displays an alignment target, and once the target and the alignment target are aligned, the first or second multiple sets of images are captured by software of the computing device.

[0035] In other aspects, software on the first and / or second mobile device is configured to receive an indication of damage corresponding to one or more images of the first and / or second plurality of sets of images and associate the indication of damage with the corresponding image.

[0036] In some embodiments, the first and second locations are further associated with different responsible entities. In other embodiments, the portal includes the software on the system computer that enables access to vehicle records, the vehicle records indicating different responsible entities.

[0037] In yet another aspect, the first and second mobile devices each comprise a plurality of mobile devices, and the first vehicle comprises a plurality of vehicles, each of the plurality of vehicles having its own associated vehicle record stored in the storage.

[0038] In other aspects, the software of the first and second mobile devices images the VIN number with the cameras of the first and second mobile devices by scanning using barcode scanning or visual character recognition, checks whether the string of letters and / or numbers determined by the scan matches an expected VIN number pattern, and if the scan is unsuccessful or the string does not match, the software of the corresponding mobile device generates control instructions to reposition the camera.

[0039] Another object is realized by providing a vehicle image capturing system including a system computer on which software executes and a mobile computing device configured to communicate with the system computer over a network, the mobile computing device having a camera, a display, and executing mobile software, the mobile software configured to receive a VIN number for at least a first vehicle. The software on the mobile computing device is configured to display one or more prompts, the prompts including a plurality of targets displayed on the screen, each target positioned and sized based on the mobile software identifying the first vehicle via the camera and determining a position of the camera relative to the vehicle, based on a predefined view of the first vehicle. When one of the targets is aligned with an alignment target displayed on the display, the mobile software captures an image of the vehicle with the camera, and the alignment target remains in a fixed position on the display while the mobile device moves around the vehicle and each of the targets moves on the display.

[0040] In some aspects, the mobile software captures a plurality of images, each image associated with one of the plurality of targets, and transmits the plurality of images over a network to a system computer where a vehicle record associated with the VIN number is associated with the plurality of images. In other aspects, each of the plurality of images is associated with a location determined by a GPS receiver of the mobile computing device. In yet other aspects, the mobile software displays one or more prompts for capturing an image of the VIN number via the camera. In yet other aspects, the mobile computing device comprises a plurality of mobile computing devices, and the software is configured to receive first and second sets of images from different mobile computing devices of the plurality of mobile computing devices and associate the first and second sets of images with the same vehicle record based on the VIN number.

[0041] In yet another aspect, the first and second sets of images are associated with different locations indicating that the same vehicle was at two different locations. In yet another aspect, each of the plurality of mobile devices is associated with a GPS receiver, and the two different locations are determined based on the GPS receiver. In yet another aspect, the first and second sets of images are a plurality of first sets of images and a plurality of second sets of images, and each set of the plurality of first sets of images is associated with a different VIN number and vehicle record. In another aspect, determining the position of the camera relative to the vehicle includes identifying at least two points on the vehicle and determining a distance and angle of the camera relative to the vehicle.

[0042] In yet another aspect, a vehicle supply chain tracking system using vehicle image capture is provided. The system includes a system computer running software communicating with one or more mobile devices and one or more robots over one or more networks, each mobile device and each robot including a camera and a GPS receiver. The software is configured to receive a plurality of VIN numbers from the one or more mobile devices, at least two of the VIN numbers captured by a camera operated by a mobile application running on the software, and the VIN numbers associated with GPS locations. The software is further configured to send instructions to the one or more robots to direct at least one of the one or more robots to each of a plurality of GPS locations such that at least one of the one or more robots moves to or is adjacent to the GPS locations and captures a plurality of images of the vehicle associated with the VIN numbers at or near the corresponding GPS locations.

[0043] Other objects of the present invention and its particular features and advantages will become more apparent from a consideration of the following drawings and the accompanying detailed description. [Brief explanation of the drawings]

[0044] [Figure 1] FIG. 1 is a functional flow diagram according to the present invention. [Figure 2] FIG. 2 is a top view of the system shown in FIG. 1 implemented in a parking lot. [Figure 3A] FIG. 3A is a cross-sectional side view of the robot of FIGS. 1 and 2. FIG. [Figure 3B] FIG. 3B is a front perspective view of the robot of FIGS. 1 to 3A. [Figure 3C] FIG. 3C is a front perspective view of the robot of FIGS. 1 to 3A. [Figure 3D] FIG. 3D is a top view of the robot showing the undercarriage camera. [Figure 3E] FIG. 3E is a cross-sectional side view of another embodiment of the robot of FIGS. 1 and 2. [Figure 3F] FIG. 3F is a detailed perspective view of the robot of FIG. 3E. [Figure 3G] FIG. 3G shows how the UAV portion of the robot of FIG. 3E is deployed. [Figure 3H] Figure 3H is a front view of the UAV with the hood / shade. [Figure 4] FIG. 4 is a flow diagram according to certain features of the system of FIG. [Figure 5] FIG. 5 is a flow diagram according to certain features of the system of FIG. [Figure 6A] FIG. 6A is a flow diagram according to certain features of the system of FIG. [Figure 6B] FIG. 6B is a flow diagram according to certain features of the system of FIG. [Figure 7A] FIG. 7A is a flow diagram according to certain features of the system of FIG. [Figure 7B] FIG. 7B is a flow diagram according to certain features of the system of FIG. [Figure 8] FIG. 8 is a screenshot of a mobile application based on the system of FIG. [Figure 9] FIG. 9 is a screenshot of a mobile application based on the system of FIG. [Figure 10] FIG. 10 is a screenshot of a mobile application based on the system of FIG. [Figure 11] FIG. 11 is a screenshot of a mobile application based on the system of FIG. [Figure 12] FIG. 12 is a screenshot of a mobile application based on the system of FIG. [Figure 13] FIG. 13 is a screenshot of a mobile application based on the system of FIG. [Figure 14]FIG. 14 is a screenshot of a mobile application based on the system of FIG. [Figure 15] FIG. 15 is a screenshot of a mobile application based on the system of FIG. [Figure 16] FIG. 16 is a screenshot of a mobile application based on the system of FIG. [Figure 17] FIG. 17 is a screenshot of a mobile application from the system of FIG. [Figure 18] FIG. 18 is a screenshot of a mobile application according to the system of FIG. [Figure 19] FIG. 19 is a diagram illustrating movement and deployment of the UAV components of the robot of FIGS. 3A-3C. [Figure 20] FIG. 20 illustrates the movement and deployment of UAVs and robots in different parking lot configurations. [Figure 21] FIG. 21 is a diagram illustrating selectable vehicle records generated by the system of FIG. [Figure 22] FIG. 22 is a diagram illustrating selectable vehicle records generated by the system of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0045] Referring now to the drawings, wherein like reference numerals indicate corresponding structure throughout the views, the following examples are presented to further illustrate and explain the present invention and should not be construed as limiting in any regard.

[0046] 1 and 2, the system architecture is detailed. The system computer 12 includes software 30 that runs on the system computer 12 and provides a portal for viewing, creating, or modifying vehicle records 26. Specifically, vehicle records 26 are associated with VIN numbers, and numerous photographs of these corresponding vehicles are taken. Initially, a batch of VIN numbers is added to the system to identify new vehicles entering the supply chain. This may be, for example, from a factory or from a shipping port where the vehicle unloaded. The VIN information 24 can be used to auto-populate vehicle details. For example, a VIN number can identify a vehicle's make, model, production date, color, and other characteristics from the number alone. Thus, the vehicle record 26 can show this information before photos are added from the robot 6 and / or mobile device 7. Both the robot 6 and the mobile device 7 preferably include a GPS system, such as an appropriate processor and antenna and receiver, for communicating with the GPS satellite network 14. The system computer 12 instructs the robot 6 / mobile device 7 where the target area 18 is located. This target area 18 may be the boundary 2 of a vehicle lot containing a number of vehicles. This target area 18 will cause the robot to move from the charging dock 10 into this boundary 2 to begin identifying the vehicle. Once the vehicle is identified by the robot's various sensors / cameras, the robot attempts to image the VIN number. In a preferred embodiment, this involves releasing the UAV near the bottom of the driver's side windshield, where the VIN number is typically expected to be displayed. This imaged VIN 22 is transmitted to the system computer along with the image and location 16. The location may be embedded in the georeferenced image. VIN information 28 is also provided to the robot 6 and / or mobile device 7 for those images and is preferably maintained in the robot's storage. This VIN information 28 ensures that once the imaged VIN 22, image, and location 16 are transmitted to the system computer 12, the vehicle is no longer on the list of VIN numbers that need to be imaged.The VIN information 28 is typically a list of VINs expected to be in the target area 18. The VIN information may also include the make, model, vehicle type, color, or other characteristics. Thus, when a robot captures an image, if a VIN number indicating a red car turns out to be black, for example, this may indicate a problem, such as the VIN number being scanned incorrectly or there being some other issue that may require manual inspection. This location can be flagged and sent to the mobile device 7 for human inspection. As various robots moving around the perimeter 2 capture VIN numbers and images, the locations of those previously imaged vehicles 20 are transmitted to the other robots. As shown, this information is transmitted from the system computer 12 back to the robots. However, it is also conceivable that the robots may be able to communicate with each other locally, using appropriate short-range networking hardware such as Bluetooth. In this way, each robot recognizes when a vehicle in its VIN information 28 list has been captured and imaged so that it knows where it should not go within the perimeter. Because each robot also knows the previous locations of the other robots, the robots can more efficiently capture images of all vehicles. The illustrated network 9 may be, for example, a cellular network such as a 3G, 4G, 5G, or other telecommunications network. In other embodiments, the robot communicates locally about its location and the captured VIN number as it moves around the lot 2 to capture images. Upon returning to the charging dock 10, which may have a Wi-Fi connection, the robot can send the images via the network 9 to the system computer 12, which stores the images in the vehicle record 26 along with the associated VIN number and location. However, as the robot 6 moves around the lot 2, the captured VIN numbers and their vehicle locations are transmitted to other robots.

[0047] The system computer may also contain storage that includes vehicle specifications 27. These vehicle specifications may include many different vehicle identifying features such as wheelbase, width, height, light shape, bumper, window / windshield size, etc. These specifications further relate the coordinates or measurements of where the VIN plate is located relative to these known identifying features, as will be further described with respect to FIG.

[0048] Referring specifically to FIG. 2, one robot 6 is located at the top of the lot 2, and a second robot 8 is located at the bottom of the lot. Typically, the robots follow the path shown by the dotted line. Typically, the target area 18 also includes a representation of the lot's parking configuration, such as a daisy-chain configuration as shown in FIG. 2 or a herringbone configuration shown in other diagrams or configurations. In this way, the robots have a basic understanding of the pattern and expected vehicle location layout. As shown, the robots can communicate with themselves and with a system computer 12 via a network 9. Alternatively, the charging dock 10 may operate as a Wi-Fi hub that also communicates with the system computer. In the example shown in FIG. 2, the front of the vehicle may be oriented toward the system computer as shown. Next, the first robot 6 would image the VIN number on the driver's side windshield and capture images of the vehicle along the driver's side. Once the robots reach the rear of the vehicle, the UAV component can move into the space between the vehicle's bumpers and take additional photos as needed, while the ground unit remains positioned along the dotted line. At this point, the location of the first vehicle's image is recorded by the robot, after which the robot may return to, for example, the passenger-side bumper and resume photographing the remaining vehicles. In this manner, the robot continues photographing the next vehicle in the line, capturing images of the VIN number and the front, driver's side, and rear of the vehicle, until the end of the line of cars. At this point, the robot then moves along the driver's side aisle, capturing images of those cars whose VIN numbers have already been scanned, and then capturing the VIN numbers and images for the next line of cars. Based on the direction the UAV camera is pointing (e.g., the direction of travel) and the location of images where known VIN numbers have already been captured, the robot's software can determine where to pick up or where new, unscanned vehicles are located to add to a particular vehicle record. Thus, for the second line of vehicles, the UAV begins taking photos from the rear bumper toward the front, eventually scanning the VIN number and associating these previous images with the scanned VIN number.Robot 6 continues to move around the lot and communicates with robot 6' about the locations of vehicles for which images have already been taken. It is also possible for the two robots to cooperate, with a first robot taking images of one side of a central row of vehicles and a second robot taking images of the other side of the row. Because the locations where the first robot's images were taken are known, the second robot can know where it needs to take other images, for example, to take an image of the passenger side panel. The VIN number scanned by the first robot can then be associated with the images taken by the second robot.

[0049] 3A-3C illustrate an exemplary robot according to the present invention. A first body 35 moves along the ground, while a second body 34, or UAV (unmanned aerial vehicle), rests within a cavity 37 in the first body 35. The cavity contains magnets 42 that help guide the UAV into position upon landing and help hold the UAV in place. The illustrated UAV has motors 39 and propellers 41, preferably four of each. The combined thrust of the propellers 41 / motors 39 is great enough to overcome the magnetic force of the registration magnets 42, but not enough to lift the first body 35. The UAV also includes a camera 32 mounted on a gimble 32' to take photographs of the vehicle so that its condition and damage can be recorded and tracked. The camera 32 may be capable of capturing images in both the visible and non-visible light spectrum. For example, infrared capabilities of camera 32 may be useful in certain lighting conditions or for vehicle / vehicle part identification, and may include polarized lenses or filters on the lenses. All cameras described herein may optionally have these capabilities.

[0050] First body 35 may also include a camera, such as camera 36, ​​that can be used to take photographs of the vehicle's undercarriage, since the robot's overall height is designed to be lower than the clearance of a typical vehicle's undercarriage that will be photographed. The UAV is tethered to first body 35 by arm 40, shown as multiple bars with pivots at both ends. The bars create a semi-rigid tether that, compared to a simple wire / cable tether, may be more resistant to crosswinds, for example, in that it provides increased stiffness when extended. However, it is also contemplated that a wire / cable tether could be used. In either case, the tether provides an electrical connection between UAV 34 and ground component 35. Semi-rigid arm 40 includes pivot 56, which allows the arm to rotate about a vertical axis as the UAV rotates. This pivot may be passive (e.g., simply a bearing or other hinge / pivot that allows free rotation) or active, in that a motor may be used to coordinate the rotation of the UAV and pivot 56.

[0051] The ability of the arm 40 to rotate about a vertical axis allows the UAV to position itself so that the direction of the added stiffness is aligned with the wind. For example, if the UAV is extended vertically from the position shown in Figures 3A-3C, the arm 40 can be considered to add stiffness in the direction into and out of the page, i.e., about an axis perpendicular to at least one of the pin axes of the pins located at both ends of the bars that make up the arm 40. Thus, the UAV and / or the pivot 56 can be rotated so that the direction of stiffness is perpendicular to the wind the UAV will encounter. As shown, the first body 35 has a space / passageway 38 through which the arm 40 can extend.

[0052] The first body also has passages 44 therein that allow air thrust from the propellers 41 to exit the vehicle and provide ventilation for the UAV. Proximity sensors 46 are preferably located on the front, rear, and sides of the vehicle. Charging terminals 48 are provided to allow the battery 50 to be recharged. In a preferred embodiment, the charging terminals 48 are connectable to a charging dock so that the vehicle can navigate itself toward the dock and be recharged if a low battery is determined. Motors 62 drive omnidirectional wheels 54, and the suspension 52 allows the vehicle to better navigate rough terrain. Specifically, the motors 62 rotate the wheels about their axes to enable forward movement and actuate secondary rollers on the periphery of the primary wheels to cause side-to-side movement of the robot or portion of the robot, as would be apparent to one skilled in the art. Conventional wheels could be used, as used on trucks and other ground-based mobile systems. The processor 58 includes controller hardware / processor 70, network hardware / processor 68, and a GPS system (e.g., GPS chip / receiver 66), and the vehicle also has appropriate hardware for controlling and / or collecting data from the motors 62, cameras 36 / 32, proximity sensors 46, UAV 34, pivot 56, and other features of the robot. In a preferred embodiment, the processor 58 includes software executing thereon that will cause the robot to move around a defined area 2 and capture images of the vehicle 4. More specifically, as the robot moves around area 2 (which may be a lot of varying shapes and sizes), a VIN number is captured by the robot navigating using the proximity sensors 46 and cameras 36 / 32 to identify the vehicle 4, and then moving around to capture the VIN number. Preferably, the processor 58 and motors 62 are contained in one or more water-resistant or waterproof compartments 60.

[0053] To capture an image of the VIN number, the UAV 34 would typically be deployed by the processor 58 commanding the motors 39 to spin up via commands delivered via a tether that includes electrical connections. Preferably, a battery 50 powers both the UAV and the ground unit, thereby improving the UAV's ability to capture photographs over extended periods of time because the UAV does not need to be large enough to lift the UAV's energy source. More specifically, in some preferred embodiments, the battery 50 powering the UAV weighs the same as or heavier than the UAV itself, not including the tether; or, more preferably, the battery weighs more than the combined UAV and its tether up to the point where the tether is electrically or mechanically connected to the first body 35. Preferably, the battery is at least 1.25 times the weight of the UAV, even more preferably at least 1.5 times the weight of the UAV, even more preferably at least 2 times the weight of the UAV, and most preferably 2.5 times or more. In a preferred embodiment, the weight of the robot, including the UAV and battery, is less than 60 pounds, more specifically, 50 pounds or less, and even more specifically, 45 pounds or less.

[0054] The UAV may also include a pitot tube for determining wind speed and direction, and an ADAHARS 72 (Air Data, Attitude and Heading Reference System), which may include an attitude system (e.g., a digital gyro) and a heading system. In this way, the orientation of the camera lens / view can be determined. ADAHARS may also include a magnetometer that can be calibrated to know a specific heading and then reference the UAV's camera. For example, by knowing that the camera is facing north and taking a picture of a vehicle's side panel, and then taking a picture of another vehicle's side panel from the same location facing south, the location of the two images may be the same, but the orientation of the images can be used to solve the problem of two different vehicles being imaged from the same location but with different perspectives. Specifically, the camera 32 is mounted on a gimbal that can rotate about a vertical axis and a horizontal axis (which itself can rotate), as shown in FIG. 3A. Based on the position of the camera lens and the position of the gimbal, the angle of the camera relative to the UAV can be determined. ADAHARS 72 will know its heading. For example, if the front is facing north and the camera is 90 degrees to the left around the gimbal's vertical axis, the robot's computer knows the camera is taking a photo facing west, but if the gimbal on its horizontal axis is rotated 180 degrees, the camera is then taking a photo facing east (potentially upside down). Thus, by knowing ADAHARS72 data, more specifically the gimbal's position combined with heading or other orientation data, the robot can distinguish between two different vehicles taking photos. By combining this information with GPS location, if the camera takes a photo between two cars, that 180-degree rotation of the horizontal gimbal axis indicates two different cars, and so for one car, there is another robot that took an image and captured the VIN, while the robot taking the photo may have obtained the VIN of the other car, and so should be associated with two different VIN records.This allows one robot to travel down the row of cars, taking pictures of the left and right side of that aisle, and based on the GPS location of each photo, software on the server can match those locations to images it has taken of the vehicle, but those images are known to be associated with a specific VIN number after scanning that VIN, so those locations are already associated with the VIN number.

[0055] The UAV portion is designed to fly in winds up to 20 mph, which is about two-thirds the maximum speed a UAV can fly. The combined robot (including UAV) weighs less than 50 pounds and has sufficient battery capacity for 8 hours, preferably 10 hours of battery life.

[0056] 3B and 3C, additional proximity sensors 46 are shown on the sides and front / rear of the vehicle. Preferably, at least one proximity sensor is present on each side. In some embodiments, there may be two different types of sensors, e.g., radar / lidar / infrared and camera. The top of the robot on the first body 35 includes a camera 32 and / or proximity sensor 46. Preferably, at least a camera is included so that the robot can move under the vehicle where images are captured to obtain photographs of the undercarriage. FIG. 3D shows an additional camera 41 that can move in and out of the body and points generally upward to capture images of the vehicle's undercarriage. This camera 41 is attached to a pivot 43 and can be retracted via an arm 39 actuated by a motor within the robot body. This is just one retraction mechanism available for this undercarriage camera 41.

[0057] FIG. 3E shows another robot embodiment similar to FIG. 3A. In FIG. 3E, a telescoping arm 40' is provided with multiple concentric hollow shafts. A cable 400 passes through the center of the concentric shafts and connects to the camera 32. In this case, the arm 40' pivots from or adjacent to one end of the robot (the rear end as shown). As shown in FIG. 3A, a pivot 56 can be used at the base of the arm 40'. Alternatively, the pivot 43' can be constrained to only allow the arm 40' to pivot about an axis extending out of the page in FIG. 3E.

[0058] FIG. 3G illustrates the UAV deployment process, in which the magnetic resistor allows the UAV to separate from the recess / cradle and the UAV begins to extend upward. As this occurs, the concentric shaft begins to retract as the UAV extends higher. In the rightmost view of FIG. 3G, arm 40' is fully extended, and cable 400 extends further than arm 40'. In this way, the arm remains relatively rigid and does not move from left to right, but rather moves back and forth. This is particularly useful for capturing VIN numbers or photographs of the hood / roof, etc., in that the arm typically remains roughly in a vertical plane aligned along the vehicle's longitudinal axis (second from the right). An additional cable 400 on top of arm 40' then allows the UAV to move outside of this plane of arm 40', for example, over the car's VIN plate, or over the roof or other location. By comparing the extension distance of the arm 40' to the height of the UAV above the ground, the robot will know the length of the cable and therefore know where the cable is so it can avoid snagging. As an example, it may be desirable to avoid the drone cable 400 getting caught on a mirror or to avoid the arm 40' / 40 colliding with the vehicle and being damaged. Thus, the semi-rigid arm 40' / 40 can provide sufficient stiffness to resist unconstrained and / or unpredictable movement of the cable 400, which could, for example, cause unwanted contact with the vehicle or get caught on obstacles such as mirrors, spoilers, or other vehicle features.

[0059] 4 and 5, the process of scanning a VIN (100) is initiated via the robot (FIG. 4) or the mobile application described herein (FIG. 5). Preferably, the VIN is scanned and a server check (102) is performed to verify that the VIN number is an expected VIN number at the particular location where the scan is being performed. Alternatively, the server may not be checked when creating a new vehicle record. If the server check (102) fails or if other errors occur during the VIN scan, error resolution (104) is performed. Specific details regarding error resolution (104) are described later in this disclosure. The robot then performs an unimaged location check (106) to verify that the VIN number has not already been scanned by another robot (or the robot itself). This may be performed by communicating with other robots locally or via a network connection, such as a cellular phone or Wi-Fi. The robot also checks an inventory of vehicles it has already scanned to verify that the VIN number has not been scanned previously. However, this internal checking process is often optional since the robot knows the GPS locations of recently scanned vehicles. However, given that sites are expected to be swapped for vehicles as they move through the supply chain, GPS location alone may not be sufficient, as another vehicle may be located at a previously scanned location. Also, when a vehicle is moved from the robot's location, the vehicle record may indicate a scan of the VIN number at another location so that if only a portion of the vehicle was moved, the robot can recognize which VIN number (and its location) has been replaced by another vehicle. Once the robot determines that it has scanned a new VIN number, it begins the process of capturing images (108). This process may involve, for example, driving a robot under the car to take pictures of the undercarriage and deploying a drone to take pictures of the vehicle's surroundings, particularly all of the vehicle's panels, bumpers, windows, roof, and other features.As these images are taken (or taken in batches), they are sent to a server (110), and as images are being taken, the robot also receives the locations of images taken by other robots working in the area. Thus, in the situation where a car is parked in the configuration shown in FIG. 3, a robot could take a photo of the driver's side (where the VIN plate would typically be) and then have another robot take a photo of the passenger side. Given that the location of the VIN number photo is known and the vehicle's orientation is also known from the photo taken from the driver's side (e.g., the vehicle is facing north or another orientation), the second robot can know which VIN number the passenger side image corresponds to, and based on a photo taken at 90 degrees to the vehicle, the orientation, as well as the vehicle's length and position, can be known from other photos or based on manufacturer specifications, known characteristics of the vehicle, such as wheelbase. Once these photos are received by the server, an update (112) to the vehicle record is made (or created, if necessary). The process for the mobile application used to take vehicle photos is generally the same as for the robot for the first few steps. However, the mobile application will prompt the user to take an image of the desired vehicle (114). This may include directing the user to another vehicle if the vehicle is found to have already been scanned. The user follows the prompt to take the image. This may be a prompt to take an image of a specific location on the vehicle, giving the user some flexibility as to how the image is taken (118), and may rather rely on the user to frame the photo correctly. In another embodiment, this involves aligning the AR target with an on-screen target (116) through a more automated or fixed structure of taking the image (118). More details about the on-screen AR target and alignment target are provided later in this disclosure. Once the image is taken, a GPS location is also provided with the photo (118).This may include the specific location of a particular photo, or the photo can be further correlated to the location of the VIN scan, and then from manufacturer specifications / measurements, given what prompts were displayed or what target alignment the image was taken, one can know where the photo was taken. As these images are taken, or optionally taken in batches, the images are sent (110) to a server and the vehicle record is updated / created (112). Additionally, the image locations are also sent to the server, and the mobile application can receive image locations of other images already taken by others performing an equal / similar image taking process using another mobile device running the application.

[0060] FIG. 6A shows further details of the VIN scanning process accomplished by the mobile phone or robot software. A camera is placed near the driver's side windshield and the VIN is scanned (100). This process allows the robot to recognize where the VIN is located, and the mobile phone user is also likely to know where the VIN is located. An image is captured and filtered / enhanced / illuminated (120) to obtain a high-quality image. An OCR or barcode or other scan (122) is performed to recognize strings and verify the string's properties (124). For example, a VIN number, often a sequence of numbers and letters, will indicate the make, model, color, and other characteristics of a car; if the sequence does not match an expected pattern, a problem is indicated. As one specific example, if the camera sees a red car, but the VIN indicates a blue car based on the string's properties, a failure (125) will occur and the VIN will be rescanned. Alternatively, manual intervention (127) may be required, which may involve flagging the vehicle / VIN / location for subsequent manual resolution or sending a message to the user via the mobile application described herein requesting a scan. Prior to initiating a scan (100) as shown in Figure 6A, or when repositioning, the robot's software will need to utilize various sensors / cameras to position the robot where the VIN is expected to be. This is detailed in Figure 6B.

[0061] As previously described, the vehicle specification 27 allows the robot to recognize vehicle identification features based on comparison with known vehicle identification features. FIG. 6B illustrates software logic that can be used to position the robot so that the VIN can be scanned (100). The vehicle specification 27 can be transmitted / received (627) at the robot as part of the VIN information 28 or as a separate download / transmission. The robot will then move (628) to a parking area, which may be area 2 defined by a geofence. The robot can direct this movement based on GPS coordinates. As this movement occurs, various cameras and proximity sensors on the robot are activated. The UAV may or may not be deployed. When the cameras / proximity sensors identify an object 130, they typically do so by identifying the size of the object. Larger objects are expected to be more significant targets, and as the robot moves, the object's location can be compared (634) to previous locations where images have already been taken / scanned. This check (634) may include verifying that a car of the same color / shape is in the same expected location, ensuring that a location that has already been scanned still houses the same vehicle. Alternatively, if the robot is scanning vehicles for the first time in a particular batch, it may know that all vehicles are targets that require imaging. Once an object is identified and selected based on the camera / proximity sensors, the robot will use its cameras and sensors to move toward the vehicle while avoiding colliding with other objects. As the robot approaches, it will begin searching for known vehicle identification features (632). This may include searching for bumpers, wheels, head / tail lights, windows, windshield, or other features of the vehicle. Typically, the camera will take a photograph and compare it to previous photographs of the vehicle, which may provide known vehicle identification information (618), and it is understood that these previous photographs can be used to generate contours of the shape of various components of the vehicle or to perform comparisons with previous photographs of known vehicles, as some examples.The comparison of known information can also measure the dimensions of vehicle features such as wheel spacing, width, and height, and these measurements can then be used to narrow down and determine the make / model / year / trim of the vehicle the robot is viewing. If the vehicle is a known type based on the comparison (618), the location of the VIN can be determined. Specifically, this known vehicle identification information may include the location of the VIN plate / VIN relative to other identifying features, and the robot can use a camera to triangulate the location of various features to know where the VIN is expected to be found, for example, on wheels and mirrors, and relative to the wheels / mirrors where the VIN is expected to be available. Thus, by comparing known vehicle identification information / features (610), the location of the vehicle's VIN can be known / approximated. The robot deploys a UAV to the VIN location (612), then scans the VIN (100), and in case of a failure (125), searches for identifying features, etc. (632), and then relocates it by repeating this process.

[0062] The received vehicle specifications (627) may be VIN information 28 or may include vehicle specifications 27 for only the vehicle for which the robot will capture an image. For example, if the list of vehicles includes only Nissan (make) Altima (model) vehicles, the Nissan Altima specifications would be sent along with the VIN information 28, and the robot would search for known identifying features of the Altima. If a batch of vehicles includes several different models and / or makes / years / trims, the received specifications (627) would cause the robot to search for vehicles expected to be present based on the batch being scanned. This may speed up vehicle comparison and identification. The list of vehicles to scan may allow the robot to narrow down the year, trim, or other further identifying features, if needed. Alternatively, the VIN information 28 may initially include all vehicle specifications, such as all makes / models / trims / years, in the system computer's storage, or may update / synchronize new / revised vehicle specifications 27 in the received specifications (627). These vehicle specifications 27 may be in the form of one or more images of the same make / model / year / trim of the vehicle, along with an indication of where the VIN plate is located and the VIN number correctly captured in relation to that / those images. In some embodiments, the vehicle specifications 27 may be determined by the system computer from these one or more images and updated based on the success of capturing the VIN number. Additionally, if a particular vehicle specification or known feature will most quickly result in a correct image and capture of the VIN number, the vehicle specifications 27 may indicate the same to the robot. The vehicle specifications 27 may also indicate the known shapes of known identifying features of the vehicle, such as bumper shape, grille shape, mirror spacing, wheelbase, window / windshield size / shape, etc. These shapes may typically be in the form of a path or surrounding outline that the robot / camera can recognize from a photograph of the vehicle by identifying color changes in large pixels, tracing the outline to create a shape, and comparing it to the known shape.

[0063] As an example, if the robot can successfully image the VIN number, with both wheels and two mirrors being most commonly identified, the robot will then explore identifying the wheels and mirrors. The learning process can also be extended to another level in that the robot can monitor the camera and captured image to show headlights, as identifying front headlights often most easily leads to identifying the wheels / mirrors. It should be understood that this is just one specific example of how to identify a VIN number and is provided to further illustrate an example of a machine learning process for identifying vehicles and quickly locating and imaging VIN numbers in an autonomous manner. To recognize vehicle identification features via a camera, the robot and its software can identify the edges of the feature by significant pixel color changes and then convert these significant color changes into a line / path. As an example, a wheel often has a rim next to a black tire, so the rim can be identified based on the significant pixel color change. A line is then drawn around this pixel change. Depending on the robot's angle, the robot moves a certain distance, recalculates the line, and compares the two to determine the center of the shape. Given that wheels are round, if the camera is looking off-center at the wheel, the calculated line may be oval. Therefore, by moving the robot / camera and recalculating the line, and then comparing the two lines, the robot can determine where the center is. Then, by identifying a second wheel and its center, triangulation calculations can be used to determine the wheelbase. Knowing the wheelbase, the selection of possible vehicles can be narrowed to only vehicles within a specific wheelbase. Next, the mirror locations or window locations or windshield locations can be determined from the narrowed selection of possible vehicles, for example, by make / model / trim / year, in that the mirrors are known relative to the wheels, and then the robot / camera and software can search for mirrors in those limited location selections.This process can continue until enough features of the vehicle have been identified to know where the VIN plate and VIN number are expected to be, and then the UAV is moved into position to image the VIN number.

[0064] Returning to FIG. 6A, the VIN scanning and verification process continues, verifying the string structure / characteristics (124) and checking the server (102) to verify that the VIN number matches the vehicle record or the list of vehicles in the batch being scanned. If any of these processes 100, 120, 122, 124, or 102 fail (125), the camera is repositioned (124) to obtain a better scan, or if repositioning fails frequently, manual intervention is used (127). Repositioning (124) may involve adjusting the camera's position due to sun / glare, deploying a screen or hood to more closely surround the VIN, using a flash, moving the robotic UAV portion to block glare, or other adjustments, resulting in a better image of the VIN for accurate VIN reading. By rotating the drone relative to the VIN, the shape of the UAV may provide a hood 300 (FIG. 3H). It should be understood that the UAV in Figure 3H is considered to have a gimble-mounted camera even though a non-gimble version is shown. Once the VIN is accurately captured, image capture (108) and vehicle record update / creation (112) proceed. If the VIN cannot be scanned, the robot can still take a photo and create a vehicle record, but log an error that uses a mobile app to direct a human to a specific vehicle and obtain the VIN number, either manually or by opening the door and scanning the barcode inside. The mobile app may direct the user to the vehicle with directional commands and display the captured image to ensure the correct VIN number matches the photo captured by the robot. Alternatively, the robot may not take a photo, and the mobile app may be used in situations where the robot cannot scan the VIN.

[0065] FIG. 7A shows additional details regarding the process of moving the robot around a lot where a batch of vehicles to be imaged is located. Here, the VIN batch is checked (126) to determine which vehicles have been scanned and their locations. If no vehicles have been scanned, the software will typically determine how many vehicles need to be scanned. A boundary is identified (128), typically indicating the perimeter of the parking lot or the portion of the parking lot where vehicles need to be scanned. The process of identifying vehicles (130) will begin using the robot's various proximity sensors and cameras. Vehicle features such as wheels, headlights, mirrors, bumpers, grilles, and other recognizable features can be identified, and as the robot moves toward a vehicle, the parking pattern can be determined (132) both in terms of the vehicle's orientation (e.g., pointing north or another direction) and the location of vehicles around that vehicle, for example, in a herringbone or grid pattern. Once a vehicle is identified (130), the software checks to determine whether the location has already been scanned (134). Previously scanned vehicles (136) will identify new vehicles (130) and repeat the process until it is determined that a new vehicle is in front of the robot to photograph. The robot proceeds and takes images (118) and GPS locations of those images, or associates those images with a single GPS location for the vehicle. The robot then sends the image locations to a server (110) and receives image locations from other robots. Once the image locations are received, they are also used to check (134) for locations already scanned. The vehicle record is updated with the image / location (112), and the robot then moves to the next location by automated software command (138).

[0066] FIG. 7B shows how a mobile application and robot are used, with the mobile application being used to capture VIN numbers and locations, and the robot being used to capture images based on these VIN locations. The mobile app is used to scan the VIN number (726). Once the VIN number is scanned, a GPS location is captured based on the GPS receiver / chip in the mobile device. These VIN locations are sent to a system computer / server (728) and received by the robot. In this way, the VIN scanning process can scan the exterior VIN plate, or if that doesn't work, scan a barcode on the inside of the door frame, or receive manual input. Thus, a human operating the mobile app can scan a batch of VIN numbers and recognize their locations, and then the robot takes over by identifying the vehicle (130), determining its location / parking pattern (132), and verifying that the vehicle hasn't already been scanned by another robot (134). Images and GPS locations are captured (118), transmitted and received (110), and the vehicle record is updated / created (112). Once the robot has captured a suitable image, it is instructed (138) to move to another location—typically associated with the scanned VIN via a mobile app whose location is known based on the GPS location. Using a mobile application in conjunction with a robot has the advantage of allowing a human to manipulate the camera position to efficiently capture the VIN number; if external imaging is not possible, a human operator can open the vehicle door and scan the barcode inside the door frame. While opening the door and scanning the door frame is a function that a robot could potentially have, it is highly complex. Thus, using a mobile device in conjunction with a robot allows for a more streamlined robot design and removes one of the more complex scanning processes from the robot's task, enabling accurate VIN number capture.

[0067] The aforementioned mobile application-based solution is provided with a software application that runs on the mobile device and communicates with a server over a network. The application provides a login page where individual users can log in and begin scanning vehicles and identifying damage. The user can create a new vehicle record (Figure 8) or a batch of vehicles and proceed to scan the VIN (Figure 9), which updates the vehicle record. If the same VIN has already been scanned, the existing record is accessed instead. Figure 9 illustrates image capture in which a VIN (or barcode) is scanned. Various information can then be determined from the VIN number, such as the manufacturer, make, model, trim, serial number, and year of manufacture, either from the VIN string itself or by communicating the VIN over a network to a server where the displayed information is stored. The user is then given instructions to scan the vehicle along an exemplary path to follow (Figure 10). At various points along the path, partially transparent or shaded circles are displayed around the car (Figure 11), and as is obvious, the mobile device has a fixed ring in the middle. When the camera is correctly positioned, the shaded circles appear as circles on the mobile device. If the camera angle relative to the vehicle is incorrect, the shaded circle will appear oval (see "Circle" to the left of the ring in Figure 11). Therefore, by placing the shaded circle in the center of the ring and roughly matching its shape to form concentric circles, the user knows that correct alignment of the camera relative to the vehicle has been achieved. The shaded circle therefore serves as a target for the ring-shaped alignment target to align with and capture the image. There is also an option to report damage by pressing a button displayed on the screen (Figures 12 / 13) and associate that damage in the image where the ring and circle are aligned.

[0068] The circles (FIG. 11) are shown over images of actual cars in FIGS. 13 and 14. The circles are generated so that as the user approaches or moves away from the car, the circle remains in place but appears smaller or larger depending on the distance. This distance is determined by the grid overlay shown in FIGS. 13 and 15 and is generated by software to position the augmented reality circles that are superimposed on the image captured by the camera. FIG. 13 shows an example of how a fixed ring relative to the phone screen is properly aligned with one of the circles. As can be seen, the circle is within the ring, and the relative size of the circle to the ring is within a suitable tolerance, indicating that the user is capturing the correct amount of the car in the frame. If the user is too far or too close, the screen may flash a direction or other instruction to instruct the user to move closer or farther away, as the case may be.

[0069] As seen in FIG. 13, the user is provided with a button to click to "Report Damage," and on the next page they can identify the type and / or cause of the damage. The user then continues the process of tracing around the vehicle and aligning the ring with the circle (FIG. 15). FIG. 16 shows the option to take detailed damage photos, and FIG. 17 shows the final result of the vehicle record creation, with multiple photos that can be viewed and selected to reflect the condition of the vehicle. FIG. 18 shows additional options for how damage can be located and identified, in that specific circles can be annotated and resized on the photos so that damage can be quickly located.

[0070] An exemplary process for generating the grid and circles (FIG. 13) is as follows: The user is asked to bring the phone close to the vehicle, and a scan is performed by the mobile device using the camera; preferably, a mobile device with multiple cameras is selected so that various elements can be triangulated relative to the camera and the lens focal length information is available. The mobile device then identifies elements of the car through image recognition, such as mirrors, headlights, and body panels. Specifically, the body panels are expected to be a consistent color or a predefined color scheme, and the wheels are expected to have a typically black tire pattern with a metal rim or hubcap. The VIN can also identify the tire and / or wheel size, which may aid in scaling the grid and placing the AR circles. When the VIN number is scanned, its color is recognized by the software, which can then locate it in the image to identify the car. Then, based on the location of the mobile device relative to the car, an AR grid is generated (FIG. 13). The grid may be visible, as shown (FIG. 13), or may simply be calculated without being shown. Next, a circle or target is placed on the device screen, and a ring or alignment target is placed in the center. As the user moves around the vehicle, the grid is continually re-evaluated in relation to the movements determined and identified by the camera. As the user walks around the vehicle, the software continues to generate and place AR circles / targets, as the circle for the right side of the vehicle may not be visible or calculated until the right side of the vehicle is in view. The grid is continually recalculated as new parts of the vehicle come into view and AR circles are added. Optionally, the image recognition features of the software can recognize various predictable vehicle features in various locations on the vehicle.

[0071] It should be understood that circles and rings are just one option by which alignment and positioning can be controlled and directed by the software. Another option is, for example, an AR path that may be oval or elliptical (from a top view of the car similar to the shape of FIG. 10 ), which can be generated by the user instructed by on-screen guides to keep the line within a certain tolerance. It should be understood that once the shape is aligned, the user may be instructed to pause while taking images, and the system will take a video that can later be analyzed for the desired individual shots around the car. The number of circles or imaging points can vary depending on the size of the car, with a large SUV having more alignment points than, for example, a compact two-door car. Thus, scanning the VIN can indicate to the software how many alignment points or imaging locations need to be generated. Referring to FIG. 14 , the user can annotate photos of damage from various drop-down menu categories. FIG. 16 shows the option to take more detailed damage photos, if desired. FIG. 18 shows the ability to annotate images with specific locations of damage.

[0072] FIG. 17 shows an example batch status screenshot that allows a user to determine inspection status and drill down further to view a specific batch. Selecting a batch allows for viewing of images and damage records. If damage is found, the vehicle handler is also recognized, and as the vehicle moves through various stages, additional vehicle handlers are associated with various sets of images. Thus, when damage first appears in an image, it is known which vehicle handler is believed to be responsible. FIG. 17 shows a specific batch drilled down by VIN number and vehicle records that can again be drilled down further to select and view images taken over the course of the transportation cycle to identify when and where the damage occurred.

[0073] As previously mentioned, the robotic solution involves a specially designed robotic device capable of navigating the tight spaces between vehicles, where the distance between mirrors is only 14 inches in the largest vehicle scenario. FIG. 19 shows side and rear views of an exemplary robotic element specifically adapted to capture images of vehicles in a large display area. That is, the base depth is less than 14 inches, preferably less than 12 inches, more preferably between 12 and 8 inches, and most preferably about 10 inches deep, to allow vehicles to move along a 14-inch aisle. The base is at least 1.5 times longer than it is wide; preferably, the length of the base is at least 18 inches, more preferably at least 20 inches, and even more preferably in the range of 22 to 30 inches. In the embodiment shown, the length is 24 inches. This length, combined with the fact that the wheel spacing is relatively close to the length of the base, provides stability, given that the maximum height of the arm / UAV is at least 1.5 times longer than the base. A wider base can increase stability.

[0074] The cameras (as well as other locations on the robot) may also be fitted with laser, radar, or other sensors so the robot can avoid crashing into adjacent vehicles. The robot is also fitted with optical, laser, or other sensors that look down and find painted parking space lines (typically yellow or white), which the robot can use to maneuver in the right place and sense vehicles.

[0075] The robot has GPS hardware, processing capabilities to track and follow location and path to ensure all vehicles are scanned, and cellular and Wi-Fi communication capabilities. The robot may also include lighting and / or flash to compensate for various environmental lighting conditions. The robot is preferably battery powered and may optionally include solar panels to charge the battery or power the robot.

[0076] The robot's onboard computer will connect to various sensors and guide the robot along the space between vehicles, as shown in Figures 19-20. In the linear parking space layout shown, using the three-vehicle example, starting in place, the camera can be rotated and positioned at the appropriate height to image the side of the vehicle. On one side of the vehicle, an arm will be deployed in the nose / tail space, and the camera will rotate to image the first and second vehicles in the path. Positioning the arm will include scanning the VIN tag so that the image can be associated with a specific vehicle. The computer is programmed to guide the robot around the vehicles, stopping, deploying the arm, and continuing to take images as needed. While three cars are shown, it should be understood that the line of scanned cars could be much longer.

[0077] A herringbone style scan is shown in Figure 20 and involves similar robotic movements, but along a different path, with the UAV deployed to the required location to capture images of the vehicle. While a straight line is shown to represent the deployment of the UAV, those skilled in the art will understand that the combination of a flexible tether and the UAV's ability to move in various directions may involve the UAV itself flying to a specific location to capture the desired image view of the vehicle.

[0078] Once images are sent to the server via cellular, Wi-Fi, or other network connection, they can be inspected by the server software and use image recognition to identify and flag cosmetic defects and damage. Flagged damage is then provided for human inspection to verify whether actual damage is present, and the image recognition system can use learning algorithms to better identify damage. These damage tags are added to the vehicle record with various images, and ultimately, upon inspection, the vehicle recipient (e.g., dealer) is given a specification sheet that allows them to visually inspect the locations of identified damage or defects, determine whether the damage actually occurred, and file claims for the various damages. Claims can then be designated for each identified damage, so the appropriate responsible party receives claims for the damage they are responsible for paying based on the location and time of the image that originally identified the damage. Insurance companies paying the damage can maintain reliable records, increase the efficiency of the claims settlement process, and reduce the number of improper or fraudulent damage claims.

[0079] In either the mobile application or robotic implementation, the relevant vehicle is imaged as it is delivered to the display lot or another responsible party during the vehicle's transport from factory to showroom. At each location the vehicle is scanned, the image is time-stamped and location-stamped, indicating where each image was taken and who was responsible for the vehicle at the time. As the VIN number is scanned at various locations, the vehicle record is updated with additional images and optional damage tags to identify any damage.

[0080] Figures 21 and 22 show exemplary vehicle records and their statuses that can be selected and drilled down further to see VIN-specific details. For example, clicking / selecting one of the records will display the information shown in Figure 17, for example, and can be viewed along with the other information for the vehicle record.

[0081] Although the present invention has been described with reference to particular arrangements of parts, features, and the like, these are not intended to describe all possible arrangements or features, and in fact many other modifications and variations will be ascertainable to those skilled in the art.

Claims

1. A vehicle image capture vehicle supply chain tracking system, comprising: the system computer is configured to receive image and location data from a plurality of mobile devices, including a first mobile device and a second mobile device, each of which is running mobile software; the first mobile device and the second mobile device are configured to transmit image and location data of a vehicle at a first location and a second location different from the first location to the system computer, respectively; and each of the first and second mobile devices further comprises a camera and a global positioning system (GPS) receiver; the software on the system computer is configured to receive from the first mobile device a VIN number of a first vehicle at a first location determined by the GPS receiver of the first mobile device as captured by the camera of the first mobile device; the software on the system computer is further configured to receive a first set of images of the first vehicle captured by the first mobile device and associate each of the first set of images with a VIN number in a vehicle record for the first vehicle; and the first set of images is stored in storage accessible by the system computer; the software of the system computer is further configured to receive from the second mobile device a VIN number for the first vehicle at a second location, the second location being a location different from the first location in that the first vehicle has been moved relative to the first location when imaged by the camera of the second mobile device, the second location being determined by the GPS receiver of the second mobile device; the software of the system computer is configured to receive a second set of images of the first vehicle imaged by the second mobile device and further associate each of the second set of images with a VIN number in a vehicle record for the first vehicle; and the second set of images is stored in the storage; The system wherein the first location and the second location are each capable of storing a plurality of vehicles, including the first vehicle, and are associated with different locations at various stages throughout a supply chain.

2. The system of claim 1 , wherein the plurality of mobile devices are robots having a ground movement device configured to move the robot over a ground surface while in contact with the ground.

3. the plurality of mobile devices are robots having a ground movement device configured to move the robot over a ground surface while in contact with the ground; 10. The system of claim 1, wherein the robot further includes at least one thruster configured to move a portion of the robot through the air to position a camera of the robot to capture multiple images of the first vehicle.

4. 10. The system of claim 1, wherein the plurality of mobile devices are computing devices selected from the group consisting of mobile phones, tablets, laptops, or other mobile devices that include a screen, a camera, a processor, and a user input device.

5. 5. The system of claim 4, wherein software executing on the computing device displays the VIN number and one or more prompts for capturing the first and second sets of images.

6. 6. The system of claim 5, wherein the one or more prompts displayed after capturing an image of the VIN number include a target displayed on a screen of each of the computing devices, the target being positioned and sized on the screen based on the software of the computing device that identifies the first vehicle via the camera and determines the position of the camera relative to the first vehicle and a default view of the first vehicle.

7. 7. The system of claim 6, wherein the screen displays an alignment target, and when the target and the alignment target are aligned, the first and second sets of images are taken by the software of the computing device.

8. 10. The system of claim 1, wherein the mobile software of the first and / or second mobile device is configured to receive an indication of damage corresponding to one or more images of the first and second sets of images and associate the indication of damage with the corresponding image.

9. The system of claim 1 , wherein the first and second locations are further associated with different responsible entities.

10. The system of claim 9, comprising software that provides a portal allowing access to the vehicle records, the vehicle records indicating various responsible parties.

11. 2. The system of claim 1, wherein the first and second mobile devices each comprise a plurality of mobile devices, and the first vehicle includes a plurality of vehicles, each of the plurality of vehicles having its own associated vehicle record stored in the storage.

12. 2. The system of claim 1, wherein the mobile software of the first and second mobile devices images the VIN number with the cameras of the first and second mobile devices by scanning using a barcode scan or visual character recognition, checks whether the string of letters and / or digits determined by the scan matches an expected VIN number pattern, and if the scan is unsuccessful or the string does not match the expected VIN number pattern, the mobile software of the corresponding mobile device generates control instructions to reposition the camera.

13. the first mobile device comprises a first plurality of mobile devices including a first mobile computer and a first robot; the second mobile device comprises a second plurality of mobile devices including a second mobile computer and a second robot; the first mobile computer images the VIN number of the first vehicle at the first location associated with a GPS location; the mobile software on the first mobile device is configured to send instructions to the first robot to orient the first robot to the GPS location such that the first robot moves to the GPS location or a location adjacent to the GPS location and captures the first set of the plurality of images of the first vehicle at the first location; the second mobile computer images the VIN number of the first vehicle at the second location associated with a second GPS location; 2. The system of claim 1, wherein the mobile software of the second mobile device is configured to send instructions to the second robot to orient the second robot to the second GPS location such that the second robot moves to the second GPS location or a location adjacent to the second GPS location and captures a second set of the plurality of images of the first vehicle at the second location.

14. 1. A multiple vehicle image capture system, comprising: a system computer that executes the software; a mobile computing device configured to communicate with the system computer over a network, the mobile computing device having a camera, a screen, and running mobile software, the mobile software configured to receive a VIN number for at least a first vehicle; the mobile software is configured to display one or more prompts, the prompts including a plurality of targets displayed on the screen, each of the targets positioned and sized on the screen based on the mobile software identifying the first vehicle through the camera and determining a position of the camera relative to the first vehicle and a default view of the first vehicle; When one of the targets is aligned with an alignment target displayed on the screen, the mobile software captures an image of the first vehicle with the camera, and the alignment target remains in a fixed position on the screen while the mobile computing device moves around the first vehicle and each of the targets moves on the screen.

15. 15. The system of claim 14, wherein the mobile software captures a plurality of images, each image associated with one of a plurality of the targets, and transmits the plurality of images over a network to the system computer where a vehicle record associated with the VIN number is associated with the plurality of images.

16. The system of claim 14 , wherein each of the plurality of images is associated with a location determined by a GPS receiver of the mobile computing device.

17. The system of claim 14 , wherein the mobile software displays one or more prompts for capturing an image of the VIN number via the camera.

18. the mobile computing device comprises a plurality of mobile computing devices; 16. The system of claim 15, wherein the mobile software is configured to receive a first set and a second set of images from different ones of the mobile computing devices and associate the first and second sets of images with the same vehicle record based on the VIN number.

19. 20. The system of claim 18, wherein the first and second sets of images are associated with different locations indicating that the same vehicle was at two different locations.

20. 20. The system of claim 19, wherein each of the plurality of mobile computing devices is associated with a GPS receiver, and the two different locations are determined based on the GPS receiver.

21. The system of claim 1 , wherein at least one or both of the different locations is a parking area.

Images