Defect mapping and repair system and method

Abstract

A system (100) for identifying and repairing one or more defects on a surface of an object (102). The system may include a robotic paint repair device (110) having a robotic arm (114) and a tool (120, 122) attached to the robotic arm. The tool may remove the one or more defects. The system may include a camera (124) disposed adjacent to the robotic paint repair device in a repair area. The camera is configured to scan a portion of the surface of the object to collect scan data including at least one of the one or more defects. The system may include a controller configured to control the camera to scan an area of ​​the surface based on first data representative of locations of the one or more defects on the surface of the object collected at a location distinct from the repair area.

Description

Priority claim

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 363,272, filed April 20, 2022, the contents of which are incorporated herein by reference. [Technical field]

[0002] The present disclosure relates to object inspection systems and methods for inspecting objects, and more particularly to object inspection and repair systems and methods for identifying defects at the location of robotically performed repairs using surface modification tools. [Background technology]

[0003] In the automotive industry, it is often necessary to prepare the surfaces of vehicle parts and replacement parts (e.g., bumpers) for various purposes (e.g., painting). It is also necessary to repair the surfaces of car parts and replacement parts due to defects that occurred during painting or coating. Typical surface preparation and repair processes include physical surface modification of the vehicle surface (e.g., sanding and polishing). A variety of tools, materials, and fluids can be used to prepare and repair surface defects.

[0004] In the automotive industry (e.g., automotive original equipment manufacturing (OEM) and aftermarket sectors), clear coat repair on mirror-finished surfaces is not automated. Technology is needed to automate this process and other coating applications (e.g., primer grinding, clear coat defect removal, clear coat polishing, etc.).

[0005] On the object detection side, historically manufactured objects were typically visually inspected by personnel to detect surface defects, imperfections, or other unwanted features. Visual inspection by personnel is costly (e.g., the need to pay personnel to visually inspect manufactured objects), causes worker fatigue from repetitive manual inspection and repair, and is unreliable due to varying detection rates based on the inspector's varying visual abilities. To address these issues, electronic systems have been introduced that use cameras, light sources, and computers to capture images of objects and use the images to detect defects. However, these systems perform the inspection prior to and at a location different from the repair location. From the point at which defects in a vehicle are identified by the system, errors are introduced in the confirmed location of those defects due to the vehicle's movement on carriages, rails, or assembly line structures, as well as vibrations and interactions with repair tools. This error can result in repairs being made on larger areas of the vehicle surface than desired. These larger areas can cause increased costs, delayed repair times, and other drawbacks. Summary of the Invention

[0006] Various examples have now been described to introduce in a simplified form a selection of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0007] The present disclosure describes systems, apparatus, methods, and techniques related to automating location error identification and defect identification repair for painting and other applications. Current processes involve time consuming manual human inspection and / or repair of the workpiece. As explained above, electronically implemented defect identification systems suffer from the drawback of introducing location errors between when the defect is located and when it is repaired. The inventors have invented systems, apparatus, methods, and techniques that allow for automated identification of defects on the surface of an object and more accurate automated repair of defects on the surface of an object. Additionally, the inventors have recognized that scanning at the time of repair can collect data that can be utilized for other purposes, such as improving understanding of the defect location and characteristics of the defect, such as the type, shape, or absence of the defect (false positive). Additionally, the inventors have recognized that scanning at the time of repair can collect data regarding one defect (e.g., location) that can be globally extrapolated to other defects on the object. This may reduce the need for scanning at the time of repair of all individual defects on the object. Rather, the coordinates of the various defects collected during the initial scan of the object may be re-identified at the time of repair, and this more accurate data may be used to effect repairs. Additionally, the various systems, devices, methods, and techniques may provide faster repair times, reduced tool wear and other waste, improved aesthetics, and increased line throughput, thus achieving a variety of realized and unrealized advantages.

[0008] The details of one or more examples of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating an assembly line with a painted vehicle, a defect scanning system, and a robotic repair device according to an example of the present application. [Figure 1A]FIG. 2 is a close-up view of an end effector portion of a robotic repair device showing one or more tools and a camera according to an example of the present application. [Diagram 2] FIG. 1 is a schematic diagram of a system including a controller, a robotic repair system with visual inspection capabilities, and an initial inspection system according to an example of the present application. [Diagram 3] FIG. 2 is a perspective view of a tracked object in combination with a conveyor assembly of the assembly line of FIG. 1, further illustrating the world coordinate system used according to an example of the present application. [Figure 4] FIG. 1 is a schematic diagram illustrating an example of a robotic paint repair system using a paint repair robot manipulating one or more surface modification tools and a second robot equipped with a camera according to an example of the present application. [Diagram 5] (A) and (B) are schematic diagrams of how a defect position error is introduced by moving an object from a first location to a second location and surface modification in the repair area, (C) further illustrates that the repair area is reduced using the systems, apparatus, methods and techniques, and (D) illustrates that the defect position can be used to globally reposition other defects according to an example of the present application. [Figure 6] 13(A)-13(C) are schematic diagrams of a method in which a position error of a second defect is introduced by vibration of an object and its vibration dynamics are analyzed to better repair the defect. [Figure 7] FIG. 1 is a schematic diagram of a robotic repair system including at least one defect repair robot equipped with a camera, where at least one defect repair robot scans a portion of a vehicle chassis while other defect repair robots operate grinding and polishing tools, according to an example of the present application. [Figure 8] 1 is a flowchart of a method for automated identification and repair of defects on an object according to an example of the present application.

[0010] In the drawings, like reference numerals refer to like elements. The above drawings, which may not be drawn to scale, illustrate various embodiments of the present disclosure. Other embodiments are contemplated as described in the detailed description. In all cases, the disclosure is presently disclosed as a representation of exemplary embodiments, and not by way of express limitation. Numerous other modifications and embodiments may be devised by those skilled in the art that are within the scope and spirit of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present disclosure provides an automated system and method for defect inspection for robotically performed repairs using an end-of-arm system with a tool for surface modification of an object surface. The automated system and method also allows for end-of-arm defect identification using one or more cameras. By way of example, the one or more cameras can be mounted approximately perpendicular, offset, or otherwise relative to the one or more tools performing the surface modification. The use of one or more cameras for defect inspection at the repair site provides various advantages, such as reduced processing time, as the size of the area in which the repair is performed can be reduced. The present application recognizes other advantages, such as improved aesthetics, reduced waste, reduced tool wear, and the collection of data that can be used for improved data analysis. Additionally, damage to the vehicle or end-of-arm system may be avoided by using the information collected by the system to avoid repairs that the system is not suited for (e.g., where the tool may accidentally collide with a nearby vehicle surface, or where no defects actually exist).

[0012] The one or more cameras can be mounted on the end effector of the motivation robot arm (with or without a tool assembly) for movement relative to the object. The surface modification tool may include a functional component configured to contact and prepare the object surface, and one or more sensors and / or actuators configured to detect working state information and / or change the working state of the end effector tool. Various sensors and / or actuators include force sensors, force torque sensors, force control units, etc.

[0013] Although one or more example embodiments are provided below, the disclosed systems and / or methods described with respect to Figures 1-8 may be implemented using any number of currently known or yet to be developed technologies. The present disclosure should not be limited to the exemplary embodiments, drawings, and technologies shown below, but may be modified within the scope of the appended claims and their full scope of equivalents.

[0014] The term "vehicle" as used herein is not limited to cars, but includes automobiles, trucks, boats, planes, helicopters, buses, or other vehicles. The term "defect" as used herein refers to unwanted features such as imperfections or unwanted surface conditions. This term "defect" includes hairs and other fibers in / on the painted surface, dust and other particles, scratches, dents, or other types of depressions in the painted surface, protrusions, raised areas in the painted surface, or areas of undesirable color such as a different color if the paint was not applied properly or cured properly. Examples of defects include micropops, which are small sub-millimeter sized areas of solvent pops breaking through the clear coat, orange peel, which is an uneven layer of paint, marl, which is an area where the vehicle has been touched, shallow paint, etc. The terms "board", "processor", "processing assembly", "server" (and other descriptive names used) may be used interchangeably, and these terms are intended to refer generally to processor-based entities not limited to a particular hardware or software configuration. The term "fluid" as used herein means any or combination of pure fluids, fluids containing particles (slurries, debris from surface modifications, etc.), or the like. "Surface modification" or similar terms include repairing, sanding, abrading, grinding, polishing, buffing, etc. of a surface. The term "substantially" means not more than 15% from the amount or value provided (e.g., between the exact value and 15 degrees from exactly parallel). The term "surface" is not limited to mirror-finished surfaces, but can include matte, metallic, or other types of finishes. In all cases, the surface need not be a painted surface.

[0015] The functions or algorithms described herein may be implemented in software in one embodiment. The software may be comprised of computer-executable instructions stored on a computer-readable medium or computer-readable storage device. These instructions may be stored in one or more non-transitory memories or other types of hardware-based storage devices, locally or on a network. Furthermore, such functions correspond to modules comprised of either software, hardware, firmware, or a combination thereof. Functions may be performed in one or more modules as desired, and the described embodiment is merely an example. The software may be executed in a digital signal processor, ASIC, microprocessor, or other type of processor, and may operate in a computer system, such as a personal computer, server, or other computer system, turning such a computer system into a specific programmed machine.

[0016] 1 is a schematic diagram illustrating an assembly line system 100 including a painted vehicle 102, a carriage 104, a rail system 106, a defect identification system 108, and a robotic repair machine 110. The robotic repair machine 110 can include a base 112, an arm 114, an end effector 116, and an assembly 118. As shown in FIG. 1A, the assembly 118 can include a first tool 120, a second tool 122, one or more cameras 124, and a light source 126.

[0017] FIG. 1 shows a painted vehicle 102 mounted on a carriage 104 in a known manner. The carriage 104 can be coupled to a rail system 106 for transporting the painted vehicle along the assembly line system 100 as indicated by arrow A. Although FIG. 1 shows a vehicle body or chassis, it is recognized that the present disclosure may use the systems, processes, techniques, and apparatus for any object (e.g., bumpers, hubcaps) and is not limited to the automotive field. Additionally, while FIG. 1 shows the assembly line system 100 as a continuous process, it is understood that the systems, processes, techniques, and apparatus may also be used in a non-continuous process (e.g., where some assembly is performed in one location, then production is stopped and the vehicle 102 is moved to another location (such as another facility) and assembly of other parts is performed at that other location). The rail system 106 is purely exemplary and may include various types of transport mechanisms. The rail system need not be continuously movable, but may be a stop station, a branch station, or other type of configuration known in the art. The carriage 104 may not be coupled to the rail system 106 in some embodiments.

[0018] After baking to harden the paint, the painted vehicle 102 (sometimes referred to herein as a chassis, body, or simply object) mounted on the carriage 104 can enter a defect identification system 108. The defect identification system 108 can globally scan all visible exterior surfaces of the painted vehicle 102 for defects, according to some examples. However, it is also possible to partially scan only some surfaces. Aspects of the defect identification system 108 are described in further detail in connection with Figures 2 and 3 of this disclosure. The defect identification system 108 can be constructed and operated in one or more manners described in U.S. Patent Application No. 15 / 932,865, filed May 9, 2018, and U.S. Patent Application No. 16 / 866,110, filed May 4, 2020, the entire disclosures of each of which are incorporated herein by reference.

[0019] The painted vehicle 102 can be moved from the defect identification system 108 along the rail system 106 to a defect repair location 128 (sometimes referred to herein as a defect repair area or a second location). Robotic repair of the defects in the painted vehicle 102 can be performed at the defect repair location 128 by a robotic repair device 110 disposed at the defect repair location 128.

[0020] The base 112 may be coupled to an arm 114 of the robotic repair machine 110. As shown in Figure 1, the robotic arm 114 may be movable in any of six dimensions relative to the base 112, and may translate or rotate along the x-, y-, and / or z-axes. The robotic repair machine 110 may include an end effector 116 having a force control unit (described below) and an assembly 118 attached thereto.

[0021] Returning to FIG. 1A, the first tool 120 and the second tool 122 can be configured to selectively interact with a surface, such as the surface of the painted vehicle 102. The first tool 120, in one embodiment, can be a back-up pad configured to hold an abrasive for polishing, grinding, etc., or other suitable abrasive tool. During the abrasive operation, the first tool 120 can abrade the surface of the painted vehicle 102 to remove material via an abrasive disc 130 or other suitable abrasive article. The first tool 120 can be attached to the assembly 118 using a suitable attachment system, such as an adhesive, hook and loop, clip system, vacuum, etc. Similarly, the second tool 122 can be a second abrasive tool (such as an abrasive pad 132) configured to polish or buff the surface of the painted vehicle 102. However, in other embodiments, the second tool 122 can have other configurations known in the art, such as a wiping medium, a fine-grained abrasive, a fluid removal tool (vacuum or air knife), etc. The second tool 122 may be attached to the assembly 118 using any suitable attachment system, such as adhesive, hook and loop, clip system, vacuum, or the like.

[0022] The assembly 118 may be configured such that the first tool 120 and the second tool 122 share substantially parallel (and substantially aligned) actuation axes A1 and B1. In other words, the first tool 120 may have a first axis A1 about which the first tool 120 is configured to rotate to perform surface modification on the workpiece. The second tool 122 may have a second axis B1 about which the second tool 122 is configured to rotate. The first axis A1 and the second axis B1 may be substantially aligned along the z-axis direction of the coordinate framework shown in FIG. 1 . However, the first tool 120 may be located on the opposite side of the assembly 118. The first tool 120 and the second tool 122 may be any or a combination of linear, rotary, orbital, or random orbital devices.

[0023] During a painting or clear coat repair process, fluid may be dispensed onto the work piece before, during, or after use of either the first tool 120 or the second tool 122. This process fluid may combine with particles from the process to create a slurry. The particles that make up this slurry are typically caused by an abrasive process, typically occurring prior to a polishing or buffing step (e.g., using the second tool 122). One or more wiping tools or embodiments (not shown) may be used to remove the slurry and / or excess liquid if desired.

[0024] The one or more cameras 124 may be mounted on the assembly 118 adjacent the first tool 120 and the second tool 122. The one or more cameras 124 may have an axis C1 (passing through the center of the lens) and be positioned substantially perpendicular to the first axis A1 and the second axis B1. However, the one or more cameras 124 may be positioned in any manner relative to the first tool 120 and the second tool 122 using one or more mirrors. It may be desirable to mount the one or more cameras 124 in a position close to the center of gravity of the assembly 118. This reduces the possibility of shading and / or unwanted vibration of the one or more cameras 124. The assembly 118 may be rotated as desired to bring either the first tool 120, the second tool 122, or the one or more cameras 124 into interface with the surface of the painted vehicle 102. With respect to the one or more cameras 124, the one or more cameras 124 can be positioned as desired by the robotic repair device 110 to scan the surface of the painted vehicle 102 in the desired area.

[0025] The one or more cameras 124 may be high resolution (e.g., 12 MP or greater) digital cameras. The one or more cameras 124 may have a zoom lens according to some examples. However, a zoom lens is not required in all examples, and in some examples the robotic repair machine 110 may move the position of the one or more cameras 124 as desired. Thus, the robotic repair machine 110 may move the one or more cameras 124 closer or farther away from the surface of the painted vehicle 102 as desired. In one example, the one or more cameras 124 may be configured to focus on an area of ​​approximately 150 mm by 150 mm at a distance of approximately 1 meter, although other area sizes and distances are contemplated, and the above areas and distances are provided for illustrative purposes. It is believed that defects in the painted vehicle 102 are typically less than 1 mm in size. 2Because the imaging area is less than 100 nm, various criteria such as resolution, zoom capability, distance, area, etc. can be manipulated as desired to achieve the desired result of identifying the presence of defects on the surface of the painted vehicle 102 using one or more cameras 124.

[0026] A light source 126 may be mounted in the assembly 118 adjacent to the one or more cameras 124. The light source 126 may be a generic white light according to some examples, although the characteristics of the light source 126 (e.g., size, color, location relative to the one or more cameras 124, etc.) may be varied as desired, as described in U.S. Patent Application Nos. 15 / 932,865 and 16 / 866,110.

[0027] 1A is purely exemplary. Other arrangements and locations relative to the first tool 120 and / or second tool 122 are also contemplated. For example, it is contemplated that the one or more cameras 124 and / or light source 126 may be mounted on a gantry system or other feature coupled to the robotic repair machine 110. Thus, the assembly 118 and / or end effector 116 need not be coupled to carry the one or more cameras 124 and / or light source 126 in some arrangements. In such arrangements, the gantry system or other feature may be manipulated to move with the movement of the arm 114.

[0028] By mounting to the assembly 118, end effector 116, force control unit (described below), and arm 114, the first and second tools 120, 122, and one or more cameras 124 can be positioned as desired within the degrees of freedom provided by the robotic repair machine 110 (most often six degrees of freedom) and other degrees of freedom within its frame of reference (e.g., a compliant force control unit). This arrangement allows the first and second tools 120, 122, and one or more cameras 124 to be positioned as desired for repair and imaging. The one or more cameras 124 and light source 126 can also be manipulated to sweep the surface of the painted vehicle 102 to collect images from multiple positions, as is known in defect identification systems.

[0029] 2 shows a schematic diagram of a system 200 including a controller 202, a defect identification system 108, and a robotic repair machine 110. The controller 202 can be in electronic communication with the defect identification system 108 and the robotic repair machine 110.

[0030] The defect identification system 108 may be configured to detect the presence of one or more defects on the surface of the painted transport vehicle 102 (FIG. 1), as described above. As shown in FIG. 2, the defect identification system 108 may include a first plurality of light sources 204 arranged along a direction or path along which the manufactured object is transported by the carrier (carriage in FIG. 1) and the conveyor (conveyor in FIG. 1). The first plurality of light sources 204 may each be selectively activated or energized and then configured to selectively and controllably emit light energy. As a non-limiting example, the first plurality of light sources 204 may each include a light emitting diode type light source, although other types of light sources may also be used. The first plurality of light sources 204 may be distributed around the periphery of the conveyor or moving assembly and may generate a substantially uniform amount of light around and on the object as it moves along the path or direction. The first plurality of light sources 204 may be arranged to generate a substantially uniform amount of intensity along this path or direction and on and around the object.

[0031] The defect identification system 108 may further include a plurality of inspection cameras 206 positioned along a direction or path along which the carrier transports the object 14. The plurality of inspection cameras 206 may cooperate to receive reflected light energy reflected from a surface of the object. Each of the plurality of inspection cameras 206 may be selectively energized and selectively activated after being energized.

[0032] The reflected light energy collected by the multiple inspection cameras 206 includes first data or image data (image information) regarding surface characteristics (e.g., visual characteristics). This first data can be used to detect defects on the surface of the object and to direct one or more cameras 206 (FIG. 1A) to scan specific portions of the surface of the object. The defect identification system 108 can be positioned to use the multiple inspection cameras 206 and light source 204 to capture the first data regarding substantially an entire visible surface of the object (e.g., the exterior surface of a vehicle).

[0033] The defect identification system 108 can have a dedicated processing assembly 208, which may be comprised of multiple distinct computer processors operating under stored program control, or a single computer processor assembly. Additionally, the processing assembly 208 can be a component of the controller 202.

[0034] The processing assembly 208 may have many functions and components not specifically shown, which may include an image tracking server or processor, a post-processing server or processor, a "NAS" or archive server or processor, a trigger board, and an encoder. The processing assembly may also include a simulator 210.

[0035] The encoder may be communicatively coupled to the tracking server. In a non-limiting embodiment, the encoder may comprise a commercially available friction wheel encoder manufactured and sold by Edon Controls, Inc. (Troy, Michigan). Other types of position encoders may also be used. The encoder is movably coupled to the conveyor or moving assembly, frictionally contacts the carrier, and rotates (e.g., spins) as the carrier moves along the conveyor or moving assembly. This rotation may provide the processing assembly 208 with continuous information regarding the position of the carrier, and therefore the position of the object as it moves along a path or direction. The simulator 210 may comprise a commercially available MATLAB® simulator and Simulink MathWorks® tool. The simulator 210 may be a separate processing system from the processing assembly 208. The simulator 210 may be communicatively coupled to a computer system and monitor that is remote from the processing assembly 208 in some examples. Thus, the simulator 210 may be communicatively coupled directly to the controller 202 in some examples.

[0036] Tracking may also be achieved using a camera-based vision tracking system, or using a 3D camera instead of the encoder. In the case of a camera-based vision system, a vision tracking server sends position data to the trigger board.

[0037] The processing assembly 208 can be electronically coupled to an output monitor and / or display assembly 212. The output monitor and / or display assembly 212 may include or be part of a display computer portion operating under stored program control. The output monitor and / or display assembly 212 may include multiple display computer portions. The processing assembly 208 can be electronically coupled to each of the multiple cameras 206, the one or more tracking cameras 214, and the one or more high speed cameras 216.

[0038] The image processing server or processor may be communicatively coupled to the image capture server or processor. The post-processing server or processor may be communicatively coupled to the image processing server or processor. The post-processing server or processor may be communicatively coupled to a "NAS" or archive server or processor. The trigger board, image server, image processing server, post-processing server, NAS, display computer portion, and tracking server are each connected to a communication network (e.g., an Ethernet network) via a switch, and thus may selectively communicate with each other via the network. The first plurality of light sources 204 may also be connected to the network. The plurality of cameras 206 and the plurality of light sources 204 may each be selectively "energized" or "activated" upon receiving a command from the trigger board or server.

[0039] Services such as Windows services (e.g., standalone programs) are also contemplated. Exemplary services include PLC services (communicating with PLCs to obtain plant data regarding vehicles), tracking services (communicating with trigger boards and other services to coordinate the scanning process), image capture services (obtaining images from cameras), image processing or GPU services (performing image processing steps to find defective areas in captured frames), classification services (where neural networks classify the found areas), cluster services (finding found areas on 3D surfaces, clustering multiple images of the same defect on the surface, determining the size of the defect, making a final decision on the type of defect, and creating images and data regarding the found defects). Services also include reporting services and overhead display services (to display images on output monitors). Such services are distributed across servers in various configurations.

[0040] The trigger board can be loaded with a table that maps the vehicle positions at which the cameras and light sources are triggered during scanning. The trigger board inputs the vehicle position from the encoder (or vision tracking system) and triggers the cameras and light sources at the specified positions. The trigger board can also have an input from a photo eye that is used to resynchronize to a predefined tracking synchronization position when the vehicle passes the photo eye. The trigger board can use the position information from the encoder to determine the identity and sequence of light sources to illuminate from the first plurality of light sources 204 and the identity and sequence of cameras to activate from the first plurality of cameras 206. In effect, a raw image of the surface of the object is captured as the object moves along a path or direction. The captured raw image data can be communicated to a processing assembly 208 (e.g., an image capture server). The captured raw image data can be communicated to other components (e.g., an image processing server, controller 202, etc.) and analyzed.

[0041] The controller 202 and / or the processing assembly 208 (e.g., via an image processing server and / or a post-processing server) execute a selected set of image processing algorithms that may cooperate effectively to process the captured raw image data (received from the raw image) to create first data (sometimes referred to as first scan data). The processed image (first data or first scan data) may include one or more regions of interest on the surface, or may include the entirety of the visible surface. The first data may also include information regarding one or more defects. Analysis may be performed to determine the identity (one or more characteristics) and location of each one or more defects on the surface. The first data may be communicated to the controller 202, a storage medium (e.g., a NAS server), and used as further described herein. Display of the first data (and indeed the scan data described below) may be in "real-time", near "real-time" (within a delay of less than four seconds), or retrieved from the storage medium for review.

[0042] The defect identification system 108 may also include one or more tracking cameras 214 coupled to a tracking server (a component of the processing assembly 208). These one or more tracking cameras 214 (and / or one or more high-speed cameras 216) may provide the tracking server with information about the position or location of the object as it moves with the movement of the conveyor or moving assembly. The one or more tracking cameras 214 and / or the high-speed cameras 216 may supplement or replace the encoders. The one or more tracking cameras 214 and / or the high-speed cameras 216 may be coupled to a trigger board and the processing assembly 208 (e.g., a tracking server). The one or more tracking cameras 214 collect object position information on the trigger board, which may be used alone or in combination with position information from the encoders. The one or more high-speed cameras 216 may collect "stereo information" (e.g., vibration information) using at least two cameras. The one or more high-speed cameras 216 are used to determine the orientation of the object within the carrier, which may be used if desired.

[0043] Although the defect identification system 108 can be very effective in identifying defects or potential defects, the defect identification system 108 can be a complex system with many components. The defect identification system 108 is typically not suitable for use near the robotic repair machine 110 due to potential shielding, vibration, or other interference. Additionally, the robotic repair machine 110 may require significant space. As previously discussed in FIG. 1, in typical practice, the vehicle (object) is transported to a location separate from the defect identification system 108 and the robotic repair machine 110 is run. Additionally, in some configurations, a single defect identification system 108 may be used to provide defect information for use by multiple downstream robotic repair machines 110.

[0044] The robotic repair machine 110 may be used to sand and polish one or more defects on a surface, as described above. The robotic repair machine 110 may have one or more cameras 124 (described above) that may be used to identify paint / clearcoat / matt or other defects to be repaired. The robotic repair machine 110 may include a movement mechanism 302 for moving the end-of-arm assembly 118 (FIG. 1) proximate to the defect repair area. The robotic repair machine 110 may include one or more sensors 304, such as force sensors or other sensors and / or actuators. The robotic repair machine 110 may include a dedicated controller that controls the movement and detection of the arm 114 and associated components. However, it is expressly contemplated that in some embodiments, the arm 114 and / or components attached thereto may have their own controller or be controlled by the controller 202. The dedicated controller may receive and execute movement and detection commands from the controller 202.

[0045] The end-of-arm assembly 118 can include various tools, as previously shown in Figure 1A, although it is expressly contemplated that in other embodiments, some components may be located elsewhere (not in assembly 118) and coupled to the arm 114, as shown in Figure 2.

[0046] The first tool 120 can be attached to the arm 114. The first tool 120 can be coupled to a first end effector 306 in some embodiments. In some examples, a second tool 122 can be attached to the arm 114. If a second tool 122 is used, it can be coupled to a second end effector 308. A fluid removal tool 310 can be attached to the arm 114, although it is expressly contemplated that in some examples some of these components can be attached to more than one or to a different arm than the arm 114. For example, the arm 114 can support the first tool 120 (e.g., a polishing tool) and the second arm can support one or more cameras 124, the second tool 122, and / or other components.

[0047] In one example, the arm 114 is moved into position by the arm movement mechanism 312. The first and second tools 120, 122, the one or more cameras 124, and the fluid removal tool 310 are also moved into position by the arm movement mechanism 312 in one embodiment or each can have their own movement mechanism for positioning on or near a surface. The robotic repair machine 110 can include a force control unit 314. The force control unit 314 can be disposed on the arm 114 to control the arm 114, the end effector system, and interaction with the work surface.

[0048] In some examples, an airline 316 and a fluid dispenser 318 may be provided from the arm 114 to the assembly 118 (FIG. 1A) to provide the air and fluid supply necessary for the operation of the first tool 120 and / or the second tool 122. The fluid removal tool 310 may also be coupled to the force control unit 314. The fluid removal tool 310 may be, for example, a cloth-based wiping medium, an air knife, a vacuum system, or other suitable tool. However, it is contemplated that in some embodiments the fluid removal tool 310 may be coupled to a force control unit separate from the force control units used for the first and second tools 120, 122. It is also contemplated that in other embodiments the fluid removal tool 310 may be a passive tool that does not have an associated force control unit. In some examples, the fluid removal tool 310 may be moved in space using mechanisms that control variables such as pitch, tilt, and yaw of the active wiping action of the fluid removal tool 310.

[0049] The force control unit 314 can maintain an appropriate force or pressure between the first tool 120, the second tool 122, and / or the fluid removal tool 310 and the surface of the object. The fluid removal tool 310 can, in some examples, work in conjunction with a fluid removal mechanism 322. The fluid removal mechanism 322 can be a pad, vacuum, brush, or scraping tool used to remove particles, debris, liquid, or slurry from the wiping media. The fluid removal mechanism 322 can help provide an appropriately absorbent and effective wiping media for multiple cleaning of the work surface.

[0050] The controller 202 can be in electronic communication with the camera (one or more cameras 124) and the robotic paint repair apparatus 110. The controller 202 can be configured to control the one or more cameras 124 to scan an area of ​​the surface (a portion of the exposed surface) based on the first data provided by the defect identification system 108. The scanning of the area by the one or more cameras 124 can collect and provide scan data representative of the location and / or other information of one or more defects on the surface of the object at the repair location. In contrast, the first data represents information collected by the defect identification system 108 at a location different from the repair location where the robotic repair apparatus 110 operates and where the one or more cameras 124 are located. The controller 202 is configured to operate the robotic arm (arm 114) to position a tool (either the first tool 120 or the second tool) based on the scan data collected by the one or more cameras 124 at the repair location.

[0051] The controller 202 is a digital controller having one or more processors and can be implemented in software or a combination of software and hardware. The controller 202 can have the functions and capabilities of the processing assembly 208 described above. Various other functions are contemplated, including as an interface between the defect identification system 108 and the robotic repair machine 110. The controller 202 can have various functions. These functions can be implemented in hardware, software, firmware, or a combination thereof and located locally or remotely. If implemented in software, these functions can be stored as one or more instructions or code on a computer-readable medium or stored on a computer-readable storage device and executed by a hardware-based processing unit. Computer-readable media includes computer-readable storage media corresponding to a data storage medium or communication media including any medium facilitating the transfer of a computer program (e.g., a signal or carrier wave following a communication protocol). In this manner, computer-readable media can correspond to (1) a non-transitory or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, code, and / or data structures used to implement the techniques described herein. A computer program product may include a computer-readable medium.

[0052] By way of example, a computer-readable storage medium includes RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that stores desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium.

[0053] However, computer readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are directed to non-transitory tangible storage media. Disks and disks include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, and Blu-ray disks, where disks typically reproduce data magnetically and disks reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer readable media.

[0054] The instructions may be executed by one or more processors (e.g., one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry). Accordingly, the term "processor" as used herein may refer to the above structures or any other structure suitable for implementing the techniques described herein. Moreover, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0055] The techniques of this disclosure may be implemented in a variety of devices or apparatuses, such as a wireless communication device or handset, a microprocessor, an integrated circuit (IC) or a set of ICs (e.g., a chipset). The various components, modules, or units are described to highlight functional aspects of devices configured to perform the techniques described in this disclosure, but do not necessarily need to be realized by different hardware units. Rather, as explained above, the various units may be combined in one hardware unit or provided with one or more processors in combination with appropriate software and / or firmware.

[0056] The functions, techniques, or algorithms described herein may be implemented in software, in one example. Software may be comprised of computer-executable instructions stored on a computer-readable medium or computer-readable storage device. These instructions may be stored in one or more non-transitory memories or other types of hardware-based storage devices, locally or on a network. Furthermore, such functions correspond to modules comprised of either software, hardware, firmware, or a combination thereof. Functions may be implemented in one or more modules as desired, and the examples described are merely examples. Software may be executed in a digital signal processor, ASIC, microprocessor, or other type of processor to operate in a computer system, such as a personal computer, server, or other computer system, and may turn such a computer system into a specific programmed machine.

[0057] FIG. 3 illustrates a global coordinate system 400 for inspecting an object 101 (here a painted vehicle 102 having a surface 103) as it is transported along a repeating object movement path or direction. The object movement path or direction can be defined in a three-dimensional world coordinate system with a world origin and world coordinate axes. As previously mentioned, the painted vehicle 102 can be mounted on a carriage 104 and coupled to a rail system 106 (only partially shown in FIG. 3). The movement path of the painted vehicle 102 can be the path that the painted vehicle 102 follows along the rail system 106. The world or global coordinate system illustrated in FIG. 3 can be defined by a point on the floor at the geometric center of the rail system 106 on which the carriage 104 and painted vehicle 102 reside, with the "z" axis pointing upward from the floor, the "y" axis pointing to the side of the conveyor, and the "x" axis pointing in a direction opposite to the conveyor's forward movement direction. FIG. 3 only provides an example of one coordinate system with one origin. The surface 103 may have one or more defects 105, as shown in Figure 3. The coordinate system of Figure 3 may be implemented by the controller 202 and / or processing assembly 208 of Figure 2 to understand the location of the one or more defects 105 relative to a global coordinate system 400. This location information may be captured as the first data described in connection with Figure 2.

[0058] FIG. 4 shows a schematic diagram of a robotic paint inspection and repair system 500 according to one example. Many aspects of the robotic paint inspection and repair system 500 have been described above in connection with FIGs. 1 and 2. The robotic paint inspection and repair system 500 can include a robotic repair unit 502 including a robotic arm 504 and a robotic inspection unit 506 (a second robot) including a robotic arm 508. The system can be controlled by a motion controller that receives instructions from one or more application controllers 510 (e.g., controller 202). The application controller 510 can receive input or provide output to a user interface 512.

[0059] The robotic repair unit 502 can include a force control unit 514 and an end effector 516 that can be aligned with the assembly 118. The robotic inspection unit 506 can include various components including one or more cameras 518 attached to the robotic arm 508. The one or more cameras can have a structure similar to the one or more cameras 124 described above. As shown in FIG. 4, the force control unit 514 can be coupled to the assembly 118 via the end effector 516. The assembly 118 can carry the first tool 120 and an optional second tool 122 described above. The first and second tools 120, 122 can be constructed in the manner described above. The one or more cameras 518 of the robotic inspection unit can perform a visual inspection on the surface 103 of the painted vehicle 102. If one or more defects are detected, they can be repaired by the robotic repair unit 502.

[0060] The robotic paint inspection and repair system 500 may differ from previous systems or devices in that the one or more cameras 518 are implemented on a robot assembly separate from the robotic repair equipment, although inspection is performed at the defect repair location 128, similar to the embodiment of Figure 1 discussed above. In other words, even if the one or more cameras 518 are not directly attached to the robotic repair unit 502, they are in close proximity to the robotic repair unit 502 as it performs polishing, buffing, etc.

[0061] The robotic repair unit 502 and the robotic inspection unit 506 may have a base fixed to a rail system configured to move with the vehicle being repaired. However, the arms 504 and 508 and other components are movable as described above. Depending on the location of the defect, the robotic repair unit 502 and the robotic inspection unit 506 may need to be moved closer to or further away from the vehicle, or higher or lower relative to the vehicle. FIG. 4 illustrates a Cartesian coordinate system showing x, y, and z axes for reference. This coordinate system is shared with the global coordinate system 400 of FIG. 3. It is recognized that in some instances, the robotic inspection unit 506 is not offset in the y-axis from the robotic repair unit 502. Rather, the robotic inspection unit 506 may be located elsewhere, such as on the same side as the robotic repair unit 502 and offset in the x-axis. The positions of the first and second tools 120, 122 may be altered by manipulation as described above.

[0062] Figures 5A and 5B show schematic diagrams of how position error of a defect in an object is introduced by movement from a first location to a second location and surface modification at the repair area. Figure 5C shows how the area where surface modification to remove the defect is performed is reduced at the repair location using one or more cameras 124 of arm 114 (Figure 1) or one or more cameras 518 of robotic inspection unit 506.

[0063] In particular, FIG. 5A first illustrates a defect 600 in a Cartesian coordinate system identified using the global coordinate system 400 by the defect identification system 108 (FIGS. 1-3). The defect 600 may have coordinates that are indicated when the defect is located in the defect detection area by the defect identification system 108. The use of a robot for defect repair requires very accurate position information regarding the location of the identified defect on the surface of the object. This position information may be captured as part of the first data discussed above. However, manufacturing typically involves assembly lines and moving objects from one manufacturing step to another, and the object may shift positionally as a result of the movement. By way of example, the object (vehicle) may shift in one or more of the x-, y-, and / or z-directions (and / or rotate along one or more of the x-, y-, or z-axes) during movement from a defect scanning location to a defect repair location. Alternatively, the rail system or other transport mechanism may shift in one or more of the z, x, or y directions (and / or rotate along one or more of the z, x, or y axes) during movement from the defect scan location to the defect repair location (see FIG. 1). Thus, the position of defect 600, which was precisely located at the defect scan location in FIG. 5A, may shift to a new position shown in FIG. 5B as a result of errors introduced during movement to the defect repair location. In other words, position errors may be introduced as the position of defect 600 moves from FIG. 5A to FIG. 5B.

[0064] It should also be noted that in some systems, the carriage (and therefore the object) continues to move while the repair process is taking place. In other words, the object is moving at the repair location and continues to move while the surface repair is being performed. This means that additional position errors continue to be introduced after the object arrives at the repair location. The systems, apparatus, and methods of the present disclosure can account for these ongoing position errors as new data is collected at the point of repair.

[0065] Typically (in previous systems), a relatively large area 602 or portion of the surface is targeted for surface modification to account for the position shift (position error) introduced in FIG. 5B. This relatively large area 602 ensures that the defect 600 is addressed even if the location is different from that recorded in the first data. This relatively large area 602 surface modification causes tool wear, material usage, increased processing time, and reduced aesthetics compared to surface modification of area 604. Furthermore, the large area 602 may prevent effective repair if a defect occurs. The surface modification of area 604 is relatively smaller than area 602. If the defect occurs near difficult areas such as edges or feature lines, a small repair area (area 604) may enable defect repair close to these areas.

[0066] Area 604 may be achieved by capturing and providing scan data (or second data) regarding the updated location of defect 600 at the repair location using one or more cameras 124 of arm 114 (FIG. 1) or one or more cameras 518 of robotic inspection unit 506.

[0067] A controller (e.g., controller 202) can be configured to use the first data and the scan data in various ways. For example, the controller can be configured to perform a comparison of the scan data and the first data. Based on the comparison of the scan data and the first data, the controller can update the first data with the location of one or more defects from the scan data (reflecting the updated locations in FIG. 5B).

[0068] FIG. 5D illustrates further determinations by the controller, including extrapolating or updating the positions of at least other defects 606. In particular, based on the update of the first data, the controller can be configured to relocate one or more defects on the entire (or a portion) of the object's surface, including the positions of one or more defects on a second portion of the surface outside the scan area of ​​the camera. For example, the controller can determine that the entire object has shifted and / or rotated in a particular manner and predict or determine the shift and / or rotation relative to the global set of defects originally identified. Based on the positions of one or more defects on the entire surface of the object, the controller can be configured to operate the robot arm to position a tool to perform surface modification on one or more defects on the second portion of the surface outside the scan by the camera. In other words, it is contemplated that the camera does not need to collect scan data for all defects on the object before surface modification is performed.

[0069] In addition to collecting and re-identifying the location data, the controller (e.g., controller 202) can be configured to collect one or more characteristics about the defect 600 in the scan data. Thus, the scan data can represent not only location information, but also characteristics such as size (depth, length, width), shape (protrusion, depression, unevenness), nature (color, hair, sand, dust), and confirmation of presence (e.g., that the defect 600 is actually present and not a false positive in the first data). Thus, based on a comparison of the scan data to the first data, the controller can update or supplement the first data to reflect one or more characteristics of the defect from the scan data. Because a more detailed set of information about the defect (including improved location data) can be collected at the time of repair, less precision may be required in some embodiments when collecting the first data. Thus, smaller, less expensive, or less sophisticated systems may be advantageously used to collect the first data.

[0070] The controller may also be configured to perform various analyses by comparing the scan data to the first data. Such analyses may indicate that the carriage may be loose on the rail system, causing increased relative positional movement. Such analyses may indicate that the accuracy of the first data has decreased because one of the cameras of the defect identification system 108 has become misaligned (perhaps due to a collision, misalignment, or wobble). The controller may issue an alert indicating the decrease in accuracy and notify personnel to check the carriage mounting, camera position, or other criteria of the system.

[0071] 6A-6C illustrate that position error is not only introduced by the movement of the object from a first location (FIG. 5A) to a second location (FIG. 5B). In addition, position error may be introduced by vibration due to the movement of the object at the repair location. FIG. 6C assumes that vibration is measured locally at the repair location. In particular, the scan data is based on multiple images taken at regular time intervals. Thus, as shown in FIG. 6A, a first image can be taken at a first time. As shown in FIG. 6B, a second image can be taken at a second time. The defect 700 in FIGS. 6A and 6B has shifted in position as a result of the vibration of the object at the repair location. As shown in FIG. 6C, a high-speed camera can capture multiple frames per second to capture the vibration dynamics of the object. The controller can be configured to determine information indicative of a shift in position of the defect 700 due to vibration of the object. This information can be used to improve the positioning of the tool on the robotic repair equipment, as well as to optimize other criteria such as optimizing the contact force of the tool, modifying the force, modifying the repair area (the repair size can be scaled by understanding the vibration direction and amplitude), modifying the repair tool speed, and skipping the repair if the vibration interferes with the repair when performing the repair. Information about the type of defect (e.g. stain / crater / micropop / orange peel, etc.) may be better estimated by the robot performing a second scan, since the robot may obtain a higher resolution image than that obtained in the first scan. Information about the true size of the defect may also be improved, since the defect size, height, and depth estimates from the first scan may be affected by shadows, which are better controlled by the moving robot arm. This information helps the robot perform better repairs and avoid attempting repairs if the defect is misclassified. The higher quality defect data from the second scan obtained by the robot also improves the quality of the data sent to the factory data analytics system, which is used to drive continuous improvement of the painting process.As an example, the initial defect detection system may contain misclassifications, such as reporting a 10% chance that a defect exists. The second scan helps to avoid repair attempts in areas where the initial detection confidence is low.

[0072] FIG. 7 shows assemblies 118A, 118B, and 118C, identical to the assembly 118 described above in FIG. 1 and FIG. 1A, mounted on a robot for operation. FIG. 7 shows further aspects considered in the present disclosure. First, one or more cameras 124 of assembly 118A are shown scanning a portion 802 of a surface 804 of a vehicle 102 in a defect repair location 128. This portion 802 (corresponding to a scan area) can include at least a first defect 806 and a second defect 808. The relative positions of the first defect 806 and the second defect 808 can be used for global realignment without the need to scan other defects (e.g., defect 810), as described above in FIG. 5D. Assembly 118B shows a first tool 120 performing a surface modification to remove defect 810. Assembly 118C shows a surface modification (e.g., polishing) being performed.

[0073] 7 shows scan data 812 being collected and transmitted as the vehicle 102 moves along the assembly line as indicated by arrow A2. The assemblies 118A, 118B, 118C may be manipulated in coordination by a robot to address defects 806, 808, 810, etc. Assembly 118A shows the robotic arm being manipulated to zoom one or more cameras 124 into a surface 804 to collect scan data.

[0074] FIG. 8 illustrates a method 900 for identifying and repairing one or more defects on a surface of an object. The method 900 includes collecting first data representative of the location of one or more defects on the surface of the object at a first location. This can be performed at the location of the defect identification system 108 (FIGS. 1 and 2) described above. The method 900 can include moving the object from the first location to a second location where repair of the one or more defects is performed by a robotic paint repair device. The method 900 can include scanning a portion of the surface of the object at the second location based on the first data. The method 900 can include contacting the surface of the object using a tool to perform a surface modification to remove the one or more defects. The location of contacting the surface is determined at least in part by scanning a portion of the surface of the object at the second location.

[0075] Method 900 may optionally include other steps, including locating one or more defects substantially across the surface of the object from the first data. Method 900 may optionally include comparing the scan data to the first data, updating the first data with the location of the one or more defects from the scan data, and relocating the one or more defects across the surface of the object. Contacting the surface to perform the surface modification occurs at a second portion outside of the scanning range of the portion of the surface of the object. Updating the first data includes updating the first data to reflect characteristics of the one or more defects from the scan data. Scanning at the second location includes using a robotic arm to move a camera closer to or further from the object to change the size of the scan area and the scan data collected. Scanning at the second location includes taking multiple images at regular time intervals. Method 900 may optionally include determining that the location of the one or more defects shifts due to vibration of the object. Method 900 may optionally include determining that moving the object from a first location to a second location causes a shift in position of the one or more defects, and determining that vibration of the object causes a shift in position of the one or more defects.

[0076] The present disclosure includes, but is not limited to, the following illustrative examples.

[0077] Example 1 is a system for identifying and repairing one or more defects on a surface of an object, the system may include any one or combination of a robotic paint repair device, a camera, and a controller. The robotic paint repair device may have a robotic arm and a tool attached to the robotic arm. The tool is configured to contact the surface of the object to perform a surface modification to remove the one or more defects. The camera is disposed adjacent to the robotic paint repair device in the repair area and configured to scan a portion of the surface of the object to collect scan data including at least one of the one or more defects. The controller is configured to communicate with the camera and the robotic paint repair device. The controller is configured to control the camera to scan an area of ​​the surface based on first data representative of a location of the one or more defects on the surface of the object collected at a location different from the repair area. The controller is configured to operate the robotic arm to position the tool based at least on the scan data.

[0078] Example 2 is the system of Example 1 where the camera is mounted on a second robot or mounted on the robotic paint repair device.

[0079] Example 3 is the system of any or a combination of Examples 1-2, wherein the controller is configured to control either the second robot or the robotic paint repair device to adjust the position of the camera relative to the object.

[0080] Example 4 is the system of any one or combination of Examples 1-3, wherein the controller is configured to perform a comparison between the scan data and the first data.

[0081] Example 5 is the system of any or combination of Examples 1-4, wherein the controller updates the first data with the location of the one or more defects from the scan data based on a comparison of the scan data and the first data.

[0082] Example 6 is the system of any or combination of Examples 1-5, wherein the controller is configured to re-determine a location of one or more defects on the portion of the surface of the object based on an update of the first data.

[0083] Example 7 is the system of any or combination of Examples 1-6, wherein the controller is configured to re-determine locations of one or more defects across a surface of the object based on an update of the first data.

[0084] Example 8 is the system of any or combination of Examples 1-7, wherein the controller is configured to operate the robotic arm to position a tool to perform surface modifications on one or more defects on a second portion of the surface outside the scan by the camera.

[0085] Example 9 is the system of any or a combination of Examples 1-7, wherein based on a comparison of the scan data and the first data, the controller updates the first data to reflect characteristics of one or more defects from the scan data.

[0086] Example 10 is the system of any one or combination of Examples 1 to 9, wherein the controller issues an alert based on a comparison between the scan data and the first data.

[0087] Example 11 is the system of any or a combination of Examples 1 to 10, wherein the scan data is based on a plurality of images taken at regular time intervals, and the controller is configured to determine a shift in position of one or more defects due to vibration of the object based on the scan data.

[0088] Example 12 is the system of any or a combination of Examples 1-11, wherein the object includes a vehicle, the surface includes a mirror-finished surface, and the first data is collected before the vehicle moves along the assembly line to a repair area.

[0089] Example 13 is the system of any or a combination of Examples 1-12, wherein the vehicle is moving along the assembly line during repair and the first data and the scan data are collected while the vehicle is moving along the assembly line.

[0090] Example 14 is a method of identifying and repairing one or more defects on a surface of an object, the method including: collecting first data indicative of the location of one or more defects on the surface of the object at a first location; moving the object from the first location to a second location where repair of the one or more defects is performed by a robotic paint repair device at the second location; scanning a portion of the surface of the object based on the first data at the second location; and contacting the surface of the object using a tool to perform a surface modification to remove the one or more defects, where the location of contacting the surface is determined at least in part by scanning the portion of the surface of the object at the second location.

[0091] Example 15 is the method of Example 14, further including comparing the scan data to the first data, updating the first data with locations of the one or more defects from the scan data, and re-determining locations of the one or more defects across an entire surface of the object.

[0092] Example 16 is a method of any one or combination of Examples 14 to 15, characterized in that the step of contacting the surface to perform surface modification is performed in a second portion outside the range of scanning the portion of the object's surface.

[0093] Example 17 is the method of any one or combination of Examples 14-16, further comprising comparing the scan data to the first data and updating the first data to reflect characteristics of one or more defects from the scan data.

[0094] Example 18 is the method of any one or combination of Examples 14-17, wherein scanning the portion of the surface at the second location includes moving the camera toward or away from the object using a robotic arm.

[0095] Example 19 is the method of any one or combination of Examples 14-18, further comprising taking a plurality of images at a time interval at the second location and determining a shift in position of the one or more defects due to vibration of the object.

[0096] Example 20 is the method of any one or combination of Examples 14-19, further including determining a shift in position of the one or more defects due to moving the object from a first location to a second location, and determining a shift in position of the one or more defects due to vibration of the object.

[0097] Example 21 is the method of any one or combination of Examples 14-20, further comprising moving the vehicle along the assembly line while contacting a surface of the object to perform surface modification to remove one or more defects.

[0098] Example 22 is the method of any or a combination of Examples 14-21, further comprising: moving the vehicle along the assembly line while collecting first data indicative of locations of one or more defects on a surface of the object at a first location and scanning a portion of the surface of the object based on the first data at a second location.

[0099] Example 23 is a method of identifying and repairing one or more defects on a surface of an object, the method including: scanning at a first location to collect first scan data; determining from the first scan data the locations of one or more defects on substantially an entire surface of the object; moving the object from the first location to a second location, where repairs to the one or more defects are performed by a robotic paint repair device; scanning only a portion of the surface of the object at the second location to collect second scan data, where the portion of the surface to be scanned is selected based on the first scan data; comparing the first scan data and the second scan data; and updating the locations of the one or more defects to reflect a shift in the locations of the one or more defects based on the comparison; and contacting the surface of the object using a tool to perform a surface modification to remove the one or more defects, where a location of contacting the surface is determined based on the shift in the locations of the defects.

[0100] Example 24 is the method of example 23, wherein updating the positions of the one or more defects to reflect a shift in the positions of the one or more defects includes redetermining positions of the one or more defects on an entire surface of the object.

[0101] Example 25 is the method of any one or combination of Examples 23-24, wherein the step of contacting the surface to perform the surface modification occurs in a second portion outside of the range of scanning the portion of the object's surface.

[0102] Example 26 is the method of any or combination of Examples 23-25, further comprising updating the first scan data to reflect characteristics of the one or more defects from the second scan data.

[0103] Example 27 is the method of any one or combination of Examples 23-26, wherein the step of scanning only a portion of the surface at the second position includes moving the camera toward or away from the object with a robotic arm.

[0104] Example 28 is the method of any one or combination of Examples 23-27, further comprising taking a plurality of images at a time interval at the second location and determining a shift in position of the one or more defects due to vibration of the object.

[0105] Example 29 is the method of any one or combination of Examples 23-28, further comprising moving the vehicle along the assembly line while contacting a surface of the object to perform a surface modification to remove one or more defects.

[0106] Example 30 is the method of any one or combination of Examples 23-29, further comprising moving the vehicle along the assembly line while scanning at a first location to collect first scan data and scanning only a portion of a surface of the object at a second location to collect second scan data.

[0107] The various embodiments 1 to 30 described above can be combined in any combination. The elements can be combined in any combination. Unless otherwise specified, these elements are optional.

[0108] Various examples are described. These and other examples are within the scope of the following claims.

Claims

1. 1. A system for identifying and repairing one or more defects on a surface of an object, comprising: a robotic paint repair apparatus having a robotic arm and a tool attached to the robotic arm, the tool configured to contact the surface and perform a surface modification on the object to remove the one or more defects; a camera positioned adjacent to the robotic paint repair device in a repair area, the camera configured to scan a portion of the surface of the object including at least one of the one or more defects to collect scan data; a controller in communication with the camera and the robotic paint repair device; The controller is configured to control the camera to scan the area of ​​the surface based on first data representing the location of the one or more defects on the surface of the object collected at a location different from a repair area, and the controller is configured to operate the robot arm to position the tool based on at least the scan data.

2. The system of claim 1 , wherein the camera is mounted to a second robot or is mounted to the robotic paint repair machine.

3. The system of claim 2 , wherein the controller is configured to control either the second robot or the robotic paint repair device to adjust a position of the camera relative to the object.

4. The system of claim 1 , wherein the controller is configured to perform a comparison between the scan data and the first data.

5. The system of claim 4 , wherein the controller updates the first data with the locations of the one or more defects from the scan data based on a comparison of the scan data and the first data.

6. 6. The system of claim 5, wherein based on the updates to the first data, the controller is configured to redetermine locations of each of the one or more defects on portions of the surface of the object that include less than the entire surface of the object, the portions of the surface of the object including locations of the one or more defects on a second portion of the surface outside the scan of the camera.

7. 6. The system of claim 5, wherein based on the updates to the first data, the controller is configured to redetermine the positions of each of the one or more defects on the entire surface of the object, including positions of the one or more defects on a second portion of the surface outside the scan of the camera.

8. 8. The system of claim 7, wherein based on the position of each of the one or more defects on the entire surface of the object redetermined by the controller, the controller is configured to manipulate the robot arm to position the tool to perform the surface modifications on those of the one or more defects on the second portion of the surface outside the scan by the camera.

9. The system of claim 4 , wherein based on a comparison of the scan data and the first data, the controller updates the first data to reflect characteristics of one or more defects from the scan data.

10. The system of claim 4 , wherein the controller issues an alert based on a comparison of the scan data and the first data.

11. 2. The system of claim 1, wherein the scan data is based on a plurality of images taken at regular time intervals, and wherein based on the scan data, the controller is configured to determine a shift in position of the one or more defects due to vibration of the object.

12. 2. The system of claim 1, wherein the object comprises a vehicle and the surface comprises a mirror-finished surface, and the first data is collected before the vehicle moves along an assembly line to the repair area.

13. 12. The system of claim 11, wherein the vehicle is moving along an assembly line during repair, and the first data and the scan data are collected while the vehicle is moving along the assembly line.

14. 1. A method for identifying and repairing one or more defects on a surface of an object, comprising: collecting first data representative of locations of the one or more defects on the surface of the object at a first location; moving the object from the first location to a second location, where repairs of the one or more defects are performed by a robotic paint repair device having a robotic arm and a tool attached to the robotic arm; scanning a portion of the surface of the object based on the first data at the second location; contacting the surface using the tool to perform a surface modification of the object to remove the one or more defects, the location of the contacting the surface being determined, at least in part, by scanning the portion of the surface of the object at the second location; The method includes:

15. comparing the scan data with the first data; updating the first data with locations of the one or more defects from the scan data; relocating the one or more defects across the surface of the object; The method of claim 14 further comprising:

16. The method of claim 15 , wherein the step of contacting a surface to effect the surface modification occurs in a second portion outside of scanning the portion of the surface of the object.

17. comparing the scan data with the first data; updating the first data to reflect characteristics of the one or more defects from the scan data; The method of claim 14 further comprising:

18. 15. The method of claim 14, wherein the step of scanning the portion of the surface at the second location comprises using the robotic arm to move a camera towards or away from the object.

19. 15. The method of claim 14, wherein scanning the portion of the surface at the second location comprises taking a plurality of images spaced apart over a duration, and further comprising determining a shift in position of the one or more defects due to vibration of the object.

20. 20. The method of claim 19, further comprising determining a shift in position of the one or more defects due to moving the object from the first location to the second location, and determining a shift in position of the one or more defects due to vibration of the object.

21. 15. The method of claim 14, further comprising moving the vehicle along an assembly line while contacting the surface to perform the surface modification of the object with the tool to remove the one or more defects.

22. 22. The method of claim 21, further comprising: moving the transport vehicle along an assembly line while collecting the first data representative of locations of the one or more defects on the surface of the object at the first location and scanning the portion of the surface of the object based on the first data at the second location.

23. 1. A method for identifying and repairing one or more defects on a surface of an object, comprising: scanning at a first location to collect first scan data; determining locations of the one or more defects on substantially the entire surface of the object from the first scan data; moving the object from the first location to a second location, where repair of the one or more defects is performed by a robotic paint repair device having a robotic arm and a tool attached to the robotic arm; scanning only a portion of the surface of the object at the second location to collect second scan data, the portion of the surface selected for scanning being based on the first scan data; comparing the first scan data and the second scan data; updating a position of the one or more defects to reflect a shift in position of the one or more defects based on the comparison; contacting the surface with the tool to perform a surface modification of the object to remove the one or more defects, the location of the contacting the surface being determined based on the shift in the position of the defect; The method includes:

24. 24. The method of claim 23, wherein updating the positions of the one or more defects to reflect a shift in the positions of the one or more defects comprises re-determining the position of each of the one or more defects across the entire surface of the object.

25. 25. The method of claim 24, wherein the step of contacting the surface to effect the surface modification occurs on a second portion of the surface outside of scanning the portion of the surface of the object.

26. 24. The method of claim 23, further comprising updating the first scan data to reflect characteristics of the one or more defects from the second scan data.

27. 24. The method of claim 23, wherein scanning only the portion of the surface at the second location comprises moving a camera with the robotic arm towards or away from the object.

28. 24. The method of claim 23, wherein scanning only the portion of the surface at the second location comprises taking a plurality of images spaced apart over a duration, and further comprising determining a shift in position of the one or more defects due to vibration of the object.

29. 24. The method of claim 23, further comprising moving the vehicle along an assembly line while contacting the surface to perform the surface modification of the object with the tool to remove the one or more defects.

30. 30. The method of claim 29, further comprising moving the vehicle along an assembly line while scanning at the first location to collect the first scan data and scanning only the portion of the surface of the object at the second location to collect the second scan data.

Images