SYSTEM AND METHOD FOR TREATING A WORK SURFACE - Patent application

Abstract

A robotic system is presented that includes a surface inspection system that receives a plurality of sampling points within a region of a workpiece surface. The system also includes a robot arm coupled to a surface treatment tool, the robot arm configured to engage the surface treatment tool with the region of the workpiece surface. The system also includes a process mapping system that is configured to approximate a surface topography of the region of the workpiece surface based on the plurality of sampling points, modify a trajectory of the robot arm based on the approximated surface topography, and generate control signals for the robot arm that include a path for the robot arm to enter the region.

Description

[Background technology]

[0001] Clearcoat repair is one of the final operations to be automated in the automotive original equipment manufacturing (OEM) sector. Technology is desired to automate this process as well as other surface treatment applications including paint applications (e.g., primer sanding, clearcoat defect removal, clearcoat polishing, etc.), adhesive dispensing, film wrapping application, or material removal systems that are amenable to the use of abrasives and / or robotic inspection and repair. Defect repair presents many challenges for automation. Summary of the Invention

[0002] A robotic system is presented that includes a surface inspection system that receives a plurality of sampling points within a region of a workpiece surface. The system also includes a robot arm coupled to a surface treatment tool, the robot arm configured to engage the surface treatment tool with the region of the workpiece surface. The system also includes a process mapping system configured to approximate a surface topography of the region of the workpiece surface based on the plurality of sampling points, modify a trajectory of the robot arm based on the approximated surface topography, and generate a control signal for the robot arm that includes a path for the robot arm to enter the region.

[0003] The systems and methods herein can enhance the ability of a robotic surface treatment unit to perform surface treatment operations on non-flat work surfaces, such as surfaces having edges or curvatures, thereby improving overall surface treatment efficiency and reducing the number of operations that need to be completed manually. [Brief description of the drawings]

[0004] The drawings, which are not necessarily drawn to scale and in different drawings, like numerals may describe like elements, illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in the present document.

[0005] [Figure 1] 1 is a schematic diagram of a robotic surface treatment system in which embodiments of the present invention are useful;

[0006] [Diagram 2] 1 illustrates a robotic defect repair method in which embodiments of the present invention may be useful.

[0007] [Diagram 3] 1 illustrates a work surface having a plurality of surface features in which embodiments herein may be useful.

[0008] [Figure 4] 1 illustrates a method for treating a workpiece surface according to an embodiment herein.

[0009] [Diagram 5] 1 illustrates a method for identifying a surface topography on a workpiece surface according to an embodiment herein.

[0010] [Figure 6] 1 illustrates a process mapping system according to an embodiment of the present disclosure.

[0011] [Figure 7A] 1 illustrates the movement of a robotic repair unit over a surface feature. [Figure 7B] 1 illustrates the movement of a robotic repair unit over a surface feature. [Figure 7C] 1 illustrates the movement of a robotic repair unit over a surface feature.

[0012] [Figure 8] 1 is a repair plan generation system architecture.

[0013] [Figure 9] 2 illustrates an example of a computing device that may be used in the embodiments illustrated in the preceding figures. [Figure 10] 2 illustrates an example of a computing device that may be used in the embodiments illustrated in the preceding figures. [Figure 11] 2 illustrates an example of a computing device that may be used in the embodiments illustrated in the preceding figures.

[0014] [Figure 12A] An example of a surface treatment calculation is shown below. [Figure 12B] An example of a surface treatment calculation is shown below. [Figure 12C] An example of a surface treatment calculation is shown below. [Figure 12D] An example of a surface treatment calculation is shown below. [Figure 12E] An example of a surface treatment calculation is shown below. [Figure 13A] An example of a surface treatment calculation is shown below. [Figure 13B] An example of a surface treatment calculation is shown below.

[0015] [Figure 14A] 1 shows the results of mathematically modeling a workpiece surface having surface features with a collection of small points. [Figure 14B] 1 shows the results of mathematically modeling a workpiece surface having surface features with a collection of small points. [Figure 14C] 1 shows the results of mathematically modeling a workpiece surface having surface features with a collection of small points. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Recent advances in imaging technology and computing systems have made the process of clearcoat inspection at production speeds feasible. In particular, stereo deflection metrology has recently been shown to be capable of providing images and locations of paint and clearcoat defects with adequate resolution, along with spatial information (providing coordinate location information and defect classification) to enable subsequent automated spot repair. As automated imaging of workpiece surfaces improves, it is equally desirable to improve the ability to automatically process workpiece surfaces. For example, in the case of clearcoat repair, it is desirable to use a robotic repair system to repair detected defects with as little manual intervention as possible. However, as described herein, workpiece surfaces that exhibit a high degree of curvature (significant deviation from being a flat surface) are particularly difficult to process using robotic systems. Additionally, the presence of sharp surface features (such as sharp bends, grooves, etc.) near the desired repair area can further complicate the task of performing the automated repair. While humans intuitively know how to modify their approach to deal with convex or concave surfaces or edges on a surface, a robotic system must be trained on how to learn about surface features and adjust pre-programmed trajectories to accommodate them when they are detected. The systems and methods herein may be useful in any surface processing application where a surface processing task must be mapped onto a surface that exhibits significant curvature and / or surface features.

[0017] As used herein, the term "vehicle" is intended to encompass a wide range of moving structures that are coated with at least one paint or clear coat during manufacture. While many examples herein relate to automobiles, it is expressly contemplated that the methods and systems described herein are also applicable to trucks, trains, boats (with or without motors), airplanes, helicopters, and the like. In addition, while vehicles are described as examples for which embodiments herein are particularly useful, it is expressly contemplated that some of the systems and methods herein may be applied to surface treatment in other industries, such as painting, adhesive treatment, or material removal, such as sanding or polishing of wood, plastics, paints, and the like.

[0018] The term "paint" is used herein broadly to refer to any of the various layers of a vehicle's e-coat, filler, primer, paint, clear coat, etc., applied in the finishing process. Moreover, the term "paint repair" includes locating and repairing any visual artifacts (defects) on or within any paint layer. In some embodiments, the systems and methods described herein use clear coat as the target paint repair layer. However, the systems and methods presented apply with little or no modification to any particular paint layer (e-coat, filler, primer, paint, clear coat, etc.).

[0019] As used herein, the term "defect" refers to an area on a work surface that interferes with visual aesthetics. For example, many vehicles have a shiny or metallic appearance after painting is completed. A "defect" may include debris trapped within one or more of the various paint layers on the work surface. A defect may also include excess paint, including stains, smudges or drips within the paint, as well as dents.

[0020] Paint repair is one of the last remaining steps in the vehicle manufacturing process that remains largely manual. Historically, this is due to two main factors: the lack of sufficient automated inspection and the difficulty of automating the repair process itself. Humans who interact with surfaces have an intuitive understanding of how to respond to surface curvatures, edges, and other features. Robotic surface treatment systems require training.

[0021] FIG. 1 is a schematic diagram of a robotic paint repair system in which embodiments of the present invention are useful. The system 100 generally includes two units: a visual inspection system 110 and a defect repair system 120. Both systems can be controlled by motion controllers 112, 122, respectively, which can receive instructions from one or more application controllers 150. The application controllers can receive input from or provide output to a user interface 160. The repair unit 120 includes a force control unit 124 that can be associated with an end effector 126. As shown in FIG. 1, the end effector 126 includes two processing tools 128. However, other arrangements are also expressly contemplated.

[0022] The current state of the art in vehicle paint repair is to manually sand / polish the imperfections using fine abrasives and / or polishing systems with or without the assistance of power tools all the way through while maintaining the desired finish (e.g., comparable to mirror finish in a clear coat). A skilled person performing such repairs utilizes extensive training as well as their senses to monitor the progress of the repair and make changes accordingly. Such advanced techniques are difficult to obtain in a robotic solution where sensing is limited.

[0023] Moreover, while abrasive material removal is a pressure-driven process, many industrial manipulators typically operate natively in a position tracking / control regime and are optimized with positional accuracy in mind. This results in extremely rigid systems with extremely stiff error response curves (i.e., small misalignments result in extremely large corrective forces) that are inherently poor at force control (i.e., joint torques and / or orthogonal forces). Closed-loop force control techniques have been used (with limited utility) to address the latter, with more recent (and more successful) force-controlled flanges providing softer (i.e., less rigid) displacement curves that are much more suitable for sensitive force / pressure-driven processes. However, the problem of robust process planning / control remains and is the focus of this research.

[0024] 2 illustrates a robotic defect repair method in which embodiments of the invention may be useful. Method 200 is an overview of how a robotic repair system repairs defects in accordance with at least some embodiments described herein.

[0025] At block 210, instructions are received from a robot controller, such as, for example, application controller 150 of Figure 1. The instructions include movement instructions for different components of the robot repair unit, such as the force control unit, the end effector motors, or the tool movement pattern.

[0026] At block 220, a robotic motion controller moves an abrasive article attached to the tool into position and prepares it to engage the defect. The defect location may be known from the inspection system or may otherwise be identified based on, for example, a CAD file of the workpiece surface.

[0027] An abrasive article is engaged with the defect at block 230. Engaging the defect may include sanding the defective area or polishing the defective area.

[0028] At block 240, the defect area is cleaned. Cleaning may include wiping off any fluids used in sanding or polishing, as well as wiping off debris. After the cleaning step, the tool may be re-engaged with the defect, as shown in block 342.

[0029] At block 250, the defective area is inspected to determine if the repair is sufficient. If additional repair is required, method 200 may include the robotic repair unit receiving new instructions, as indicated by arrow 260, and the method may be repeated. Inspecting the defect repair may include obtaining 252 a post-repair image, which may be presented to a repair operator or stored as desired. Inspecting may also include verifying the repair, as indicated at block 254, which may include comparing pre-repair and post-repair images, detecting whether the defect is visible / noticeable to the human eye, or another suitable verification technique.

[0030] FIG. 3 illustrates a work surface having multiple surface features, for which embodiments herein may be useful. The work surface 300, shown in FIG. 3 as a car hood, includes a surface having multiple surface features 302 and some flat regions 304. Currently, defects are detected by an inspection system and provided to a human reviewer, who indicates whether the defect is in a repairable or non-repairable region. Repairable defects 310 are defects that are in regions on the work surface 300 that a robotic repair unit can repair using a repair trajectory based on a flat repair surface. In contrast, defects that are too close to the surface features 302 are considered manual repair defects 320, or defects that cannot be processed by the robotic surface processing system and therefore must be repaired manually. Currently, in many applications, imaged defects are mapped to regions of the vehicle using predefined routines and "repair / no repair" criteria. For example, "steep" regions or regions on curved (concave or convex) surfaces are "no repair" regions.

[0031] It would be desirable to have a system or method that allows a robotic surface processing system, such as system 100, to process a surface proximate to or over an area that includes a surface feature. It would also be desirable to have a system that can determine the surface topography proximate to a defect location without access to a CAD (computer-aided design) model of the vehicle or other work surface, since it is not always possible to obtain such a model from the vehicle manufacturer. For example, when repairs are made in an aftermarket environment, such as a repair shop, the only way to gather surface data may be through the use of direct measurement.

[0032] As described herein, in some embodiments, a defect is identified and a number of points on the surface around the defect are measured. Thus, a point cloud is obtained and based on this point cloud, a topography of the surface is determined. In some embodiments, based on the surface topography, surface features are identified. In some embodiments, a set of curvature metrics are calculated at a number of sampling points obtained from a model fitted from the measurements. At each sampling point, these metrics are calculated with respect to two orthogonal vectors (these vectors coincide with the principal axes of curvature along which the surface is most curved at that point). Explicitly, at each sampling point, two vectors aligned with the principal axes of curvature are calculated along with the associated magnitude of curvature and the derivatives of the curvature along those principal axes. As described herein, a CAD model of the surface may not be available, so mathematical modeling is required to approximate the curvature near the defect in order to identify surface features.

[0033] FIG. 4 illustrates a method for treating a workpiece surface according to an embodiment herein. Method 400 may be implemented in a system such as system 100. However, some steps of method 400 may be performed at separate locations, for example, a defect may be identified by an inspection system, as shown in block 410, at a different location than a repair robot that repairs the defect in block 470. Furthermore, the mathematical model estimating the curvature may be generated locally, either at the inspection system, at the robot repair location, by a local robot controller, or at a third location, such as by a cloud-based server or a remote robot controller. Method 400 is described in the context of robotic repair of paint or clearcoat defects on a vehicle. However, similar methods may be useful in other industries.

[0034] In block 410, defects are detected on the vehicle surface. The defects may be dents, scratches, particles embedded in a clearcoat or paint layer, air bubbles, or dirt. The defects may be identified using a vision system that images the work surface and identifies the defects based on the images, as shown in block 402. The defects may also be identified based on a CAD model, for example, by feeding information to the CAD model so that the defect location may be defined with respect to known surface features, as shown in block 404. In block 406, the defects may be identified at least in part by a manual process, such as an individual identifying the defect as a scratch and marking it for repair so that a vision system can identify its location, or by feeding the measured location to a CAD system. Other options for identifying defect types or locations on the vehicle surface are also envisioned.

[0035] At block 420, the repair surface is measured. In some embodiments, a CAD model provides an accurate measurement of the defect location and the surface topography around the defect location, as shown in block 414. However, it is not always possible to obtain a CAD model for every workpiece surface. Therefore, it may be possible to use a vision system, as shown in block 412, as described in more detail with respect to FIG. 5 and method 500. It may also be possible to obtain some surface measurements manually, as shown in block 416, or using another method, as shown in block 418.

[0036] In block 430, a mathematical model is generated and used to approximate the surface topography at a number of points on the workpiece surface, such as the points measured in block 420. Model 400 can be useful for high-resolution and low-resolution surface topography modeling in different embodiments. In some embodiments, the low-resolution surface topography relies on a small number of surface sampling measurements, such as 25 points or less, 20 points or less, 15 points or less, 10 points or less. In embodiments where a bivariate quadratic model is used (F(x,y)=ax 2 +by 2For a surface topography model (e.g., a, b, c, d, and e), a minimum of five points are required to solve for the five constant values ​​(e.g., a, b, c, d, and e). As described herein, in low resolution scenarios, a surface topography model can be fitted, which can then serve as a basis onto which surface processing operations can be mapped. In this embodiment, it is assumed that the surface measurements used to fit the model are provided in a frame that is centered over the defect (hence no translation constants are required in the fitting model). However, in some embodiments, it is expressly contemplated that translation constants may be required if the surface measurements are provided in a frame that is not centered over the defect. Other models may be used and may require more or fewer points to solve for each constant.

[0037] In a high-resolution scenario, a larger number of surface samples are obtained, for example, at least 50 points, or at least 100 points, or at least 200 points, or at least 300 points, or more points are sampled. The more points sampled, the more accurately the surface topography model will approximate the underlying surface. For example, in a 6 cm diameter area, 300 sampling points can be used to approximate the surface topography and detect the surface curvature. In a high-resolution sampling scenario, it is possible to fit a higher-order mathematical model that more accurately captures the topographical features that characterize the underlying surface (grooves, sharp bends, etc.). This higher-order model can then be used to detect these features by identifying regions where the derivative of the curvature "crosses zero" along the principal axis of curvature. It may then be possible to adapt a robot trajectory based on the detected surface features and / or surface topography.

[0038] Approximating the surface using a model allows the robot controller to obtain a very minimal set of data describing the surface. The mathematical model generated in block 430 may be sufficient to approximate the surface around the defect to allow tailoring a custom repair operation to that area of ​​the workpiece surface. The mathematical model generated in block 430 allows the robot controller to perform a high quality repair without full knowledge of the underlying surface data. Instead, the surface processing system can receive surface measurements and use these measurements to plan and map the repair to the target surface. The method can include any polynomial surface approximation method, such as a quadratic fit, a cubic fit, a quartic fit, etc., as shown in block 434. Such models allow approximation of surface characteristics directly from equations without the need for continued sampling. The mathematical model may be fitted using least squares linear regression in some embodiments, as shown in block 434. Other models may be used, as shown in block 438. For example, if more data is available, the mathematical model may consist of a higher dimensional representation of the surface, including, in one embodiment, surface fitting using splines (B-splines, NURBS, etc.), or, in another embodiment, a support vector machine.

[0039] Features near the surface defect are detected at block 440. The features may be concave surfaces, convex surfaces, edges, protrusions, or other surface features that may interfere with the normal repair trajectory.

[0040] In block 450, a surface treatment template is obtained and modified based on the detected features, or based on the known or approximated surface curvature, or based on the surface topography. A repair template is a set of waypoints in a 2D plane with associated process parameters (e.g. applied force, disk speed, robot dynamics, etc.) assigned to each point or between each point. It defines a general set of instructions for the robot to perform surface treatment operations.

[0041] The template may be modified depending on the calculated curvature metric. For example, it may be desirable to modify a process parameter (e.g., applied force) assigned to or between two waypoints to be proportional to the magnitude of the maximum surface curvature that exists between those two points. As another example, the force profile applied to the entire set of waypoints in the repair template may be modified (e.g., by scaling up or down) depending on whether the surface is detected to be concave or convex at those points.

[0042] The shape of the template may be modified according to the approximated curvature metric. For example, the repair template may be warped to curve around the detected surface feature, allowing the robot to address defects near the surface feature without having to perform highly dynamic maneuvers that may cause scuffing or other undesirable effects.

[0043] Once the template has been appropriately modified (both its process parameters and its geometry), the modified template is then mapped onto the surface from the 2D reference plane above the defect using the fitted surface model.

[0044] A template may be obtained based on the detected defect type or size, as shown in block 452. A template may be obtained based on the type of surface, e.g., paint type or color, clear coat type, or other surface adjustment, as shown in block 454. A template may also be obtained based on the area of ​​the work surface being repaired, as shown in block 456. In the example of a defect in a repair area on the hood of an automobile, a repair template may be obtained with increased feathering at the edge as opposed to if the repair area is on the bottom edge of the door, which may not require increased feathering. As shown in block 462, the modification may maintain the repair trajectory to some extent, for example, the waypoint may undergo a transformation in which its position is shifted only radially relative to a center point defined in the repair template. The modification may also include distortion metrics, as shown in block 464, allowing the defect to be addressed without interfering with surface features. As shown in block 466, other modification methods may be used to allow the robotic repair tool to adequately address the defect while avoiding the surface features.

[0045] At block 460, waypoints associated with the repair are generated and transmitted to a robotic repair system, such as system 100. The waypoints may be generated based on known defect locations. The waypoints may cause the robotic repair unit to move a repair tool to the defect location and follow a modified trajectory. The waypoints may be based on an approximated surface around the defect to adequately treat or repair the workpiece surface. The waypoints may be generated by mapping the modified trajectory to the surface. The mapping may include Cartesian mapping or another suitable technique.

[0046] The robotic repair unit follows the modified trajectory to successfully repair the defect at block 470. Using method 400, the robotic repair unit can repair surfaces for which there is no known geometric model available.

[0047] Currently, many areas on the vehicle surface are classified as "no go" zones for the robotic repair unit. Defects in areas with curved surfaces or edges require a modification of the repair trajectory. The human performing the repair can intuitively adjust how to repair the curved surface by adjusting the motion, force, and speed. The systems and methods described herein may assist in modifying the repair trajectory to reduce the number of "no go" zones on the vehicle. However, it is expressly contemplated that the systems and methods herein may still classify some areas as unrepairable based on the detected topography. Detecting areas that are too difficult to repair based on the detected topography eliminates the need for manual labeling and continuous management of "no go zones" as the part may undergo design changes.

[0048] FIG. 5 illustrates a method for identifying surface features on a workpiece surface according to an embodiment herein. The use of point cloud methods for feature detection is not unknown. For example, Kim et.al, "Extraction of Ridge and Valley Lines from Unorganized Points." Multimedia tools and applications 63.1 (2013), pp 265-279, describes such methods. As described herein, the mathematical models of method 500 differ in how polynomial surfaces are fitted to the workpiece surface and how extreme crossings are detected. In some embodiments, method 500 relies on sampling of points around the location of interest. These points are then used to fit many polynomial models to "patches" around the surface, which are then stitched together to provide a nearly continuous representation of the surface for efficient interpolation. However, other methods of fitting an analytical surface to a series of points may be appropriate. In one embodiment, a NURBS (Non-uniform Rational B-Splines) surface is fitted to the set of measurements and then used to calculate the curvature metrics (principal axes of curvature, magnitude of curvature, and curvature derivatives as described above). Other methods may be suitable.

[0049] At block 510, a plurality of points on the surface are obtained. The sample points are obtained from a separate system, such as a vision system, that samples an area around the detected defects. The points are systematically sampled based on a grid measurement, in one embodiment, as shown in FIG. 13A. FIG. 13A shows points in a planar grid projected onto the target workpiece surface that is used to measure the local surface topography. The points are systematically sampled based on a grid measurement, in one embodiment, as shown in FIG. 13A. In this example, a plane is defined at the defect location and aligned to the surface normal at that location. A radius R of the central defect location is then calculated. mA grid of points within the surface is projected onto the surface to provide a sampling of depth measurements for this plane. These projected points are used to fit a surface approximation model.

[0050] In block 520, an approximation model is constructed. In one embodiment, this is based on the radius R of the defect point measured in the sampling plane mentioned above. f This is done by fitting a series of local approximation models, one to each sample point in . Each local surface model is bounded by the radius R of the sample point at the center of the patch, measured in the sampling plane. p The local model is fitted using "patches" of nearby points that are within the surface. This local model is an analytical approximation of the subregion of the repair area on the underlying surface and can be used to calculate a curvature metric in that subregion. In one embodiment, the local model may be constructed from a bivariate polynomial function that is fitted to the subregion using least squares regression.

[0051] Other methods may be used to fit a global approximation model directly from the entire set of measurement points taken from the surface. Such methods may use a surface approximation constructed from splines. For example, one such approach is presented in Simple Method for Constructing NURBS Surfaces from Unorganized Points, Leal et al., Proceedings of the 19th International Meshing Roundtable pp 161-175.

[0052] In some embodiments, only a local fit function is used for each sampling point, so no global fit is required and surface quantities can be compared across local approximations of the surface.

[0053] At block 530, curvature metrics are approximated with a set of sample points generated from the mathematical model. These points may be sampled from the analytical surface model at a finer resolution than the resolution of the original surface measurements. Using the approximation model fit to 38, a set of metrics describing the surface topography is calculated at each sample point. These metrics include the magnitude (k1, k2) of the surface curvature along the principal axes of curvature (v1, v2), as well as the associated curvature derivatives (e1, e2) along each axis.

[0054] The analytical model used for this description is presented below as Equation 1. However, it is explicitly contemplated that other models are possible. Modeling the surface by a series of bivariate quadratic approximation functions gives the following implicit form of each approximation function: F(x,y)=z(x,y)-(ax 2 +by 2 +cxy+dx+ey+d) Equation 1

[0055] Taking the gradient of F and normalizing it gives us the vector N that is normal to the surface, as shown in Equation 2 below:

number

[0056] We then take the eigenvalue decomposition of the gradient of the normal vector (▽N). The two eigenvectors associated with the two largest (based on absolute value) eigenvalues ​​are the vectors that lie along the principal axes of curvature. Let H be the Hessian (matrix of second partial derivatives) of F, then the associated curvature along vector v can be calculated using Equation 3 (as shown in Kim et al.):

number

[0057] Each sample point has two principal axes of curvature, a minimum and a maximum, each with an associated curvature value. Each curvature value represents the inverse of the radius of a circle that is tangent to the surface along the associated axis. The principal axis of curvature at each point (v1, v2) can be found by taking the eigenvalue decomposition of the gradient of the normal vector at that same point, as described above. The k and v metrics provide an analytical way to determine how much and in what direction the surface is "folded" at any given sample point.

[0058] The derivative of the curvature along the vector v can be calculated using Equation 4 (also given in Kim et al.).

number

[0059] At block 540, the derivative of the curvature along each of the principal axes of curvature is calculated at each point sampled from the analytical model.

[0060] At block 550, the zero crossing points of the derivative of the curvature along the major axis of curvature are approximated. The zero crossings in the derivative of the curvature (e.g., where e1, e2 change sign between adjacent points) correlate with valleys and ridges on the surface, e.g., surface features. Thus, surface features can be detected by comparing local approximations of the curvature derivative (e1, e2) between sample points.

[0061] It is well noted in the literature that the principal axis of curvature is directionally ambiguous, which can lead to random flips of the computed eigenvectors (v1,v2), which in turn can lead to erroneous feature point detections (the signs of e1,e2 also flip).

[0062] With this in mind, when checking whether a zero crossing of the derivative of curvature (e) exists between two sample points, the relevant principal directions are locally aligned. For example, if the eigenvector v1 at pt1 is found not to be within 90 degrees of the eigenvector v1 calculated at the adjacent point, the representation at pt1 is inverted (both v1 and e1 are negated). Once they are aligned, the change in sign of e1 compared between these adjacent points reveals the relevant feature points.

[0063] Feature points are detected when the sign of the derivative of the curvature changes after aligning the curvature axes between adjacent points. The sign of the curvature value (k) at the intersection indicates whether the surface feature is a ridge or a valley, and the magnitude of the curvature value provides a metric of how sharp the ridge or valley is. For example, a feature point can be ignored if the calculated curvature magnitude (|k|) at that feature point is found to be below a user-defined tolerance, as this point may not be important enough to require special consideration when performing repairs.

[0064] Once a feature point is found to be between two sample points, the values ​​of the curvature derivative at those points are used to determine a more accurate estimate of the intersection point using Equation 5 below.

number

[0065] This is in contrast to the method of Kim et al., which assigns the minimum of the derivative of the curvature to the "detected ridge point". Using a global approximation model, it is conceivable that more accurate methods of finding the exact intersection are possible. These methods can be performed using root-finding techniques such as Newton-Raphson.

[0066] To enable a robotic repair unit to repair a curved surface, a path may be projected directly onto a CAD model representing that surface. This assumes that the trajectory planner has full access to the CAD data and is thereby able to perform such path projection. In practice, these CAD files can be difficult to manage due to their complexity, dynamic nature (e.g., one part may go through multiple redesigns), and the underlying intellectual property agreements that must be established before such data can be shared. Method 500 allows the robotic repair unit to complete method 400 without the robotic controller needing to be aware of and manage these CAD files, instead by allowing sensors or auxiliary systems to provide sample points from which the surface is locally approximated to the robotic controller.

[0067] Once a model approximating the surface has been fitted, for example using method 500, the waypoints defining the templated robot repair trajectory can be mapped from the 2D plane to the 3D approximation. One way this mapping can be accomplished is by orthogonal projection. Other, more sophisticated means of mapping these path points are also contemplated, including means that attempt to limit radial distortions introduced by the 2D to 3D projection of the repair path.

[0068] FIG. 6 illustrates a process mapping system according to embodiments herein. The process mapping system 600 may be incorporated into a robot controller of a robotic repair unit in some embodiments. In other embodiments, the process mapping system 600 may be remote from the robotic repair unit 670 as shown in FIG. 6. Additionally, while FIG. 6 illustrates an embodiment in which the trajectory database 650 is separate and remote from the process mapping system 600, it is expressly contemplated that the process mapping system 600 may be located on a server having the trajectory database 650. It is expressly contemplated that component configurations other than those illustrated in FIG. 6 are contemplated.

[0069] The defect detection system 610 detects defects on the work surface and provides defect and surface information to the process mapping system 600. The defect detection system 610 may be an imaging system as described with respect to FIG. 1. However, other systems 610 may be possible. The defect detector 612 detects defects on the work surface in the vehicle repair example. However, in an adhesive dispensing embodiment, the adhesive target area detector 612 detects or identifies where adhesive should be dispensed. In a film wrapping embodiment, the target detector 612 detects areas where the film will be wrapped or areas where the film may be wrapped unevenly. In a material removal embodiment, the target removal area 612 detects where material should be removed from the surface. The surface sampler 616 samples the surface around the detected defects / adhesive dispensing targets / target removal areas. The defect detection system 610 may also include other functions 618. For example, the defect detection system 610 may be a camera system that images the surface before and after the defect removal / adhesive dispensing / material removal operation.

[0070] The robotic repair unit 670 is a mechanical unit responsible for implementing the repair trajectory on the work surface. The robotic repair unit 670 includes a robotic arm 672 that moves a polishing tool 680 into position for the repair operation. The robotic arm 672 may have multiple movement mechanisms (motorized or otherwise) that move the polishing tool 680 through the trajectory, as shown in Figures 7A-7C. The robotic repair unit 670 may also include a force control unit 674 that applies a force to the polishing tool 680 via an end effector.

[0071] The defect information receiver 602 obtains information about the defects, such as from a defect detection system 610. The defect information obtainer 602 may obtain the defect locations relative to the CAD model or may receive other location information.

[0072] The surface information acquirer 604 acquires the surface sampling information acquired by the surface sampler 616. Based on the acquired surface sampling, the model generator 620 can generate a mathematical model to approximate the curvature and detect surface features near the defect. The surface approximator 622 approximates the surface. This may be done using a polynomial approximation, a set of polynomial "patches" as described, an approximation using a spline-based method (B-splines, NURB surfaces, etc.), a support vector machine, or another suitable approximation. The curvature approximator 624 determines the magnitude and direction of the principal curvature at each of the sampling points based on the surface approximation. The derivative calculator 626 calculates the derivative of the curvature with respect to the principal axis of curvature at the set of sample points acquired from the fitting model. Based on the derivative calculation, the feature identifier 628 identifies a set of feature points, which may be, for example, a set of points that correspond to a valley, a ridge, a region of increased curvature, a region of convex curvature, a region of concave curvature, a region of an abrupt transition between convex and concave curvature (e.g., indicative of a narrow groove), or another feature.

[0073] The trajectory generator 640 generates a modified trajectory based on the surface curvature. The defect obtainer 642 obtains information about the defects, for example, from the defect detection system 610. The obtained defect information may include defect type, size, location, severity, or other information that may be useful for selecting a trajectory template. The trajectory obtainer 644 may obtain the trajectory template from a trajectory database 650. The trajectory database 650 may include a plurality of trajectories that may be selected based on defect parameters, such as defect type, defect location, defect size, and defect severity, as shown in block 654. The trajectory may also be selected based on the abrasive article 652 or the selected tool. For example, the size or type of the abrasive disc or back-up pad may dictate the trajectory selection, at least to some extent. Information about the workpiece surface, as shown in block 656, such as the workpiece surface material (e.g., wood, plastic, metal, paint, clear coat, layers thereof, etc.), may affect the trajectory selection. Similarly, process constraints 662 of the robotic repair unit 670, such as the maximum force that the force control unit 674 can exert, the maximum RPM of the polishing tool, or the clearance ranges required for different components, may influence the trajectory selection. Based on known information about the defect, the surface, and the robotic repair unit 670, an initial trajectory shape may be selected from trajectory shapes 658, such as a spiral, circle, ellipse, hypotrochoid, rosette, or other suitable shape.

[0074] Based on all of the acquired information, the trajectory generator 640 modifies the received trajectory. For example, each of a number of parameters may be specified at or between waypoints. The angle of attack may be adjusted as well as (or in addition to) the end effector speed, the RPM of the rotary tool, the vibration rate of the vibration tool, the applied force, the adhesive dispensing rate, the film dispensing rate, etc.

[0075] A topography acquirer 646 obtains the curvature information from the curvature approximator 624 and any identified features from the feature identifier 628, similar to obtaining the surface approximation from the surface approximator 622 in a low resolution scenario. A trajectory corrector 648 modifies the acquired trajectory based on the surface topography and any detected curvature and surface features. The trajectory may be modified by a scalar amount, e.g., contracted or expanded to correspond to the curvature, or adjusted in another manner, e.g., stretched or compressed. Based on the modified trajectory, a number of waypoints may be generated by a waypoint generator 649. The waypoints may be generated to guide the robot arm 672 from a previous defect or from a current location, to the defect location, and through the modified trajectory to complete the repair.

[0076] 7A-7C illustrate movement of a robotic repair unit over a surface feature. In FIG. 7A, a repair robot 700 is approaching a work surface 710, bringing a tool 712 close to the defect. As shown in FIG. 7B, the tool 712 engages the work surface 710 at an angle on a curved first side 714. As shown in FIG. 7C, the tool 712 engages the work surface 710 on a curved second side 716. A single trajectory allows the robotic repair unit 700 to adjust the angle of attack of the tool 712 to move from the first side 714 to the second side 716 of the curved work surface to address defects along the illustrated surface feature that would previously have required manual repair.

[0077] FIG. 8 is a surface processing system architecture. The surface processing architecture 800 illustrates one embodiment of an implementation of a defect detection and ranking system 810. As an example, the surface processing system architecture 800 can provide computing, software, data access, and storage services that do not require end-user knowledge of the physical location or configuration of the system delivering the services. In various embodiments, the remote server can deliver the services over a wide area network, such as the Internet, using an appropriate protocol. For example, the remote server can deliver applications over a wide area network, which can be accessed through a web browser or any other computing component. The software or components shown or described in FIGS. 1-7 and corresponding data can be stored on a server at a remote location. Computing resources in a remote server environment can be aggregated at a remote data center location, or they can be distributed. The remote server infrastructure can deliver services through shared data centers, which appear as a single access point to the user. Thus, the components and functions described herein can be provided from a remote server at a remote location using a remote server architecture. Alternatively, they may be provided by a traditional server, installed directly on the client device, or provided in other manners.

[0078] In the example shown in Figure 8, several items are similar to those shown in the previous figures. The figure specifically illustrates that the process mapping system 810 can be located at a remote server location 802. Thus, the computing device 820 accesses those systems through the remote server location 802. An operator 850 can similarly access a user interface 822 using the computing device 820.

[0079] 8 illustrates that it is also contemplated that some elements of the system described herein are located at the remote server location 802 while others are not. As an example, the storage 830, 840, or 660, or the robotic repair system 870, can be located at a location separate from the location 802 and accessed via a remote server at the location 802. Regardless of where they are located, they can be accessed directly by the computing device 820, hosted at the remote site by a service, provided as a service, or accessed by a connection service present at the remote location, over a network (either a wide area network or a local area network). Data can also be stored virtually anywhere and accessed by or transferred to the parties intermittently. For example, a physical carrier can be used instead of or in addition to an electromagnetic carrier.

[0080] It should also be noted that elements of the systems described herein, or portions thereof, can be located on a wide variety of different devices, including, but not limited to, servers, desktop computers, laptop computers, embedded computers, industrial controllers, tablet computers, or other mobile devices, such as palmtop computers, cell phones, smartphones, multimedia players, personal digital assistants, and the like.

[0081] 9-12 show examples of computing devices that may be used in the embodiments shown in the preceding figures.

[0082] Figure 9 is a simplified block diagram of an exemplary embodiment of a handheld or mobile computing device that may be used as a user's or client's handheld device 16 (such as computing device 820 of Figure 9) on which the present system (or portions thereof) may be deployed. For example, a mobile device may be deployed within an operator compartment of computing device 820 for use in generating, processing, or displaying data. Figure 10 is another embodiment of a handheld or mobile device.

[0083] 9 provides a schematic block diagram of components of a client device 916 capable of executing some of the components shown and described herein. The client device 916 interacts with or executes some of and interacts with some of the components. The device 916 is provided with a communication link 913 that allows the handheld device to communicate with other computing devices and under some embodiments provides a channel for automatically receiving information, such as by scanning. Examples of the communication link 913 include those that enable communication via one or more communication protocols, such as wireless services used to provide cellular access to a network, as well as protocols that provide local wireless connections to a network.

[0084] In another embodiment, the application may be received on a removable Secure Digital (SD) card that is connected to the interface 915. The interface 915 and communication link 913 communicate with a processor 1617 (which may also embody a processor) along a bus 919, which is also connected to memory 1621 and input / output (I / O) components 923, as well as a clock 925 and a position information system 927.

[0085] I / O components 923, in one embodiment, are provided to facilitate input and output operations, and devices 916 may include input components such as buttons, touch sensors, optical sensors, microphones, touch screens, proximity sensors, accelerometers, orientation sensors, etc., and output components such as display devices, speakers, and / or printer ports. Other I / O components 923 may be used as well.

[0086] Clock 925 illustratively includes a real-time clock component that outputs the time and date, and may also provide timing functions for processor 917.

[0087] Illustratively, the location information system 927 includes components that output the current geographic location of the device 916. This may include, for example, a global positioning system (GPS) receiver, a LORAN system, a dead reckoning system, a cellular triangulation system, or other positioning systems. It may also include, for example, mapping or navigation software that generates desired maps, navigation routes, and other geographic features.

[0088] The memory 921 stores an operating system 929, network settings 931, applications 933, application configuration settings 935, data storage 937, communication drivers 939, and communication configuration settings 941. The memory 921 may include all types of tangible, volatile computer readable memory and non-volatile computer readable memory devices, and may also include computer storage media (discussed below). The memory 921 stores computer readable instructions that, when executed by the processor 917, cause the processor to perform computer-implemented steps or functions in accordance with the instructions. The processor 1617 may also be activated by other components to facilitate their functions.

[0089] 10 shows that the device may be a smartphone 1001. The smartphone 1071 has a touch-sensitive display 1073 that displays icons or tiles or other user input mechanisms 1075 that can be used by a user to run applications, make calls, perform data transfer operations, etc. Generally, smartphones 1071 are built on mobile operating systems and offer more advanced computing capabilities and connectivity than feature phones.

[0090] It should be noted that other configurations of the device 1016 are possible.

[0091] FIG. 11 is a block diagram of a computing environment that can be used in the embodiments shown in the preceding figures.

[0092] 11 is an example of a computing environment in which elements of the systems and methods described herein, or portions thereof (for example), may be deployed. Referring to FIG. 11, an exemplary system for implementing some embodiments includes a general-purpose computing device in the form of a computer 1110. Components of the computer 1110 may include, but are not limited to, a processing unit 1120 (which may include a processor), a system memory 1130, and a system bus 1121 that couples various system components, including the system memory, to the processing unit 1120. The system bus 1121 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The memory and programs described with respect to the systems and methods described herein may be deployed in the corresponding portions of FIG. 11.

[0093] Computer 1110 typically includes a variety of computer readable media. Computer readable media may be any available media that can be accessed by computer 1110 and includes both volatile and nonvolatile media, and removable and non-removable media. By way of example, and not limitation, computer readable media may include computer storage media and communication media. Computer storage media is distinct from and does not include modulated data signals or carrier waves. Computer storage media includes hardware storage media, including both volatile and nonvolatile, removable and non-removable media, implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by computer 1110. Communication media may embody computer readable instructions, data structures, program modules or other data in a transport mechanism and include any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0094] The system memory 1130 includes computer storage media in the form of volatile and / or nonvolatile memory such as read only memory (ROM) 1131 and random access memory (RAM) 1132. A basic input / output system (BIOS) 1133, containing the basic routines that help to transfer information between elements within the computer 1110, such as during start-up, is typically stored in the ROM 1131. The RAM 1132 typically contains data and / or program modules that are immediately accessible to and / or presently being operated on by the processing unit 1120. By way of example, and not limitation, FIG. 11 illustrates operating system 1134, application programs 1135, other program modules 1136, and program data 1137.

[0095] The computer 1110 may also include other removable / non-removable, volatile / non-volatile computer storage media. By way of example only, Figure 11 illustrates a hard disk drive 1141, a non-volatile magnetic disk 1152, an optical disk drive 1155, and a non-volatile optical disk 1156, which read from or write to non-removable, non-volatile magnetic media. The hard disk drive 1141 is typically connected to the system bus 1121 through a non-removable memory interface, such as interface 1140, and the optical disk drive 1155 is typically connected to the system bus 1121 by a removable memory interface, such as interface 1150.

[0096] Alternatively, or in addition, the functions described herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0097] The drives and their associated computer storage media discussed above and illustrated in Figure 11 provide storage of computer readable instructions, data structures, program modules and other data for the computer 1110. In Figure 11, for example, hard disk drive 1141 is illustrated as storing operating system 1144, application programs 1145, other program modules 1146, and program data 1147. Note that these components can either be the same as or different from operating system 1134, application programs 1135, other program modules 1136, and program data 1137.

[0098] A user may enter commands and information into the computer 1110 through input devices such as a keyboard 1162, a microphone 1163, and a pointing device 1161, such as a mouse, trackball, or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite receiver, scanner, or the like. These and other input devices are connected to the processing unit 1120 through a user input interface 1160 that is often coupled to the system bus, although they may be connected by other interface and bus structures. A visual display 1191 or other type of display device is also connected to the system bus 1121 via an interface, such as a video interface 1190. In addition to the monitor, computers may also include other peripheral output devices, such as speakers 1197 and printer 1196, which may be connected through an output peripheral interface 1195.

[0099] The computer 1110 operates in a networked environment using logical connections, such as a Local Area Network (LAN) or a Wide Area Network (WAN), to one or more remote computers, such as a remote computer 1180.

[0100] When used in a LAN networking environment, the computer 1110 is connected to the LAN 1171 through a network interface or adapter 1170. When used in a WAN networking environment, the computer 1110 typically includes a modem 1172 or other means for establishing communications over the WAN 1173, such as the Internet. In a networked environment, program modules may be stored in remote memory storage devices. Figure 11 illustrates, for example, that remote application programs 1185 may reside on the remote computer 1180.

[0101] A robotic system is presented that includes a surface inspection system that receives a plurality of sampling points within a region of a workpiece surface. The system also includes a robot arm coupled to a surface treatment tool, the robot arm configured to engage the surface treatment tool with the region of the workpiece surface. The system also includes a process mapping system configured to approximate a surface topography of the region of the workpiece surface based on the plurality of sampling points, modify a trajectory of the robot arm based on the approximated surface topography, and generate a control signal for the robot arm that includes a path for the robot arm to enter the region.

[0102] The robotic system may be implemented such that the work surface is a vehicle.

[0103] The robotic system may be implemented such that the surface inspection system includes a vision system.

[0104] The robotic system may be implemented such that the vision system includes a camera.

[0105] The robotic system may be implemented such that the area includes a defect and the tool engaged with the surface is a material removal tool.

[0106] The robotic system may be implemented such that the surface treatment tool is a sander or a polishing tool.

[0107] The robotic system may be implemented such that the sampling points are identified using a 3D model of the vehicle.

[0108] The robotic system may be implemented such that the 3D model is a CAD model.

[0109] The robotic system may be implemented such that the sampling points are identified by measuring the vehicle.

[0110] The robotic system may be implemented such that the trajectory is mapped onto an approximated surface.

[0111] The robotic system may be implemented such that the surface topography is approximated from fewer than 10 sampling points.

[0112] The robotic system may also include approximating a curvature at each of the sampling points within the region. The approximated surface topography includes the approximated curvature.

[0113] The robotic system may be implemented such that approximating the curvature includes sampling the approximated surface.

[0114] The robotic system may be implemented such that the sampling points are first sampling points and sampling the approximated surface includes sampling a second set of sampling points, each of the points in the second set being a sample taken from the model around each of the first sampling points.

[0115] The robotic system may be implemented such that the number of sampling points is less than 50.

[0116] The robotic system may also include calculating a derivative of the approximated curvature at each of the sampling points and identifying surface features within the region based on the derivative calculation.

[0117] The robotic system may be implemented with more than 100 sampling points.

[0118] The robotic system may be implemented such that a path includes a series of waypoints through an area.

[0119] The robotic system may be implemented such that a trajectory is selected based on defect size, defect location, defect type, or defect severity.

[0120] The robotic system may be implemented such that the trajectory is modified based on surface features identified within the approximated surface topography.

[0121] The robotic system may be implemented such that the trajectory is stretched or compressed.

[0122] The robotic system may be implemented such that the trajectory shape is a circle, an ellipse, a rosette, a spiral, or a hypotrochoid.

[0123] The robotic system may be implemented such that the trajectories are scaled proportionally.

[0124] The robotic system may be implemented such that the modified trajectory includes a modified tool force, disk velocity, or tool velocity.

[0125] The robotic system may be implemented such that the robotic arm executes the control signals and follows a path.

[0126] The robotic system may be implemented such that the approximation is a polynomial approximation.

[0127] The robotic system may be implemented such that the polynomial approximation is a third order polynomial.

[0128] The robotic system may be implemented such that the number of sampling points is less than 20.

[0129] A surface feature detection system for detecting features on a workpiece surface is presented, the system including a surface sampling receiver that receives a plurality of surface samples from a surface imaging system. The system also includes a surface approximator that approximates a surface at each of the plurality of surface samples using an approximation. The system also includes a curvature approximator that approximates a curvature and a derivative of the curvature at each of the plurality of surface samples. The system also includes a feature detector that identifies zero crossing points in the derivative of the curvature as detected feature points based on the derivative of the curvature approximation at each of the surface samples.

[0130] The system may also include a communications component that transmits an indication of the detected features to the display component.

[0131] The system may be implemented such that detected features are overlaid onto an approximated surface.

[0132] The system may be implemented such that detected features are overlaid on the sampled surface.

[0133] The system may be implemented such that a number of surface samples are received from a CAD model.

[0134] The system may be implemented such that a number of surface samples are received from the surface measurement system.

[0135] The system may be implemented such that the surface measurement system includes a camera.

[0136] The system may be implemented such that the approximation is a polynomial approximation.

[0137] The system may be implemented such that the polynomial approximation is a third order polynomial.

[0138] The system may be implemented such that multiple surface samples are within a radius of a target area on the work surface.

[0139] The system may be implemented such that the work surface is a vehicle surface, the target area includes a defect, and the surface features are valleys, ridges, areas of increased curvature, areas of convex curvature, areas of concave curvature, and areas of abrupt transitions between convex and concave curvature on the vehicle surface proximate the defect.

[0140] The system may be implemented such that a number of surface samples are received from a vision system that images the workpiece surface.

[0141] The system may also include a communication component that outputs the detected features to a trajectory corrector that modifies the repair trajectory to address the defects.

[0142] The system may be implemented such that the shape of the trajectory is stretched or compressed based on the detected features.

[0143] The system may be implemented such that the trajectory is scaled up or down proportionally.

[0144] The system may be implemented such that the trajectory corrector maps the trajectory template onto the approximated surface to obtain a corrected trajectory.

[0145] The system may also include a repair evaluator that classifies the region as robotically repairable or not robotically repairable based on the approximated curvature and a derivative of the approximated curvature.

[0146] The system may be implemented such that if an area is repairable by the robot, the trajectory corrector generates a trajectory through that area.

[0147] The system may be implemented such that the trajectory includes a number of waypoints and a force and velocity of the surface treatment tool contacting the workpiece surface at each of the number of waypoints.

[0148] The system may be implemented such that the surface preparation tool is a rotating tool and each of the plurality of waypoints includes a rotational speed.

[0149] The system may be implemented such that the surface is approximated from fewer than 10 sampling points.

[0150] The system may be implemented such that approximating the curvature includes sampling the approximated surface.

[0151] The system may be implemented such that the sampling area is a first sampling area and sampling the approximated surface includes sampling a second set of sampling areas, each of the areas of the second set being a sample taken from the model around each of the first sampling areas.

[0152] The system may be implemented such that the number of sampling regions is less than 50.

[0153] The system may be implemented so that the number of sampling regions exceeds 100.

[0154] The system may be implemented such that the number of sampling regions is less than 20.

[0155] The system may be implemented such that the modified trajectory includes modified tool force, disk velocity, or tool speed.

[0156] The system may be implemented such that a trajectory template is selected based on defect size, defect location, defect type, or defect severity.

[0157] The system may be implemented such that the trajectory is modified based on the identified surface features.

[0158] A method for removing material from a workpiece surface is presented, the method including identifying a target area on the workpiece surface for material removal, sampling the surface around the target area on the workpiece surface, modeling the surface and detecting a surface topography based on the model, and modifying a surface treatment trajectory based on the detected surface topography. The surface treatment trajectory includes a path of travel through the target area. The method also includes sending a control signal to a robotic material removal system, the control signal including the modified trajectory.

[0159] The method may be implemented such that modeling the surface comprises approximating the surface at each of a plurality of sampled surface locations.

[0160] The method may be implemented such that the surface approximation is a polynomial approximation.

[0161] The method may be implemented such that the target area includes a defect.

[0162] The method may be implemented such that the work surface includes a vehicle.

[0163] The method may be implemented such that sampling the surface includes a vision system imaging the surface.

[0164] The method may be implemented such that the vision system includes a camera.

[0165] The method may be implemented such that the surface treatment trajectory comprises a force profile along the path of movement.

[0166] The method may be implemented such that the modified surface treatment trajectory comprises a modified path of movement through the target area.

[0167] The method may be implemented such that the modified travel path has a smaller area than the original travel path.

[0168] The method may be implemented such that the modified path of movement is linearly translated from the original path of movement.

[0169] The method may be implemented such that the modified travel path is stretched or compressed from the original travel path.

[0170] The method may be implemented such that the modified travel path includes an angle of attack of a surface engaging tool of the robotic material removal system.

[0171] The method may be implemented such that a vision system identifies a target area as containing a defect.

[0172] The method may also include detecting surface features from the surface topography.

[0173] The method may also include classifying the target area as robotically repairable based on the detected surface topography.

[0174] The method may be implemented such that the surface engaging tool is a material removal tool.

[0175] The method may be implemented such that the surface treatment tool is a sander or an abrasive tool.

[0176] The method may be implemented such that the surface topography is sampled from a 3D model of the vehicle.

[0177] The method may be implemented such that the 3D model is a CAD model.

[0178] The method may also include approximating the curvature at each of the sampled surface locations and detecting surface features based on derivative values ​​of the approximated curvature.

[0179] The method may be implemented such that the surface topography is approximated from fewer than 10 sampled surface locations.

[0180] The method may be implemented such that the surface topography is approximated from fewer than 20 sampled surface locations.

[0181] The method may be implemented such that the surface topography is approximated from fewer than 50 sampled surface locations.

[0182] The method may be implemented such that the surface topography is approximated from more than 100 sampled surface locations.

[0183] The method may be implemented such that approximating the curvature includes sampling the approximated surface.

[0184] The method may be implemented such that the sampling area is a first sampling area and sampling the approximated surface includes sampling a second set of sampling areas, each of the areas of the second set being a sample taken from the model around each of the first sampling areas.

[0185] The method may be implemented such that the template surface treatment trajectory is selected based on defect size, defect location, defect type, or defect severity.

[0186] The method may be implemented such that a template surface treatment trajectory is selected based on the identified surface features. EXAMPLES

[0187] Example 1 The edge is found on the analytical surface defined by the following equation:

number

[0188] R M (radius of the whole circle) is set to 0.06, and R P The path radius is set to 0.015. The measurement points x are spaced apart from each other by a distance d in the grid pattern, as shown in FIG. x = 0.003 apart. The projection of these points onto the analytical surface is shown in Fig. 12B. Each light dot represents a sample of interpolation points found by evaluating a local polynomial patch fit from the original sampling set. Fig. 12A shows how local models can be patched together to approximate the shape of the underlying surface. Fig. 12C shows a sampling of patches, where each patch is used to fit a polynomial model, which is then used to locally approximate the surface. Fig. 12D shows the v plotted at all points. max 12E. Point 1210 indicates an umbilicus where the surface is essentially locally spherical. Point 1212 correlates to a region where the magnitude of curvature is close to zero. The intersection of points 1210 and 1212 indicates a region where the surface is locally flat. The detected intersection of the derivatives of the curvature is shown in FIG. 12E. Point 1220 is an illustrated "valley" on the surface and point 1222 is a "ridge". Example 2

[0189] High-resolution sampling is performed on a surface with unknown topography. To do this, points are sampled from a CAD model that represents a smooth, well-behaved surface. The distance x between sample points is d x = 0.003. M = 0.03, and R P = 0.0075. The sampling plane is taken to be aligned with the surface normal at the central sampling point on the CAD geometry. Figures 13A and 13B show how the algorithm was used to detect features of the underlying surface.

[0190] The strength of the algorithm shown in Example 2 is its ability to capture the underlying nature of a surface using a set of measurements from the surface. The system can scan or otherwise obtain measurements from the surface to detect areas that are highly dynamic in nature and quantify the dynamic variations using curvature measurements as described herein. This allows for robust and efficient detection of surface features. The feature points detected by the algorithm described herein are accurate to the resolution used in the sampling grid, as shown by the slight S-curve of the feature points shown in FIG. 13A. A higher sampling resolution for the fitting model will improve the accuracy of the ridge lines detected for the fitting model.

[0191] Example 3 14A-14C show the results of mathematically modeling a work surface having surface features with a small collection of points. A set of measurements 1402, 1404 are sampled on a CAD surface 1400 representing the vehicle. An analytical model is fitted to these points and the derivative of the curvature is measured at each sample point. When the derivative of the curvature changes sign, a feature line 1402 is detected. Repairs near this area may then be transformed to avoid this feature line or to be performed parallel and / or perpendicular to it. Path speed, tool force, and tool velocity may also be adjusted.

[0192] FIG. 14A shows a sampled surface 1420 on a workpiece surface 1410. The workpiece surface 1410 is shown in FIGS. 14A-14C as a vehicle with a defect located near a ridge on a wheel well. FIGS. 14B-14C show another view of the sampled area. Sampled points 1422 and the outer ring 624 are received from a sampling system, such as the vision system of FIG. 1. A mathematical surface fit is represented by web 1428. The model is locally accurate and diverges away from the defect. Arrows 1426 indicate the normal from the surface. In some embodiments, the points 1422 are taken from the frame of reference of the normal 1426. However, it is contemplated that in other embodiments, the point cloud 1422 can be taken from any frame of reference. If the points 1422 are taken in a different frame of reference, the normal plane to the defect is corrected.

Claims

1. A surface inspection system that receives a plurality of sampling points within an area of a work surface, A robotic repair arm coupled to a surface treatment tool and configured to engage the surface treatment tool with the area of the work surface, A process mapping system, comprising a robotic system, The process mapping system, based on the plurality of sampling points, Approximates the surface topography of the area of the work surface, Based on the approximated surface topography, corrects the trajectory of the robotic arm, A robotic system configured to generate a control signal for the robotic arm, including a path for the robotic arm to enter the area.

2. The robotic system according to claim 1, wherein the work surface is a vehicle.

3. The robotic system according to claim 2, wherein the area contains a defect and the tool engaged with the surface is a material removal tool.

4. The robotic system according to claim 1, wherein the trajectory is mapped onto the approximated surface.

5. Further comprising approximating the curvature at each of the sampling points within the area, The robotic system according to claim 1, wherein the approximated surface topography includes the approximated curvature.

6. The robotic system according to claim 5, wherein approximating the curvature includes sampling the approximated surface.

7. Calculating the derivative of the approximated curvature at each of the sampling points, Identifying surface features within the area based on the calculation of the derivative, The robotic system according to claim 1, further comprising.

8. The robotic system according to claim 1, wherein the trajectory is corrected based on surface features identified within the approximated surface topography.

9. A method of removing material from a work surface, Identifying a target area on the work surface for material removal, Sampling the surface around the target area on the work surface, Modeling the surface and detecting surface topography based on the model, Based on the detected surface topography, correcting a surface treatment trajectory, the surface treatment trajectory including a movement path through the target area. A method comprising a step of transmitting a control signal to a robotic material removal system, wherein the control signal includes the modified trajectory. **Claim 10** The step of modeling the surface The method of claim 9, wherein the step of modeling the surface includes approximating the surface at each of a plurality of sampled surface positions.

Images