SYSTEM AND METHOD FOR TREATING A WORK SURFACE - Patent application

Abstract

A robotic system is presented that includes a surface inspection system that receives sampling information for a plurality of areas within a region of a workpiece surface. The system also includes a robot arm coupled to a surface engagement tool, the robot repair arm configured to engage the surface treatment tool to the region of the workpiece surface. The system also includes a process mapping system configured to approximate a surface topography in the region of the workpiece surface based on the sampling information and generate a surface treatment plan for the region based on the approximated surface topography, the surface treatment plan including a trajectory. The surface treatment plan includes one of a force profile along the trajectory, a velocity profile of the surface engagement tool along the trajectory, a rotational velocity profile of the surface engagement tool along the trajectory, and a trajectory modification that accounts for the presence of surface features identified in the approximated surface topography. The process mapping system is also configured to generate control signals for the robot arm that include the surface treatment plan.

Description

[Background technology]

[0001] Clearcoat repair is one of the final operations to be automated in the automotive original equipment manufacturing (OEM) sector. Technology is desired to automate this process as well as other surface treatment applications including paint applications (e.g., primer sanding, clearcoat defect removal, clearcoat polishing, etc.), adhesive dispensing, film wrapping application, or material removal systems that are amenable to the use of abrasives and / or robotic inspection and repair. Defect repair presents many challenges for automation. Summary of the Invention

[0002] A robotic system is presented that includes a surface inspection system that receives sampling information for a plurality of areas within a region of a workpiece surface. The system also includes a robot arm coupled to a surface engagement tool, the robot repair arm configured to engage the surface treatment tool to the region of the workpiece surface. The system also includes a process mapping system configured to approximate a surface topography in the region of the workpiece surface based on the sampling information and generate a surface treatment plan for the region based on the approximated surface topography, the surface treatment plan including a trajectory. The surface treatment plan includes one of a force profile along the trajectory, a velocity profile of the surface engagement tool along the trajectory, a rotational velocity profile of the surface engagement tool along the trajectory, and a trajectory modification that accounts for the presence of surface features identified in the approximated surface topography. The process mapping system is also configured to generate control signals for the robot arm that include the surface treatment plan. [Brief description of the drawings]

[0003] The drawings, which are not necessarily drawn to scale and in different drawings, like numerals may describe like elements, illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in the present document. [Figure 1] 1 is a schematic diagram of a robotic surface treatment system in which embodiments of the present invention are useful; [Diagram 2] 1 illustrates a robotic defect repair method in which embodiments of the present invention may be useful. [Diagram 3] 1 illustrates a work surface having a plurality of surface features in which embodiments herein may be useful. [Figure 4] 1 illustrates a method for treating a workpiece surface according to an embodiment herein. [Diagram 5] 1 illustrates a method for identifying surface features on a workpiece surface, according to an embodiment herein. [Figure 6A] 1 illustrates feature detection on a surface according to embodiments herein. [Figure 6B] 1 illustrates feature detection on a surface according to embodiments herein. [Figure 6C] 1 illustrates feature detection on a surface according to embodiments herein. [Figure 7A] 1 illustrates a corrective trajectory according to an embodiment herein. [Figure 7B] 1 illustrates a corrective trajectory according to an embodiment herein. [Figure 7C] 1 illustrates a corrective trajectory according to an embodiment herein. [Figure 8A] 1 illustrates a method for modifying a trajectory according to an embodiment herein. [Figure 8B] 1 illustrates a method for modifying a trajectory according to an embodiment herein. [Figure 8C] 1 illustrates a method for modifying a trajectory according to an embodiment herein. [Figure 8D] 1 illustrates a method for modifying a trajectory according to an embodiment herein. [Figure 9A] 1 illustrates a corrective trajectory according to an embodiment herein. [Figure 9B] 1 illustrates a corrective trajectory according to an embodiment herein. [Figure 9C] 1 illustrates a corrective trajectory according to an embodiment herein. [Figure 9D] 1 illustrates a corrective trajectory according to an embodiment herein. [Figure 9E] 1 illustrates a corrective trajectory according to an embodiment herein. [Figure 10] 1 illustrates a method for generating a modified surface treatment strategy according to an embodiment herein. [Figure 11] 1 illustrates a process mapping system, according to an embodiment of the present disclosure. [Figure 12] This is the surface treatment strategy generation system architecture. [Figure 13] 2 illustrates an example of a computing device that may be used in the embodiments illustrated in the preceding figures. [Figure 14] 2 illustrates an example of a computing device that may be used in the embodiments illustrated in the preceding figures. [Figure 15] 2 illustrates an example of a computing device that may be used in the embodiments illustrated in the preceding figures. [Figure 16A] 13 illustrates an example parameter adjustment for a corrective trajectory. [Figure 16B] 13 illustrates an example parameter adjustment for a corrective trajectory. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004] Recent advances in imaging technology and computing systems have made the process of clearcoat inspection at production speeds feasible. In particular, stereo deflectometry has recently been shown to be capable of providing images and locations of paint and clearcoat defects with adequate resolution, along with spatial information (providing coordinate location information and defect classification) to enable subsequent automated spot repair. As automated imaging of workpiece surfaces improves, it is equally desirable to improve the ability to automatically process workpiece surfaces. For example, in the case of clearcoat repair, it is desirable to use a robotic repair system to repair detected defects with as little manual intervention as possible. However, as described herein, workpiece surfaces that exhibit a high degree of curvature (significant departure from being a flat surface) are particularly difficult to process using robotic systems. Additionally, the presence of sharp surface features (such as sharp bends, grooves, etc.) near the desired repair area can further complicate the task of performing the automated repair. While humans intuitively know how to modify their approach to deal with convex or concave surface features, robotic systems must be trained on what the surface features are and how to adjust their pre-programmed trajectories to accommodate them when they are detected. The systems and methods herein may be useful in any surface processing application where a surface processing task must be mapped onto a surface that exhibits significant curvature and / or surface features.

[0005] As used herein, the term "vehicle" is intended to encompass a wide range of moving structures that are coated with at least one paint or clear coat during manufacture. While many of the examples herein relate to automobiles, it is expressly contemplated that the methods and systems described herein are also applicable to trucks, trains, boats (with or without motors), airplanes, helicopters, and the like. Additionally, while vehicles are described as examples for which embodiments herein are particularly useful, it is expressly contemplated that some of the systems and methods herein may be applied to surface treatment in other industries, such as painting, adhesive treatment, or material removal, such as sanding or polishing of wood, plastics, paints, and the like.

[0006] The term "paint" is used herein broadly to refer to any of the various layers of a vehicle's e-coat, filler, primer, paint, clear coat, etc., applied in the finishing process. Moreover, the term "paint repair" includes locating and repairing any visual artifacts (defects) on or within any paint layer. In some embodiments, the systems and methods described herein use clear coat as the target paint repair layer. However, the systems and methods presented apply with little or no modification to any particular paint layer (e-coat, filler, primer, paint, clear coat, etc.).

[0007] As used herein, the term "defect" refers to an area on a work surface that interferes with visual aesthetics. For example, many vehicles have a shiny or metallic appearance after painting is completed. A "defect" may include debris trapped within one or more of the various paint layers on the work surface. A defect may also include excess paint, including stains, smudges or drips within the paint, as well as dents.

[0008] Paint repair is one of the last remaining steps in the vehicle manufacturing process that remains largely manual. Historically, this is due to two main factors: the lack of sufficient automated inspection and the difficulty of automating the repair process itself. Humans who interact with surfaces have an intuitive understanding of how to respond to surface curvatures, edges, and other features. Robotic surface treatment systems require training.

[0009] FIG. 1 is a schematic diagram of a robotic paint repair system in which embodiments of the present invention are useful. The system 100 generally includes two units: a visual inspection system 110 and a defect repair system 120. Both systems can be controlled by motion controllers 112, 122, respectively, which can receive instructions from one or more application controllers 150. The application controllers can receive input from or provide output to a user interface 160. The repair unit 120 includes a force control unit 124 that can be associated with an end effector 126. As shown in FIG. 1, the end effector 126 includes two processing tools 128. However, other arrangements are also expressly contemplated.

[0010] The current state of the art in vehicle paint repair is to manually sand / polish the imperfections using fine abrasives and / or polishing systems with or without the assistance of power tools all the way through while maintaining the desired finish (e.g., comparable to mirror finish in a clear coat). A skilled person performing such repairs utilizes extensive training as well as their senses to monitor the progress of the repair and make changes accordingly. Such advanced techniques are difficult to obtain in a robotic solution where sensing is limited.

[0011] Moreover, while abrasive material removal is a pressure-driven process, many industrial manipulators typically operate natively in a position tracking / control regime and are optimized with positional accuracy in mind. This results in extremely rigid systems with extremely stiff error response curves (i.e., small misalignments result in extremely large corrective forces) that are inherently poor at force control (i.e., joint torques and / or orthogonal forces). Closed-loop force control techniques have been used (with limited utility) to address the latter, with more recent (and more successful) force-controlled flanges providing softer (i.e., less rigid) displacement curves that are much more suitable for sensitive force / pressure-driven processes. However, the problem of robust process planning / control remains and is the focus of this research.

[0012] 2 illustrates a robotic defect repair method in which embodiments of the invention may be useful. Method 200 is an overview of how a robotic repair system repairs defects in accordance with at least some embodiments described herein.

[0013] At block 210, instructions are received from a robot controller, such as, for example, application controller 150 of Figure 1. The instructions include movement instructions for different components of the robot repair unit, such as the force control unit, the end effector motors, or the tool movement pattern.

[0014] At block 220, a robotic motion controller moves an abrasive article attached to the tool into position and prepares it to engage the defect. The defect location may be known from the inspection system or may otherwise be identified based on, for example, a CAD file of the workpiece surface.

[0015] An abrasive article is engaged with the defect at block 230. Engaging the defect may include sanding the defective area or polishing the defective area.

[0016] At block 240, the defect area is cleaned. Cleaning may include wiping off any fluids used in sanding or polishing, as well as wiping off debris. After the cleaning step, the tool may be re-engaged with the defect, as shown in block 342.

[0017] At block 250, the defective area is inspected to determine if the repair is sufficient. If additional repair is required, method 200 may include the robotic repair unit receiving new instructions, as indicated by arrow 260, and the method may be repeated. Inspecting the defect repair may include obtaining 252 a post-repair image, which may be presented to a repair operator or stored as desired. Inspecting may also include verifying the repair, as indicated at block 254, which may include comparing pre-repair and post-repair images, detecting whether the defect is visible / noticeable to the human eye, or another suitable verification technique.

[0018] FIG. 3 illustrates a work surface having multiple surface features, for which embodiments herein may be useful. The work surface 300, shown in FIG. 3 as a car hood, includes a surface having multiple surface features 302 and some flat regions 304. Currently, defects are detected by an inspection system and provided to a human reviewer, who indicates whether the defect is in a repairable or non-repairable region. Repairable defects 310 are defects that are in regions on the work surface 300 that a robotic repair unit can repair using a repair trajectory based on a flat repair surface. In contrast, defects that are too close to the surface features 302 are considered manual repair defects 320, or defects that cannot be processed by the robotic surface processing system and therefore must be repaired manually. Currently, in many applications, imaged defects are mapped to regions of the vehicle using predefined routines and "repair / no repair" criteria. For example, "narrow" regions or regions on curved (concave or convex) surfaces are "no repair" regions.

[0019] It would be desirable to have a system or method that allows a robotic surface processing system, such as system 100, to process a surface proximate to or over an area that includes a surface feature. It would also be desirable to have a system that can determine the surface topography proximate to a defect location without access to a CAD (computer-aided design) model of the vehicle or other work surface, since it is not always possible to obtain such a model from the vehicle manufacturer. For example, when repairs are made in an aftermarket environment, such as a repair shop, the only way to gather surface data may be through the use of direct measurement.

[0020] As described herein, in some embodiments, a defect is identified and a number of points on the surface surrounding the defect are measured. Thus, a point cloud is obtained and based on this point cloud, a topography of the surface is determined. In some embodiments, based on the surface topography, surface features are identified. In some embodiments, a set of curvature metrics are calculated at a number of sampling points obtained from a model fitted from the measurements. At each sampling point, these metrics are calculated with respect to two orthogonal vectors (these vectors coincide with the principal axes of curvature along which the surface is most curved at that point). Explicitly, at each sampling point, two vectors aligned with the principal axes of curvature are calculated along with the associated magnitude of curvature and the derivatives of the curvature along those principal axes. As described herein, a CAD model of the surface may not be available, so mathematical modeling is required to approximate the curvature near the defect in order to identify surface features.

[0021] FIG. 4 illustrates a method for treating a workpiece surface according to an embodiment herein. Method 400 may be implemented in a system such as system 100. However, some steps of method 400 may be performed at separate locations, for example, a defect may be identified by an inspection system, as shown in block 410, at a different location than a repair robot that repairs the defect in block 470. Furthermore, the mathematical model estimating the curvature may be generated locally, either at the inspection system, at the robot repair location, by a local robot controller, or at a third location, such as by a cloud-based server or a remote robot controller. Method 400 is described in the context of robotic repair of paint or clearcoat defects on a vehicle. However, similar methods may be useful in other industries.

[0022] In block 410, defects are detected on the vehicle surface. The defects may be dents, scratches, particles embedded in a clearcoat or paint layer, air bubbles, or dirt. The defects may be identified using a vision system that images the work surface and identifies the defects based on the images, as shown in block 402. The defects may also be identified based on a CAD model, for example, by feeding information to the CAD model so that the defect location may be defined with respect to known surface features, as shown in block 404. In block 406, the defects may be identified at least in part by a manual process, such as an individual identifying the defect as a scratch and marking it for repair so that a vision system can identify its location, or by feeding the measured location to a CAD system. Other options for identifying defect types or locations on the vehicle surface are also envisioned.

[0023] At block 420, the repair surface is measured. In some embodiments, a CAD model provides an accurate measurement of the defect location and the surface topography around the defect location, as shown in block 414. However, it is not always possible to obtain a CAD model for every workpiece surface. Therefore, it may be possible to use a vision system, as shown in block 412, as described in more detail with respect to FIG. 5 and method 500. It may also be possible to obtain some surface measurements manually, as shown in block 416, or using another method, as shown in block 418.

[0024] In block 430, a mathematical model is generated and used to approximate the surface topography at a number of points on the workpiece surface, such as the points measured in block 420. Model 400 can be useful for high-resolution and low-resolution surface topography modeling in different embodiments. In some embodiments, the low-resolution surface topography relies on a small number of surface sampling measurements, such as 25 points or less, 20 points or less, 15 points or less, 10 points or less. In embodiments where a bivariate quadratic model is used (F(x,y)=ax 2 +by 2For a surface topography model (e.g., a, b, c, d, and e), a minimum of five points are required to solve for the five constant values ​​(e.g., a, b, c, d, and e). As described herein, in low resolution scenarios, a surface topography model can be fitted, which can then serve as a basis onto which surface processing operations can be mapped. In this embodiment, it is assumed that the surface measurements used to fit the model are provided in a frame that is centered over the defect (hence no translation constants are required in the fitting model). However, in some embodiments, it is expressly contemplated that translation constants may be required if the surface measurements are provided in a frame that is not centered over the defect. Other models may be used and may require more or fewer points to solve for each constant.

[0025] In a high-resolution scenario, a larger number of surface samples are obtained, for example, at least 50, or at least 100, or at least 200, or at least 300, or more points are sampled. The more points sampled, the more accurately the surface topography model will approximate the underlying surface. For example, in a 6 cm diameter area, 300 sampling points can be used to approximate the surface topography and detect the surface curvature. In a high-resolution sampling scenario, it is possible to fit a higher-order mathematical model that more accurately captures the topographical features that characterize the underlying surface (grooves, sharp bends, etc.). This higher-order model can then be used to detect these features by identifying regions where the derivative of the curvature "crosses zero" along the principal axis of curvature. It may then be possible to adapt a robot trajectory based on the detected surface features and / or surface topography.

[0026] Approximating the surface using a model allows the robot controller to obtain a very minimal set of data describing the surface. The mathematical model generated in block 430 may be sufficient to approximate the surface around the defect to allow custom repair work to be tailored to that area of ​​the workpiece surface. The mathematical model generated in block 430 allows the robot controller to perform high quality repairs without full knowledge of the underlying surface data. Instead, the surface processing system can receive surface measurements and use these measurements to plan and map the repair to the target surface. This method can include any polynomial surface approximation method, such as a quadratic fit, a cubic fit, or a quartic fit, as shown in block 434. Such models allow for approximation of surface characteristics directly from equations without the need for continued sampling. The mathematical model may be fitted using least squares linear regression in some embodiments, as shown in block 434. Other models may be used, as shown in block 438. For example, if more data is available, the mathematical model may consist of a higher dimensional representation of the surface, including surface fitting using splines (B-splines, NURBS, etc.) in one embodiment, or a support vector machine in another embodiment.

[0027] Features near the surface defect are detected at block 440. The features may be concave surfaces, convex surfaces, edges, protrusions, or other surface features that may interfere with the normal repair trajectory.

[0028] In block 450, a surface treatment template is obtained and modified based on the detected features, or based on the known or approximated surface curvature, or based on the surface topography. A repair template is a set of waypoints in a 2D plane with associated process parameters (e.g. applied force, disk speed, robot dynamics, etc.) assigned to each point or between each point. It defines a general set of instructions for the robot to perform surface treatment operations.

[0029] The template may be modified depending on the calculated curvature metric. For example, it may be desirable to modify a process parameter (e.g., applied force) assigned to or between two waypoints to be proportional to the magnitude of the maximum surface curvature that exists between those two points. As another example, the force profile applied to the entire set of waypoints in the repair template may be modified (e.g., by scaling up or down) depending on whether the surface is detected to be concave or convex at those points.

[0030] The shape of the template may be modified according to the approximated curvature metric. For example, the repair template may be warped to curve around the detected surface feature, allowing the robot to address defects near the surface feature without having to perform highly dynamic maneuvers that may cause scuffing or other undesirable effects.

[0031] Once the template has been appropriately modified (both its process parameters and its geometry), the modified template is then mapped onto the surface from the 2D reference plane above the defect using the fitted surface model.

[0032] A template may be obtained based on the detected defect type or size, as shown in block 452. A template may be obtained based on the type of surface, e.g., paint type or color, clear coat type, or other surface adjustment, as shown in block 454. A template may also be obtained based on the area of ​​the work surface being repaired, as shown in block 456. In the example of a defect in a repair area on the hood of an automobile, a repair template may be obtained with increased feathering at the edge as opposed to if the repair area is on the bottom edge of the door, which may not require increased feathering. As shown in block 462, the modification may maintain the repair trajectory to some extent, for example, the waypoint may undergo a transformation in which its position is shifted only radially relative to a center point defined in the repair template. The modification may also include distortion metrics, as shown in block 464, allowing the defect to be addressed without interfering with surface features. As shown in block 466, other modification methods may be used to allow the robotic repair tool to adequately address the defect while avoiding the surface features.

[0033] At block 460, waypoints associated with the repair are generated and transmitted to a robotic repair system, such as system 100. The waypoints may be generated based on known defect locations. The waypoints may cause the robotic repair unit to move a repair tool to the defect location and follow a modified trajectory. The waypoints may be based on an approximated surface around the defect to adequately treat or repair the workpiece surface. The waypoints may be generated by mapping the modified trajectory to the surface. The mapping may include Cartesian mapping or another suitable technique.

[0034] The robotic repair unit follows the modified trajectory to successfully repair the defect at block 470. Using method 400, the robotic repair unit can repair surfaces for which there is no known geometric model available.

[0035] Currently, many areas on the vehicle surface are classified as "no go" zones for the robotic repair unit. Defects in areas with curved surfaces or edges require a modification of the repair trajectory. The human performing the repair can intuitively adjust how to repair the curved surface by adjusting the motion, force, and speed. The systems and methods described herein may assist in modifying the repair trajectory to reduce the number of "no go" zones on the vehicle. However, it is expressly contemplated that the systems and methods herein may still classify some areas as unrepairable based on the detected topography. Detecting areas that are too difficult to repair based on the detected topography eliminates the need for manual labeling and continuous management of "no go zones" as the part may undergo design changes.

[0036] FIG. 5 illustrates a method for identifying surface topography on a workpiece surface according to an embodiment herein. The use of point cloud methods for feature detection is not unknown. For example, Kim et.al, "Extraction of Ridge and Valley Lines from Unorganized Points." Multimedia tools and applications 63.1 (2013), pp 265-279, describes such methods. As described herein, the mathematical model of method 500 differs in how polynomial surfaces are fitted to the workpiece surface and how extreme crossings are detected. Furthermore, in some embodiments, method 500 relies on sampling patches around the defect locations of interest, piecing together corresponding polynomial fits to provide a nearly continuous representation of the surface for efficient interpolation. However, other fitting methods may be suitable, such as fitting a NURBS surface and then using equations from that model to calculate derivatives, curvatures, and then derivatives of curvatures. Other methods may be suitable.

[0037] At block 510, a number of points on the surface are obtained. The sample points are obtained, for example, from a separate system, such as a vision system, that samples an area around the detected defect. The points are systematically sampled based on a grid measurement, as described in U.S. Provisional Patent Application No. 63 / 203,407. A plane is defined at the defect location and aligned to the surface normal at that location. A grid of points within a radius Rm of the central defect location is then projected onto the surface to provide a sampling of depth measurements for this plane. These projected points are used to fit a surface approximation model. The points in the planar grid are projected onto the workpiece surface of interest, which is used to measure the local surface topography. The Z axis (pointing out of the page) at the sampling center point is aligned with the surface normal of the point of interest (e.g., the defect). Then, a grid of points within a radius Rm from the center of the grid is projected onto the surface to provide a sampling of depth measurements for this plane. M Depth values ​​are measured from points within the surface. These points are used to fit a surface approximation model.

[0038] In block 520, an approximation model is constructed. In one embodiment, this is based on the radius R of the defect point measured in the sampling plane mentioned above. f This is done by fitting a series of local approximation models, one to each sample point in . Each local surface model is scaled by the radius R of the sample point at the center of the patch, measured in the sampling plane. p The local model is fitted using "patches" of nearby points that are within the surface. This local model is an analytical approximation of the subregion of the repair area on the underlying surface and can be used to calculate a curvature metric in that subregion. In one embodiment, the local model may be constructed from a bivariate polynomial function that is fitted to the subregion using least squares regression.

[0039] Other methods may be used to fit a global approximation model directly from the entire set of measurement points taken from the surface. Such methods may use a surface approximation constructed from splines. For example, one such approach is presented in Simple Method for Constructing NURBS Surfaces from Unorganized Points, Leal et al., Proceedings of the 19th International Meshing Roundtable pp 161-175.

[0040] In some embodiments, only a local fit function is used for each sampling point, so no global fit is required and surface quantities can be compared across local approximations of the surface.

[0041] At block 530, curvature metrics are approximated with a set of sample points generated from the mathematical model. These points may be sampled from the analytical surface model at a finer resolution than the resolution of the original surface measurements. Using the approximation model fit to 38, a set of metrics describing the surface topography is calculated at each sample point. These metrics include the magnitude (k1, k2) of the surface curvature along the principal axes of curvature (v1, v2), as well as the associated curvature derivatives (e1, e2) along each axis.

[0042] The analytical model used for this description is presented below as Equation 1. However, it is explicitly contemplated that other models are possible. Modeling the surface by a series of bivariate quadratic approximation functions gives the following implicit form of each approximation function: F(x,y)=z(x,y)-(ax 2 +by 2 +cxy+dx+ey+d) Equation 1

[0043] Taking the gradient of F and normalizing it gives us the vector N that is normal to the surface, as shown in Equation 2 below:

[0044]

number

[0045] We then take the eigenvalue decomposition of the gradient of the normal vector (▽N). The two eigenvectors associated with the two largest (based on absolute value) eigenvalues ​​are the vectors that lie along the principal axes of curvature. Let H be the Hessian (matrix of second partial derivatives) of F, then the associated curvature along vector v can be calculated using Equation 3 (as shown in Kim et al.):

[0046]

number

[0047] Each sample point has two principal axes of curvature, a minimum and a maximum, each with an associated curvature value. Each curvature value represents the inverse of the radius of a circle that is tangent to the surface along the associated axis. The principal axis of curvature at each point (v 1, v2) can be found by taking the eigenvalue decomposition of the gradient of the normal vector at that same point, as described above. The k and v metrics provide an analytical way to determine how much and in what direction the surface is "folded" at any given sample point.

[0048] The derivative of the curvature along the vector v can be calculated using Equation 4 (also given in Kim et al.).

[0049]

number

[0050] At block 540, the derivative of the curvature along each of the principal axes of curvature is calculated at each point sampled from the analytical model.

[0051] At block 550, the zero crossing points of the derivative of the curvature along the major axis of curvature are approximated. The zero crossings in the derivative of the curvature (e.g., where e1, e2 change sign between adjacent points) correlate with valleys and ridges on the surface, e.g., surface features. Thus, surface features can be detected by comparing local approximations of the curvature derivative (e1, e2) between sample points.

[0052] It is well noted in the literature that the principal axis of curvature is directionally ambiguous, which can lead to random flips of the computed eigenvectors (v1,v2), which in turn can lead to erroneous feature point detections (the signs of e1,e2 also flip).

[0053] With this in mind, when checking whether a zero crossing of the derivative of curvature (e) exists between two sample points, the relevant principal directions are locally aligned. For example, if the eigenvector v1 at pt1 is found not to be within 90 degrees of the eigenvector v1 calculated at the adjacent point, the representation at pt1 is inverted (both v1 and e1 are negated). Once they are aligned, the change in sign of e1 compared between these adjacent points reveals the relevant feature points.

[0054] Feature points are detected when the sign of the derivative of the curvature changes after aligning the curvature axes between adjacent points. The sign of the curvature value (k) at the intersection indicates whether the surface feature is a ridge or a valley, and the magnitude of the curvature value provides a metric of how sharp the ridge or valley is. For example, a feature point can be ignored if the calculated curvature magnitude (|k|) at that feature point is found to be below a user-defined tolerance, as this point may not be important enough to require special consideration when performing repairs.

[0055] Once a feature point is found to be between two sample points, the values ​​of the curvature derivative at those points are used to determine a more accurate estimate of the intersection point using Equation 5 below.

[0056]

number

[0057] This is in contrast to the method of Kim et al., which assigns the minimum of the derivative of the curvature to the "detected ridge point". Using a global approximation model, it is conceivable that more accurate methods of finding the exact intersection are possible. These methods can be performed using root-finding techniques such as Newton-Raphson.

[0058] To enable the robotic repair unit to repair a curved surface, a path may be projected directly onto a CAD model representing that surface. This assumes that the trajectory planner has full access to the CAD data and can thereby perform such path projection. In practice, these CAD files can be difficult to manage due to their complexity, dynamic nature (e.g., one part may go through multiple redesigns), and the underlying intellectual property agreements that must be established before such data can be shared. Method 500 allows the robotic repair unit to complete method 400 without the robotic controller needing to be aware of and manage these CAD files, instead by allowing sensors or auxiliary systems to provide sample points from which the surface is locally approximated to the robotic controller.

[0059] Once a model approximating the surface has been fitted, for example using method 500, the waypoints defining the templated robot repair trajectory can be mapped from the 2D plane to the 3D approximation. One way this mapping can be accomplished is by orthogonal projection. Other, more sophisticated means of mapping these path points are also contemplated, including means that attempt to limit radial distortions introduced by the 2D to 3D projection of the repair path.

[0060] Once the surface curvature information is modeled, a continuous supply of data is available to adjust process parameters at points along the repair trajectory, such as the applied force, the tool's approach angle, the travel speed, the rotation speed (for rotary tools), or the dispense speed (for dispensers). Each of these parameters can be adjusted within the process constraints to address surface features at each point along the trajectory. The analytical models described herein provide a smooth surface topography onto which discrete repair points can be mapped. The surface topography can be sampled as finely or coarsely as needed. Using discrete sampling, a spline curve can be fitted, which can then be used as the basis for modification of the applied force at each point on the path.

[0061] 6A-6B show a mapped trajectory on a surface according to an embodiment herein. FIG. 6A shows the result of mathematically modeling a work surface with surface features, the process of which is illustrated in the examples described in U.S. Provisional Patent Application No. 63 / 203,407, filed July 21, 2021. A set of measurements 602, 604 is sampled on a CAD surface 600 representing a vehicle. An analytical model is fitted to these points and the derivative of the curvature is measured at each sample point. When the derivative of the curvature reverses sign, a feature line 602 is detected. Repairs near this area may then be transformed to avoid this feature line or to be performed parallel and / or perpendicular to the feature line. Path speed, tool force, and tool velocity may also be adjusted.

[0062] 6B shows one example trajectory applied to a modeled surface 630. The normal 632 to the contact surface is shown, illustrating how the angle of attack can be changed to accommodate the surface topography.

[0063] 7A-7C show various ways in which the trajectory may be altered. FIG. 7A shows how the angle of attack of the surface treatment unit 700 may be changed to address a defect in a recessed area. The surface treatment unit 700, in the illustrated embodiment, must bring the defect into contact with the tool 702. On the left, the tool 702 will collide with the surface portion 710 and will not interact with the defect until significant damage is caused by biting into the surface 710, leaving a very noticeable surface treatment mark, which is undesirable in material removal applications. Instead, once the surface topography is provided to the robot controller, the angle of attack can be adjusted to cause the tool 702 to interact with the region 710 containing the defect, resulting in a much less noticeable surface treatment mark, as seen on the right.

[0064] In the context of a broader repair strategy generation process by the robot controller, knowledge of the surface topography may alter the order of defect repair, for example, repairing the defect while the tool 702 with the smaller surface area is attached to the unit 700. Alternatively, in a system with two units 700, the unit 700 with the smaller tool 702 may be assigned to address the illustrated defect.

[0065] FIG. 7B illustrates a nonlinear transformation 720 of a trajectory around a curved surface. Surface treatment near a surface feature can generally be done in two ways: by incorporating the surface feature into the repair strategy, as shown in FIG. 7B, or by avoiding the surface feature, as shown in FIG. 7C. In FIG. 7B, a defect 722 is located on a curved surface 728. The initial defect strategy may be, for example, a circular repair 724 around the defect 722. However, a circular repair mapped onto a surface containing the defect 722 may leave undesirable marks on the surface, as the robot must perform highly dynamic maneuvers to follow the contoured surface. This is not preferred. Instead, it is desirable to deform the repair trajectory and wrap it around the curvature of the surface 728 to incorporate that curvature, resulting in trajectory 726. In some embodiments, the transformation from trajectory 724 to trajectory 726 is an in-phase transformation, where every point on trajectory 724 is mapped to a point on trajectory 726. While FIG. 7B illustrates a transformation that extends the trajectory 726 beyond the radius of the trajectory 724 in at least some directions, it is also expressly contemplated that an in-phase transformation may occur that maintains the trajectory 726 within the boundaries of the trajectory 724. Additionally, while FIG. 7B illustrates a circular trajectory 724 transforming into a boomerang-like shape 726, it is expressly contemplated that other shapes may be suitable or appropriate, such as a bean shape, an ellipse, or another suitable shape that includes, for example, a scalar increase or decrease in the trajectory diameter. The shape of the trajectory 724 may be determined based on the shape or size of the defect to be repaired, and the resulting shape and size of the trajectory 726 may be determined based on the shape, size, or magnitude of the curvature of the surface feature. Although an in-phase transformation that generally maintains the trajectory shape has been described, other transformations may be appropriate based on the surface topography and defect specifications. For example, a spiral may be preferred around a groove, while a zigzag pattern may be used within the groove. A spiral to line transformation is an example of a non-in-phase transformation. Similarly, in a non-homogenous transformation, all points in the region beyond the feature line can be mapped to a line adjacent to the feature line.

[0066] FIG. 7C illustrates a transformation 730 that modifies the geometry of the trajectory. An initial trajectory 736 is selected for the defect 732. However, the trajectory 736 may bring the surface treatment tool too close to the surface feature 734, causing damage outside of the desired defect repair zone. Therefore, a modified trajectory 738 is generated. As shown in FIG. 7C, the modified trajectory 738 is off-center from the defect 732, although in other embodiments it may be centered on the defect 732. The linear transformation may stretch, rotate, or translate the defect, as illustrated in FIG. 7C.

[0067] However, it may not be possible to achieve defect repair by simply stretching, compressing, or translating the track.

[0068] 8A-8D show an exemplary trajectory correction process described in the method of FIG. 8B. As shown in FIG. 8D, the surface feature can be identified by a boundary indicated by point 802. In some embodiments, it cannot be crossed by the repair trajectory, as shown in FIG. 8A. The challenge in programming the robotic surface processing unit is then to smoothly process the area under the red dot away from the boundary point 802 so that the repair can be performed. FIGS. 8A-8D show an embodiment in which the repair path is a curved repair path. While it may be easy for a human to make a sharp turn (e.g., a 90° turn) during a repair, an acceleration penalty is incurred for the robotic unit to do so. Furthermore, stopping the operation of the robotic system and restarting it in a different direction can result in jerky motion. In the case of an abrasive repair, surface marks may be left on the work surface. Furthermore, the acceleration penalty may take more time to perform. In the example of vehicle defect repair, it is desirable to repair the defect as quickly as possible so that the vehicle can quickly proceed through the repair process. A smooth repair trajectory allows the robotic repair to occur in target areas that would otherwise be inaccessible to the robot, yet still keeps the repair within an acceptable process window.

[0069] 8B illustrates a method for trajectory correction process 850. As shown in block 852, the detected feature points (e.g., from method 500) are projected onto a 2D plane above the defect, as shown by defect point 810. The feature points are connected, in one embodiment, into feature lines, which are spaced apart from each other by a distance R c This is done by considering that feature points within are considered to be connected.

[0070] In block 860, a set of discrete mapping points M(θ)→d is generated, where θ is the angle from a fixed axis in the 2D plane, and d is the distance from the center to the closest boundary point in the direction of θ (whether it is a point on the original template boundary or a point on the detected feature line). This is done by discretely sampling θ and projecting a ray from the repair center to the closest boundary point. An initial trajectory 822 with an initial boundary 820 is obtained. The trajectory 822 can be obtained based on multiple factors, including process constraints such as how fast a tool or robot can move within the area of ​​the detected surface features. The trajectory 822 may also be obtained based on the specifications of the defect or work surface. For example, it may be desirable to maintain orange peel on the vehicle surface or blend the repair area into an untreated portion of the vehicle surface. Similarly, the trajectory 822 may be selected based on the detected surface features.

[0071] At block 870, the transformation of the trajectory parameters is completed. A trajectory as described herein includes not only a series of waypoints along which the surface treatment tool travels, but also other process parameters attributable to the tool, including the angle of attack or contact, the tool speed, the speed of movement of the robot, the force applied (or the dispensing rate of an adhesive or film, etc.), or any other relevant process parameters, such as the type or size of the abrasive article, temperature, humidity, etc. Each of these parameters may need to be transformed at each set of waypoints, and the waypoints themselves may also need to be transformed.

[0072] As described above, a set of mapping points is first generated that mark where the outer boundary intersects with the detected feature lines. In some embodiments, the mapping points undergo a smoothing step (before interpolation) to remove sharp "kinks" (such as where the outer boundary 820 meets the feature line 810). This smoothing is done by applying a two-stage process that includes first applying Gaussian filtering to the distance mapping to smooth edges, and secondly, iteratively shrinking regions of the resulting smoothed distance map that violate the original unsmoothed boundary, since Gaussian filtering can cause constraint violations. For example, the violating regions in the smoothed distance mapping may be iteratively scaled by an inverse bell curve until they adequately match the original template boundary and the imposed feature lines. The inverse bell curve is modeled by Equation 6:

[0073]

number

[0074] In Equation 6, m and d are adjustable parameters that control the overall shape of the bell curve, and t is the angular distance from the identified constraint violation to which the coefficient p is applied. In this example, when shrinking a point in the distance mapping, the radial direction of the mapping point remains unchanged.

[0075] The result is a set of smooth mapping points that lie within the original boundary constraints defined by the initial template and the detected surface features.

[0076] These mapping points are then interpolated. This can be done using linear segments or via splines. Herein, B-splines are used to perform the mapping point interpolation. However, it is explicitly envisioned that other methods are possible. The result is a continuous, smooth outer mapping boundary function that produces the maximum allowable distance given any angle θ.

[0077] A smooth mapping boundary can then be used to map the set of waypoints to lie on one side of the feature line, which in one embodiment is done by proportionally scaling the distance from the defect where the waypoint is located by a factor d / r, where d is the sampled distance on the outer mapping boundary function (same angle as the original untransformed points) and r is the radius of the untransformed template mapping (820).

[0078] Transformations of the track shape or other parameters may be linear or non-linear. For example, a linear transformation of the track shape may simply shrink or stretch the repair. A non-linear transformation of the track shape may distort the repair to fit around an edge.

[0079] Similar transformation processes can be performed for other process parameters, for example the force applied at any given waypoint can be similarly scaled by d / r or a ramp function can be applied as described in the examples.

[0080] The transformation step of block 870 may also include energy minimization methods, such as potential field methods. In one energy minimization method, "virtual springs" are assigned between the mapping points of a mesh covering the repair template. The mesh of points is then subjected to a forcing function that shifts the mesh gradually away from the no-go areas (e.g., the left side of the circular repair illustrated herein). Other forces may be applied as well, such as forces that keep the mesh roughly centered on the defect. Since the points can shift under the influence of these forces, the virtual springs assigned between the mesh points act to stabilize the mesh and maintain the mesh topology during the transformation. Once these mapping points have moved to one side of the feature line, they can be used to map other points from the untransformed shape to take on the approximate shape of the transformed mapping.

[0081] Systems and methods are described herein that focus on adjusting the trajectory. However, it is explicitly assumed that the travel path is only one part of the surface treatment operation. In the defect repair example, the robot speed, the tool rotational (and / or vibration) speed, the angle of attack, and also the applied force are all important to achieve a high quality repair. In fact, force and velocity may be more important to the repair quality than the trajectory alone. As shown in block 880, the interpolation process 870 may be repeated for each process constraint. However, some of these parameters may be interrelated. For example, force and velocity may be mapped together.

[0082] Further, systems and methods are described herein that assume a calculated trajectory to proceed forward. However, while the systems and methods herein enhance the ability of a robotic repair unit to perform surface treatment operations on a variety of surface topologies, it is expressly contemplated that there may still be areas where the method 850 is unable to generate a trajectory that satisfies all of the set constraints, and an indication may be returned that the target area is "no go" or a non-robot repairable region.

[0083] 8A-8D show a method for generating a repair trajectory that avoids surface boundaries, however, it is explicitly contemplated that the repair trajectory may cross or encompass surface features detected on the surface topography, as described in Figures 7A-7C.

[0084] In block 895, the repair point is mapped by using the arctangent of its coordinates to calculate (θ) where the repair point is located. The fitted b-spline is evaluated at (θ) to obtain a distance mapping value d. The repair point is then scaled towards the center of the defect by a factor d / r, where r is the radius of the unmapped template.

[0085] In the specific example shown in Figures 8A, 8C, and 8D, a 1D Gaussian filter is applied to smooth the transition between the boundary 820 and the intersection of the feature 810. In this specific example, the smoothing occurs after map generation and before constraint satisfaction / map scaling. This results in a smoother mapping of the sample mapping points 832. Although a 1D Gaussian filter is described, other smoothing operations may be performed. Additionally, smoothing may be performed in a different order or multiple times. In the illustrated example, smoothing is performed to shrink the resulting mapping to within a desired boundary, e.g., to one side of a surface boundary. This can be done by repeatedly multiplying the smoothed distance map with the inverse bell curve 840 of Figure 8C in areas where the constraints are violated. An interpolated B-spline is then fitted to the smoothed point map to create a smooth continuous representation of the discrete mapping, as shown in Figure 8D.

[0086] 9A shows a modified trajectory 910 aligned on a mapped portion of a workpiece surface 900. FIG.

[0087] FIG. 10 illustrates a method of modifying a repair strategy according to an embodiment herein. As described in more detail in U.S. Provisional Patent Application No. 63 / 203,407, filed July 21, 2021, multiple surface points are sampled and patched so that they overlap. Edges can be detected wherever patches are fitted. The patched area 912 includes the repair trajectory 910 in some embodiments. The surrounding area 904 may include sampled areas without patches or without mathematical model fitting. The surrounding area 904 may provide perimeter data to help better understand the patched area 912. Any analytical model may be used, such as linear, nonlinear, spline surface fitting, or another suitable model.

[0088] Figures 9B-9E show modified trajectories based on detected surface features detected, for example, using the method of Figure 5, and modified, for example, according to the method described in the method of Figure 8B. Figure 9B shows trajectory 914 near surface feature 912. Figure 9C shows trajectory 934 abutting feature 932. Figure 9D shows trajectory 924 modified to address a defect between two surface features 922. And Figure 9E shows trajectory 942 between two features 944.

[0089] 10 illustrates a method for generating a modified surface processing strategy according to an embodiment herein. Method 1000 is described in the context of repairing defects on a workpiece surface for purposes of understanding only, and applies to other surface processing operations such as material removal, adhesive dispensing, film wrapping, etc. Method 1000 may be performed locally by a robot controller or by a surface processing module in a cloud server that sends a trajectory with process parameters to a robot controller for execution.

[0090] A set of topographical points is received at block 1010. This point cloud may be received from a vision system in some embodiments.

[0091] At block 1020, a model is applied to these points, for example, using method 500. These points may include indications of where surface features are detected near the detected defects.

[0092] At block 1030, a trajectory template is received. The trajectory template may be received, for example, from a trajectory database. In some embodiments, the acquired trajectory includes a repair plan of forces, angles, and velocities to apply to points on the trajectory. In other embodiments, the repair plan parameters are independently selected based on the modified trajectory.

[0093] In block 1040, trajectories are mapped to the defects based on the detected surface features, for example, as described and shown in FIG.

[0094] At block 1050, process parameters are selected for the trajectory. The process parameters may be assigned based on discrete points along the mapped trajectory or based on another method. Multiple process parameters are available for modification to address both defects and any detected surface features. Tool selection 1052 may, for example, adjust the size or type of abrasive article. The applied force 1054 may be increased or decreased. The tool angle of attack 1056 may be increased or decreased. The tool rotational speed 1058 may be increased or decreased. The tool speed 1062 through the trajectory may be increased or decreased. Each of these parameters is set for each point along the trajectory to form a repair plan.

[0095] At block 1060, the repair plan is executed by the robotic repair unit.

[0096] FIG. 11 illustrates a process mapping system according to embodiments herein. The process mapping system 1100 may be incorporated into a robot controller of a robotic repair unit in some embodiments. In other embodiments, the process mapping system 1100 may be remote from the robotic repair unit 1170 as shown in FIG. 11. Additionally, while FIG. 11 illustrates an embodiment in which the trajectory database 1150 is separate and remote from the process mapping system 1100, it is expressly contemplated that the process mapping system 600 may be located on a server having the trajectory database 1150. It is expressly contemplated that configurations of components other than the configuration shown in FIG. 11 are envisioned.

[0097] The defect detection system 1110 detects defects on the workpiece surface and provides defect and surface information to the process mapping system 1100. The defect detection system 1110 may be an imaging system as described with respect to FIG. 1. However, other systems 1110 may be possible. The defect detector 1112 detects defects on the workpiece surface in the vehicle repair example. However, in an adhesive dispensing embodiment, the adhesive target area detector 1112 detects or identifies where adhesive should be dispensed. In a material removal embodiment, the target removal area 1112 detects where material should be removed from the surface. The surface sampler 1116 samples the surface around the detected defects / adhesive dispensing targets / target removal areas. The defect detection system 1110 may also include other functions 1118. For example, the defect detection system 1110 may be a camera system that images the surface before and after the defect removal / adhesive dispensing / material removal operation.

[0098] The robotic repair unit 1170 is a mechanical unit responsible for implementing the repair trajectory on the work surface. The robotic repair unit 1170 includes a robotic arm 1172 that moves a polishing tool 1180 into position for the repair operation. The robotic arm 1172 may have multiple movement mechanisms (motorized or otherwise) that move the polishing tool 1180 through the trajectory, as shown in Figures 7-9. The robotic repair unit 1170 may also include a force control unit 1174 that applies force to the polishing tool 1180 via an end effector.

[0099] The defect information receiver 1102 obtains information about the defects, such as from a defect detection system 1110. The defect information obtainer 1102 may obtain defect locations relative to a CAD model or may receive other location information.

[0100] The surface information acquirer 1104 acquires the surface sampling information acquired by the surface sampler 1116. Based on the acquired surface sampling, a model generator 1120 can generate a mathematical model to approximate the curvature and detect surface features near the defect. A surface approximator 1122 approximates the surface. This can be done using a polynomial approximation, an approximation using a spline surface, or another suitable approximation. A curvature approximator 1124 determines the magnitude and direction of the curvature at each of the sampling points based on the surface approximation. A derivative calculator 1126 calculates the derivative of the curvature along the surface. Based on the derivative calculation, a feature identifier 1128 identifies a set of feature points, for example, an edge, a concave curve, a convex curve, or another feature.

[0101] The trajectory generator 1140 generates a modified trajectory based on the surface curvature. The defect obtainer 642 obtains information about the defect, for example from the defect detection system 1110. The obtained defect information may include defect type, size, location, severity, or other information that may be useful for selecting a trajectory template. The feature obtainer 1146 obtains information from the feature identifier about any identified features near the defect that may interfere with the defect repair.

[0102] The trajectory corrector 1148 modifies the trajectory template based on known information about the defects from the defect obtainer 1142 and known information about the surface features from the feature obtainer 1146. The trajectory obtainer 1134 can obtain the trajectory template from a trajectory database 1150. The trajectory database 1150 can include a plurality of trajectories that can be selected based on defect parameters such as defect type, defect location, defect size, defect severity, etc., as shown in block 1154. The trajectory can also be selected based on the selected abrasive article 1152 or tool. For example, the size or type of the abrasive disc or back-up pad may dictate the trajectory selection, at least to some extent. Information about the workpiece surface, as shown in block 1156, such as the workpiece surface material (e.g., wood, plastic, metal, paint, clear coat, layers thereof, etc.), can affect the trajectory selection. Similarly, process constraints 1162 of the robotic repair unit 1170, such as the maximum force that the force control unit 1174 can exert, the maximum RPM of the polishing tool, or the clearance ranges required for different components, may influence the trajectory selection. Based on known information about the defect, the surface, and the robotic repair unit 1170, an initial trajectory shape may be selected from trajectory shapes 658, such as a spiral, circle, ellipse, hypotrochoid, rosette, or other suitable shape.

[0103] Based on all of the acquired information, the trajectory corrector 1148 modifies the received trajectory template. The in-phase transformer 1138 can perform an in-phase transformation on the trajectory template. The translator 1138 can translate the trajectory template or the transformed template. Similarly, the in-phase transformer 1138 can translate the template or the translated template. The force adjuster 1144 selects the force to be applied along the trajectory. The force adjuster may select the force to be applied at the discrete points with smoothing between the points where different forces are applied, or may select the force profile to be applied along different portions of the trajectory template. The speed adjuster 1145 adjusts the speed at which the tool moves through the trajectory and may be selected based on the speed at which the tool should move through the discrete points, the acceleration or deceleration between the points, or the speed to be achieved at each point or along a portion of the trajectory. The tool speed adjuster 1143 adjusts the speed at which the tool moves, e.g., rotational motion, orbital motion, oscillatory motion, etc.

[0104] Based on the modified trajectory, waypoints may be generated by waypoint generator 1149. Waypoints may be generated to guide the robot arm 1172 from the previous defect or from a current location, to the defect location, and through the modified trajectory to complete the repair.

[0105] The systems and methods herein sample the workpiece surface and fit an analytical model to the sampled points. Based on the detected surface features, the surface processing system generates a repair plan by generating a trajectory that incorporates known information about defects and surface topography on the workpiece surface. The repair plan also includes process variables along the trajectory based on the known defect information and surface topography.

[0106] FIG. 12 is a surface processing system architecture. The surface processing architecture 800 illustrates one embodiment of an implementation of the defect detection and ranking system 1210. As an example, the surface processing system 1200 can provide computing, software, data access, and storage services that do not require end-user knowledge of the physical location or configuration of the system delivering the services. In various embodiments, the remote server can deliver the services over a wide area network, such as the Internet, using an appropriate protocol. For example, the remote server can deliver applications over a wide area network, which can be accessed through a web browser or any other computing component. The software or components shown or described in FIGS. 1-11 and corresponding data can be stored on a server at a remote location. Computing resources in a remote server environment can be aggregated at a remote data center location, or they can be distributed. The remote server infrastructure can deliver services through shared data centers, which appear as a single access point to the user. Thus, the components and functions described herein can be provided from a remote server at a remote location using a remote server architecture. Alternatively, they may be provided by a traditional server, installed directly on the client device, or provided in other manners.

[0107] In the example shown in Figure 12, some items are similar to those shown in the previous figures. Figure 12 specifically illustrates that the process mapping system 1210 can be located at a remote server location 1202. Thus, the computing device 1220 accesses those systems through the remote server location 1202. The operator 1250 can also access the user interface 1222 using the computing device 1220.

[0108] 12 shows that it is also contemplated that some elements of the system described herein are located at the remote server location 1202 while others are not. As an example, the storage 1230, 1240, or 1260, or the robotic repair system 1270, can be located at a location separate from the location 1202 and accessed via a remote server at the location 1202. Regardless of where they are located, they can be accessed directly by the computing device 1220 over a network (either a wide area network or a local area network), hosted at the remote site by a service, provided as a service, or accessed by a connection service present at the remote location. Data can also be stored virtually anywhere and accessed by or transferred to the parties intermittently. For example, a physical carrier can be used instead of or in addition to an electromagnetic carrier.

[0109] It should also be noted that elements of the systems described herein, or portions thereof, can be located on a wide variety of different devices, including, but not limited to, servers, desktop computers, laptop computers, embedded computers, industrial controllers, tablet computers, or other mobile devices, such as palmtop computers, cell phones, smartphones, multimedia players, personal digital assistants, and the like.

[0110] 13-15 show examples of computing devices that can be used in the embodiments shown in the preceding figures.

[0111] Figure 13 is a simplified block diagram of an exemplary embodiment of a handheld or mobile computing device that may be used as a handheld device 1316 of a user or client (such as computing device 1220 of Figure 12) on which the system (or portions thereof) may be deployed. For example, a mobile device may be deployed within an operator compartment of computing device 1220 for use in generating, processing, or displaying data. Figure 140 is another embodiment of a handheld or mobile device.

[0112] 13 provides a schematic block diagram of components of a client device 1316 capable of executing some of the components shown and described herein. The client device 1316 interacts with or executes some of and interacts with some of the components. The device 1316 is provided with a communication link 1313 that allows the handheld device to communicate with other computing devices and under some embodiments provides a channel for automatically receiving information, such as by scanning. Examples of communication link 1313 include wireless services used to provide cellular access to a network, as well as protocols that provide local wireless connections to a network, enabling communication via one or more communication protocols.

[0113] In another embodiment, the application may be received on a removable Secure Digital (SD) card that is connected to the interface 1315. The interface 1315 and communication link 1313 communicate with a processor 1317 (which may also embody a processor) along a bus 1319, which is also connected to memory 1321 and input / output (I / O) components 1323, as well as a clock 1325 and a position information system 1327.

[0114] I / O components 1323, in one embodiment, are provided to facilitate input and output operations, and devices 1316 may include input components such as buttons, touch sensors, optical sensors, microphones, touch screens, proximity sensors, accelerometers, orientation sensors, etc., and output components such as display devices, speakers, and / or printer ports. Other I / O components 1323 may be used as well.

[0115] The clock 1325 illustratively includes a real-time clock component that outputs the time and date, and may also provide timing functions for the processor 1317.

[0116] Illustratively, the location information system 1327 includes components that output the current geographic location of the device 1316. This may include, for example, a global positioning system (GPS) receiver, a LORAN system, a dead reckoning system, a cellular triangulation system, or other positioning systems. It may also include, for example, mapping or navigation software that generates desired maps, navigation routes, and other geographic features.

[0117] Memory 1321 stores operating system 1329, network settings 1331, applications 1333, application configuration settings 1335, data storage 1337, communication drivers 1339, and communication configuration settings 1341. Memory 1321 may include all types of tangible, volatile computer readable memory and non-volatile computer readable memory devices. It may also include computer storage media (discussed below). Memory 1321 stores computer readable instructions that, when executed by processor 1317, cause the processor to perform computer-implemented steps or functions in accordance with the instructions. Processor 1317 may also be activated by other components to facilitate their functions.

[0118] 14 shows that the device may be a smartphone 1401. The smartphone 1471 has a touch-sensitive display 1473 that displays icons or tiles or other user input mechanisms 1475. The mechanisms 1475 may be used by a user to run applications, make calls, perform data transfer operations, etc. Generally, smartphones 1471 are built on mobile operating systems and offer more advanced computing capabilities and connectivity than feature phones.

[0119] It should be noted that other configurations of the device 1416 are possible.

[0120] FIG. 15 is a block diagram of a computing environment that can be used in the embodiments shown in the preceding figures.

[0121] FIG. 15 is an example of a computing environment in which elements of the systems and methods described herein, or portions thereof (for example), may be deployed. Referring to FIG. 15, an exemplary system for implementing some embodiments includes a general-purpose computing device in the form of a computer 1510. Components of the computer 1510 may include, but are not limited to, a processing unit 1520 (which may include a processor), a system memory 1530, and a system bus 1521 that couples various system components, including the system memory, to the processing unit 1520. The system bus 1521 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The memory and programs described with respect to the systems and methods described herein may be deployed in the corresponding portions of FIG. 15.

[0122] Computer 1510 typically includes a variety of computer readable media. Computer readable media may be any available media that can be accessed by computer 1510 and includes both volatile and nonvolatile media and removable and non-removable media. By way of example, and not limitation, computer readable media may include computer storage media and communication media. Computer storage media is distinct from and does not include modulated data signals or carrier waves. Computer storage media includes hardware storage media, including both volatile and nonvolatile, removable and non-removable media, implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by computer 1510. Communication media may embody computer readable instructions, data structures, program modules or other data in a transport mechanism and include any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0123] The system memory 1530 includes computer storage media in the form of volatile and / or nonvolatile memory such as read only memory (ROM) 1531 and random access memory (RAM) 1532. A basic input / output system (BIOS) 1533, containing the basic routines that help to transfer information between elements within the computer 1510, such as during start-up, is typically stored in ROM 1531. RAM 1532 typically contains data and / or program modules that are immediately accessible to and / or presently being operated on by the processing unit 1520. By way of example, and not limitation, FIG. 15 illustrates operating system 1534, application programs 1535, other program modules 1536, and program data 1537.

[0124] The computer 1510 may also include other removable / non-removable, volatile / non-volatile computer storage media. By way of example only, Figure 15 illustrates a hard disk drive 1541, a non-volatile magnetic disk 1552, an optical disk drive 1555, and a non-volatile optical disk 1556, which read from or write to non-removable, non-volatile magnetic media. The hard disk drive 1541 is typically connected to the system bus 1521 through a non-removable memory interface, such as interface 1540, and the optical disk drive 1555 is typically connected to the system bus 1521 by a removable memory interface, such as interface 1550.

[0125] Alternatively, or in addition, the functions described herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0126] The drives and their associated computer storage media discussed above and illustrated in Figure 15 provide storage of computer readable instructions, data structures, program modules and other data for the computer 1510. In Figure 15, for example, hard disk drive 1541 is illustrated as storing operating system 1544, application programs 1545, other program modules 1546, and program data 1547. Note that these components can either be the same as or different from operating system 1534, application programs 1535, other program modules 1536, and program data 1537.

[0127] A user may enter commands and information into the computer 1510 through input devices such as a keyboard 1562, a microphone 1563, and a pointing device 1561, such as a mouse, trackball, or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite receiver, scanner, or the like. These and other input devices are connected to the processing unit 1520 through a user input interface 1560 that is often coupled to the system bus, although they may be connected by other interface and bus structures. A visual display 1591 or other type of display device is also connected to the system bus 1521 via an interface, such as a video interface 1590. In addition to the monitor, computers may also include other peripheral output devices, such as speakers 1597 and printer 1596, which may be connected through an output peripheral interface 1595.

[0128] The computer 1510 operates in a networked environment using logical connections, such as a Local Area Network (LAN) or a Wide Area Network (WAN), to one or more remote computers, such as a remote computer 1580.

[0129] When used in a LAN networking environment, the computer 1510 is connected to the LAN 1571 through a network interface or adapter 1570. When used in a WAN networking environment, the computer 1510 typically includes a modem 1572 or other means for establishing communications over the WAN 1573, such as the Internet. In a networked environment, program modules may be stored in remote memory storage devices. Figure 15 illustrates, for example, that remote application programs 1585 may reside on the remote computer 1580.

[0130] A robotic system is presented that includes a surface inspection system that receives sampling information for a plurality of areas within a region of a workpiece surface. The system also includes a robot arm coupled to a surface engagement tool, the robot repair arm configured to engage the surface treatment tool with the region of the workpiece surface. The system also includes a process mapping system configured to approximate a surface topography of the region of the workpiece surface based on the sampling information and generate a surface treatment plan for the region based on the approximated surface topography, the surface treatment plan including a trajectory. The surface treatment plan includes one of a force profile along the trajectory, a velocity profile of the surface engagement tool along the trajectory, a rotational velocity profile of the surface engagement tool along the trajectory, and a trajectory modification that accounts for the presence of surface features identified in the approximated surface topography. The process mapping system is also configured to generate a control signal for the robot arm that includes the surface treatment plan.

[0131] The system may be implemented such that the surface treatment plan includes trajectory modifications that take into account the presence of surface features identified in the approximated surface topography, the surface features including concave, convex, or edge areas.

[0132] The system may be implemented such that the surface treatment plan includes an angle of attack profile of the surface engaging tool along the trajectory.

[0133] The system may be implemented such that the surface treatment plan includes trajectory modifications that account for the presence of surface features identified in the approximated surface topography. The trajectory modifications include an isomorphic transformation of the trajectory template.

[0134] The system may be implemented such that the surface treatment plan includes trajectory modifications that take into account the presence of surface features identified in the approximated surface topography. The trajectory modifications include discontinuous trajectories.

[0135] The system may be implemented such that the discontinuous trajectory skips over identified surface features.

[0136] The system may be implemented such that the transformation extends a trajectory over the identified surface features.

[0137] The system may be implemented such that the transformation moves the trajectory away from the identified surface feature.

[0138] The system may be implemented such that the orbit has a perimeter with a shape, and the transformation changes that shape.

[0139] The system may be implemented such that the shape is stretched from a first shape to a second shape.

[0140] The system may be implemented such that a shape is compressed from a first shape to a second shape.

[0141] The system may be implemented such that a portion of the shape is compressed.

[0142] The system may be implemented such that the compression avoids identified surface features.

[0143] The system may be implemented such that the compression avoids boundaries around identified surface features.

[0144] The system may be implemented such that a portion of the shape is stretched.

[0145] The system may be implemented such that the stretching extends across the identified surface features.

[0146] The system may be implemented such that the work surface is a vehicle.

[0147] The system may be implemented such that the surface inspection system includes a vision system.

[0148] The system may be implemented such that the vision system includes a camera.

[0149] The system may be implemented such that the region includes a defect and the surface engaging tool is a material removal tool.

[0150] The system may be implemented such that the surface treatment tool is a sander or an abrasive tool.

[0151] The system may be implemented such that the surface treatment plan includes trajectory corrections that account for the presence of surface features identified in the approximated surface topography, and the process mapping system identifies the surface features by mapping each of the sampling areas within the region, approximating a curvature at each of the sampling areas within the region, calculating a derivative of the approximated curvature at each of the sampling areas, and identifying the surface features within the region based on the derivative calculation.

[0152] The system may be implemented such that a trajectory includes a series of waypoints through a region.

[0153] The system may be implemented such that a trajectory template is selected based on defect size, defect location, defect type, or defect severity.

[0154] The system may be implemented such that the surface treatment plan includes trajectory modifications that take into account the presence of identified surface features in the approximated surface topography, the trajectory modifications being based on the identified surface features.

[0155] The system may be implemented such that the track is stretched or compressed.

[0156] The system may be implemented such that the orbit shape is a circle, an ellipse, a rosette, a spiral, or a hypotrochoid.

[0157] The system may be implemented such that the trajectory is scaled up or down proportionally.

[0158] The system may be implemented such that the trajectory corrections include modified tool force, disk velocity, or tool speed.

[0159] The system of claim 1 , wherein the robotic arm executes the control signals and follows the trajectory.

[0160] 23. The system of claim 22, wherein the approximated curvature is a polynomial approximation, a spline surface, or a support vector machine.

[0161] The system may be implemented such that the approximated curvature is a polynomial approximation, including polynomial patches.

[0162] The system may be implemented such that the surface treatment plan includes trajectory modifications that account for the presence of surface features identified in the approximated surface topography, the trajectory modifications including mapping a trajectory template boundary to lie on one side of an identified surface boundary in the surface topography, interpolating the mapped trajectory, and mapping a number of waypoints from the mapped trajectory.

[0163] The system may be implemented such that mapping includes fitting a spline to a number of waypoints to create a continuous path.

[0164] The system may be implemented such that the interpolation includes B-spline, quadratic, cubic, quintic, NURB, piecewise continuous spline, and polychain interpolation.

[0165] The system may also include a smoothing step.

[0166] The system may be implemented such that the smoothing step applies a Gaussian filter.

[0167] The system may be implemented such that a second smoothing is applied, the second smoothing comprising multiplying the mapped trajectory with an inverse bell curve.

[0168] The system may be implemented such that the identified surface boundaries are constraining features.

[0169] The system may be implemented such that the identified surface boundary includes surface features and buffer spaces from the surface features.

[0170] The system may be implemented such that the trajectory template has a perimeter, and mapping the trajectory template includes mapping the perimeter to the identified surface boundary.

[0171] The system may be implemented such that the surface treatment plan includes trajectory modifications that take into account the presence of surface features identified in the approximated surface topography, where identifying the surface features includes an energy minimization method.

[0172] The system may be implemented such that the energy minimization method includes a potential field method, a, or snake interpolation.

[0173] The system may be implemented such that the method includes a dynamic energy minimization smoothing step, during which penalties are assigned for crossing boundaries of surface features.

[0174] A system for generating a repair plan for defects on a workpiece surface is presented, the system including a surface sampling receiver for receiving a surface topography of the workpiece surface, a defect indication receiver for receiving an indication of the defects on the workpiece surface, a process constraint receiver for receiving parameter constraints for a robotic repair unit, and a trajectory corrector for correcting a trajectory template. The trajectory corrector includes a converter for converting the trajectory template into a transformed trajectory based on the surface topography. The system also includes a repair plan generator for generating a repair plan based on the transformed trajectory and for setting process conditions along the transformed trajectory. The repair plan generator includes a force regulator for setting an applied force of a tool on the workpiece surface, a speed regulator for setting a speed at which the tool moves across the workpiece surface, a tool speed regulator for setting a rotational speed of the tool, and a control signal generator for communicating the generated repair plan to a robot controller, the robot controller automatically implementing the repair plan and completing the defect repair based on the repair plan.

[0175] The system may be implemented such that the translation trajectory is spaced from the surface feature by a boundary.

[0176] The system may be implemented such that the transformation trajectory is translated.

[0177] The system may be implemented such that the transformed trajectory is off-centered from the defect.

[0178] The system further includes a converter that projects the surface features onto a two-dimensional plane, maps the trajectory template onto a boundary, and transforms the mapped trajectory template, including modifying the trajectory parameters; Mapping the transformed trajectory onto the surface topography may be implemented to obtain the transformed trajectory.

[0179] The system may include smoothing the interpolated trajectory.

[0180] The system may also include iteratively smoothing the interpolated trajectory until the constraints are satisfied.

[0181] The system may be implemented such that the constraints are surface boundaries within the surface topography.

[0182] The system may be implemented such that the representation of the surface features is generated by a surface model generator that includes a surface approximator that uses approximations to approximate the surface at each of a plurality of surface samples, a curvature approximator that approximates the curvature and a derivative of the curvature at each of the plurality of surface samples, and a feature detector that identifies points of zero curvature as surface features based on the derivative of the curvature approximation at each of the surface samples.

[0183] The system may be implemented such that a number of surface samples are received from a CAD model.

[0184] The system may be implemented such that a number of surface samples are received from the surface measurement system.

[0185] The system may be implemented such that the surface measurement system includes a camera.

[0186] The system may be implemented such that the approximation is a polynomial approximation.

[0187] The system may be implemented such that the polynomial approximation is a third order polynomial.

[0188] The system may be implemented such that multiple surface samples are within a radius of the defect.

[0189] The system may be implemented such that the work surface is a vehicle surface and the surface feature is a concave, convex, or edge on the vehicle surface proximate to the defect.

[0190] The system may be implemented such that a number of surface samples are received from a vision system that images the workpiece surface.

[0191] The system may also include an angle of attack adjuster that sets the angle of attack of the tool.

[0192] The system may be implemented such that the modified trajectory includes a discontinuous trajectory.

[0193] The system may be implemented such that the discontinuous trajectory skips over identified surface features.

[0194] The system may be implemented such that the transformation extends a trajectory over the identified surface features.

[0195] The system may be implemented such that the transformation moves the trajectory away from the identified surface feature.

[0196] The system may be implemented such that the orbit has a perimeter with a shape, and the transformation changes that shape.

[0197] The system may be implemented such that the shape is stretched from a first shape to a second shape.

[0198] The system may be implemented such that a shape is compressed from a first shape to a second shape.

[0199] The system may be implemented such that a portion of the shape is compressed.

[0200] The system may be implemented such that the compression avoids identified surface features.

[0201] The system may be implemented such that the compression avoids boundaries around identified surface features.

[0202] The system may be implemented such that a portion of the shape is stretched.

[0203] The system may be implemented such that the stretching extends across the identified surface features.

[0204] The system may be implemented such that the work surface is a vehicle.

[0205] The system may be implemented such that a surface sampling receiver receives surface samples from the vision system.

[0206] The system may be implemented such that the vision system includes a camera.

[0207] The robotic system may be implemented such that the tool is a sander or an abrasive tool.

[0208] The system may be implemented such that a trajectory template is selected based on defect size, defect location, defect type, or defect severity.

[0209] The system may be implemented such that the trajectory template has a shape, the shape being a circle, an ellipse, a rosette, a spiral, or a hypotrochoid.

[0210] The system may be implemented such that modifying the trajectory includes mapping a trajectory template boundary to be on one side of an identified surface boundary in the surface topography, interpolating the mapped trajectory, and mapping a number of waypoints from the mapped trajectory.

[0211] The system may be implemented such that mapping includes fitting a spline to a number of waypoints to create a continuous path.

[0212] The system may be implemented such that the interpolation includes B-spline, quadratic, cubic, quintic, NURB, piecewise continuous spline, and polychain interpolation.

[0213] The system may also include a smoothing step.

[0214] The system may be implemented such that the smoothing step applies a Gaussian filter.

[0215] The system may be implemented such that a second smoothing is applied, the second smoothing comprising multiplying the mapped trajectory with an inverse bell curve.

[0216] The system may be implemented such that the identified surface boundaries are constraining features.

[0217] The system may be implemented such that the identified surface boundary includes surface features and buffer spaces from the surface features.

[0218] The system may be implemented such that approximating the surface includes an energy minimization method.

[0219] The system may be implemented such that the energy minimization method includes a potential field method or a snake interpolation.

[0220] The system may be implemented such that the method includes a dynamic energy minimization smoothing step, during which penalties are assigned for crossing boundaries of surface features.

[0221] The system may be implemented such that the approximation is a polynomial approximation, a spline surface, or a support vector machine.

[0222] The system may be implemented such that the polynomial approximation includes a polynomial patch.

[0223] A method for removing material from a workpiece surface is presented, the method including identifying a target area on the workpiece surface for material removal, sampling the surface around the target area on the workpiece surface, modeling the surface and detecting a surface topography based on the model, modifying a surface treatment trajectory using a converter based on the detected surface topography, the converted surface treatment trajectory comprising a continuous curve through a series of waypoints, and generating a repair plan at each of the waypoints comprising an applied force, a velocity, a rotational tool speed of a tool, and a tool angle relative to the workpiece surface. The method also includes sending control signals to a robotic material removal system, the control signals comprising the repair plan.

[0224] The method may be implemented such that the converter transforms the surface treatment trajectory by projecting the surface treatment trajectory onto a two-dimensional plane, mapping the surface treatment trajectory to a surface boundary in the surface topography, transforming the mapped surface treatment trajectory, and mapping the interpolated trajectory onto the surface topography of the workpiece surface to obtain a transformed surface treatment trajectory.

[0225] The method may also include smoothing the mapped surface treatment trajectory.

[0226] The method may be implemented such that the smoothing comprises applying a Gaussian filter.

[0227] The method may be implemented such that the interpolation comprises applying linear interpolation.

[0228] The method may be implemented such that the linear interpolation includes B-spline, quadratic, cubic, quintic, NURB, piecewise continuous spline, and polychain interpolation.

[0229] The method may be implemented such that modeling the surface includes approximating the surface at each of a plurality of sampled surface locations, approximating a curvature at each of the sampled surface locations, and detecting surface features based on a derivative of the approximated curvature.

[0230] The method may be implemented such that the surface approximation is a polynomial approximation, a linear regression approximation, or a least squares approximation.

[0231] The method may be implemented such that the target area includes a defect.

[0232] The method may be implemented such that the work surface includes a vehicle.

[0233] The method may be implemented such that sampling the surface includes a vision system imaging the surface.

[0234] The method may be implemented such that the vision system includes a camera.

[0235] The method may be implemented such that the surface treatment trajectory comprises a path of movement through the target area.

[0236] The method may be implemented such that the applied force at each of the waypoints is a modified applied force modified by a force modifier based on the surface topography.

[0237] The method may be implemented such that the velocity at each of the waypoints is a corrected velocity modified by a velocity modifier based on the surface topography.

[0238] The method may be implemented such that the rotary tool speed at each of the waypoints is a modified rotary tool speed modified by a rotary tool speed modifier based on the surface topography.

[0239] The method may be implemented such that the tool angle at each of the waypoints is a modified tool angle modified by a tool angle modifier based on the surface topography.

[0240] The method may be implemented such that modifying the surface treatment trajectory comprises an in-phase transformation of the trajectory template.

[0241] The method may be implemented such that the modified trajectory includes a discontinuous trajectory.

[0242] The method may be implemented such that the discontinuous trajectory skips over identified surface features.

[0243] The method may be implemented such that the transformation extends the trajectory over the identified surface features.

[0244] The method may be implemented such that the transformation moves the trajectory away from the identified surface feature.

[0245] The method may be implemented such that the trajectory has a perimeter having a shape, and the transformation changes that shape.

[0246] The method may be implemented such that a shape is stretched from a first shape to a second shape.

[0247] The method may be implemented such that a shape is compressed from a first shape to a second shape.

[0248] The method may be implemented such that a portion of the shape is compressed.

[0249] The method may be performed such that the compression avoids identified surface features.

[0250] The method may be implemented such that the compression avoids boundaries around identified surface features.

[0251] The method may be implemented such that a portion of the shape is stretched.

[0252] The method may be implemented such that the stretching extends across the identified surface features.

[0253] The method may be implemented such that the shape is a circle, an ellipse, a rosette, a spiral, or a hypotrochoid.

[0254] The method may be implemented such that the trajectory template is selected based on defect size, defect location, defect type, or defect severity.

[0255] The method may be implemented such that the trajectory is modified based on the identified surface features.

[0256] The method may be implemented such that the surface approximation is a polynomial approximation, a spline surface, or a support vector machine.

[0257] The method may be implemented such that the polynomial approximation includes a polynomial patch.

[0258] The method may be implemented such that the trajectory template has a perimeter and mapping the trajectory template includes mapping the perimeter to the identified surface boundary.

[0259] The method may be implemented such that approximating the surface comprises an energy minimization method.

[0260] The method may be implemented such that the energy minimization method includes a potential field method, a, or snake interpolation.

[0261] The method may be implemented such that it includes a dynamic energy minimization smoothing step during which a penalty is assigned for crossing boundaries of surface features.

[0262] The method may be implemented such that the tool is a sander or an abrasive tool.

[0263] A robotic surface processing system is presented that includes a surface inspection system that receives sampling information for a plurality of areas within a region of a workpiece surface to approximate a surface topography within the region. The system also includes a robot arm coupled to a surface engagement tool, the robot repair arm configured to contact the surface treatment tool with the region of the workpiece surface. The system also includes a process mapping system configured to identify surface-constraining features within the region and generate a surface treatment plan for the region based on the identified surface-constraining features, the surface treatment plan including a trajectory. The surface treatment plan includes one of a force profile along the trajectory, a velocity profile of the surface-engaging tool along the trajectory, a rotational velocity profile of the surface-engaging tool along the trajectory, and a trajectory modification that accounts for the surface-constraining features. The system is configured to generate a control signal for the robot arm that includes the surface treatment plan.

[0264] The system may be implemented such that the surface treatment plan includes an angle of attack profile of the surface engaging tool along the trajectory.

[0265] The system may be implemented such that the surface treatment plan includes trajectory modifications that take into account the surface constraining features, the trajectory modifications including an isomorphic transformation of the template trajectory.

[0266] The system may be implemented such that the surface treatment plan includes trajectory modifications that take into account the surface constraining features. The trajectory modifications include discontinuous trajectories.

[0267] The system may be implemented such that the discontinuous trajectory jumps over surface constraining features.

[0268] The system may be implemented such that the transformation stretches the trajectory over the surface constraining feature.

[0269] The system may be implemented such that the transformation moves the trajectory away from the surface constraining feature.

[0270] The system may be implemented such that the template trajectory has a perimeter with a shape, and the transformation changes that shape.

[0271] The system may be implemented such that the shape is stretched from a first shape to a second shape.

[0272] The system may be implemented such that a shape is compressed from a first shape to a second shape.

[0273] The system may be implemented such that a portion of the shape is compressed.

[0274] The system may be implemented such that the compression avoids identified surface features.

[0275] The system may be implemented such that the compression avoids boundaries around identified surface features.

[0276] The system may be implemented such that a portion of the shape is stretched.

[0277] The system may be implemented such that the stretching extends across the identified surface features.

[0278] The system may be implemented such that the work surface is a vehicle.

[0279] The system may be implemented such that the region includes a defect and the surface engaging tool is a material removal tool.

[0280] The system may be implemented such that the surface treatment tool is a sander or an abrasive tool.

[0281] The system may be implemented such that the surface inspection system includes a vision system.

[0282] The system may be implemented such that the vision system includes a camera.

[0283] The system may be implemented such that the process mapping system identifies surface constraining features by approximating each of the sampling areas within the region, approximating a curvature at each of the sampling areas within the region, calculating a derivative of the approximated curvature at each of the sampling areas, and identifying surface constraining features within the region based on the derivative calculation.

[0284] The system may be implemented such that a trajectory includes a series of waypoints through a region.

[0285] The system may be implemented such that the template trajectory is selected based on defect size, defect location, defect type, or defect severity.

[0286] The system may be implemented such that the trajectory is modified based on the identified surface features.

[0287] The system may be implemented such that the track is stretched or compressed.

[0288] The system may be implemented such that the shape is a circle, an ellipse, a rosette, a spiral, or a hypotrochoid.

[0289] The system may be implemented such that a robotic arm executes the control signals and follows a path.

[0290] The system may be implemented such that the approximation of the sampling area is a polynomial approximation.

[0291] The system may be implemented such that the polynomial approximation is a third order polynomial.

[0292] The system may be implemented such that the surface treatment plan includes trajectory modifications that take into account surface constraint features, the trajectory modifications including mapping a trajectory template boundary to lie on one side of an identified surface boundary in the surface topography, interpolating the mapped trajectory, and mapping a number of waypoints from the mapped trajectory.

[0293] The system may be implemented such that mapping includes fitting a spline to a number of waypoints to create a continuous path.

[0294] The system may be implemented such that the interpolation includes B-spline, quadratic, cubic, quintic, NURB, piecewise continuous spline, and polychain interpolation.

[0295] The system also includes a smoothing step.

[0296] The system may be implemented such that the smoothing step applies a Gaussian filter.

[0297] The system may be implemented such that a second smoothing is applied, the second smoothing comprising multiplying the mapped trajectory with an inverse bell curve.

[0298] The system may be implemented such that the identified surface boundaries are surface constraining features.

[0299] The system may be implemented such that the identified surface boundary includes surface features and buffer spaces from the surface features.

[0300] The system may be implemented such that the trajectory template has a perimeter, and mapping the trajectory template includes mapping the perimeter to the identified surface boundary.

[0301] The system may be implemented such that identifying the surface constraining features includes an energy minimization method.

[0302] The system may be implemented such that the energy minimization method includes a potential field method, a, or snake interpolation.

[0303] The system may be implemented such that the method includes a dynamic energy minimization smoothing step, during which penalties are assigned for crossing boundaries of surface features. EXAMPLES

[0304] Example 1 16A-16B illustrate an example adjustment of a trajectory 1600. After the repair is transformed into a 2D plane, the forces applied at each waypoint are adjusted, as shown in FIG.

[0305] For example, the force applied at or near a waypoint may be scaled proportionally depending on how close the waypoint is to a detected feature line 1610. In this example, a nominal force value is selected for each waypoint during the repair (which may be part of a selected repair template). The distance from each waypoint to the nearest feature line is measured. Feature points that are within a certain distance (indicated by the scale boundary in Figure X) are scaled proportionally. This scaling can be performed according to a simple ramp function 1650 as shown in Figure 16B.

[0306] As described, once the forces at each waypoint have been assigned and appropriately corrected, they are interpolated to provide a continuous signal for the force tool to track during the repair. This interpolation may be done by linear interpolation, spline interpolation, or other methods.

[0307] Example 1 shows the process of modifying the forces applied at each waypoint. This process may be repeated for other trajectory parameters including velocity, angle of attack, tool rotation rate, etc.

Claims

1. A robot system comprising: a surface inspection system that receives sampling information for a plurality of areas within a region of a work surface; a robot arm coupled to a surface engagement tool, the robot repair arm configured to engage the surface treatment tool with the region of the work surface; a process mapping system, wherein the process mapping system, based on the sampling information, approximates the surface topography of the region of the work surface, and generates a surface treatment plan for the region based on the approximated surface topography including a trajectory, a force profile along the trajectory, a speed profile of the surface engagement tool along the trajectory, a rotational speed profile of the surface engagement tool along the trajectory, and one of trajectory corrections considering the presence of surface features identified in the approximated surface topography, and is configured to generate a control signal for the robot arm including the surface treatment plan.

2. The system according to claim 1, wherein the surface treatment plan includes a trajectory correction considering the presence of surface features identified in the approximated surface topography, and the surface features include concave surfaces, convex surfaces, or edges within the region.

3. The system according to claim 1, wherein the surface treatment plan includes a trajectory correction considering the presence of surface features identified in the approximated surface topography, and the trajectory correction includes a discontinuous trajectory.

4. The system according to claim 3, wherein the transformation moves the trajectory away from the identified surface features.

5. The system according to claim 1, wherein the trajectory includes a series of waypoints passing through the region.

6. The system according to claim 4, wherein the template of the trajectory is selected based on defect size, defect position, defect type, or defect severity.

7. The system according to claim 6, wherein the surface treatment plan includes a trajectory correction considering the presence of surface features identified in the approximated surface topography, and the trajectory correction is based on the identified surface features.

8. The system according to claim 1, wherein the robot arm executes the control signal and follows the trajectory.

Images