Robot laser ablation system that generates surface-fitting motion paths via a program.

Abstract

We provide a laser ablation system equipped with a robot and processing circuitry. [Solution] The robot has an end effector having a laser component and a camera. The circuit receives an input image of a workpiece from the camera in response to a memory command, segments the image to generate image segments that depict the workpiece, applies a surface fitting algorithm to generate a polynomial surface that fits the inner surface of the segments, calculates a set of surface normal vectors crossing the surface to generate a set of waypoints offset from the polynomial surface, programs a motion path connecting the multiple waypoints, and performs operations including laser ablation of the workpiece surface based on the motion path.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 709,238, filed on October 18, 2024, the entire content of which is incorporated herein by reference for all purposes.

[0002] The present disclosure generally relates to robotic systems and methods for laser ablation. In particular, the present disclosure relates to automated systems and methods for generating surface - conforming motion paths for performing laser ablation of workpieces.

Background Art

[0003] In various industrial environments, including the manufacture, maintenance, and repair of aircraft, coatings such as paint are removed from surfaces. Existing techniques such as chemical stripping, plastic media blasting, and sanding each have associated drawbacks. Solvents used to remove paint often have harmful environmental impacts, and thus, government agencies regulate the use of these techniques and the disposal of the hazardous waste they generate. Plastic media blasting generates a hazardous waste stream in the form of plastic media and removed coatings while avoiding the use of such solvents. Additionally, sanding with handheld tools places an ergonomic burden on the operator and can generate particulates that require protective gear to protect the operator. Sanding can also result in over - removal of material, especially when hand tools are used.

[0004] One recently considered method for removing coatings is laser ablation. Laser ablation can be performed in a containment environment equipped with filters and scrubbers that can effectively capture the waste flow. Furthermore, laser ablation allows for precise fine-tuning of how much material is removed, avoiding over- or under-removal of material. One technical challenge associated with laser ablation of manufactured parts, present both during manufacturing and in the downstream stages of the repair and maintenance cycle, is that the precise shape of the workpiece changes from the ideal shape of the workpiece (i.e., the shape specified by the original CAD model, for example) due to tolerances in manufacturing and wear of the part in use.

[0005] Proper control of the laser during laser ablation requires knowing the precise distance of the laser to the part; otherwise, too much or too little material may be removed. Currently, laser ablation requires (1) manual laser path generation using part CAD, and (2) manual programming of a robot to follow an ablation path corresponding to the dimensions of the scanned part. This process is extremely time-consuming and requires a large support infrastructure, such as robot programmers and laser engineers, to implement it on a large scale, and it cannot account for the differences between the actual part "as is" and the CAD design on which the part was originally based. There is an opportunity to improve such laser ablation processes to enable the large-scale adoption of these techniques. [Overview of the project] [Means for solving the problem]

[0006] Taking the above into consideration, a robotic laser ablation system is provided, comprising a robot having an end effector that moves with multiple degrees of freedom, a laser component attached to the end effector, a camera configured to capture light and depth information in an input image of a workpiece, a processing circuit, and associated memory for storing instructions for the processing circuit. When a stored instruction is executed, the instruction causes the processing circuit to receive an input image, segment the input image to generate image segments that depict at least a portion of the workpiece, apply a surface fitting algorithm to generate a polynomial surface that fits the surface in the image segment, calculate a set of surface normal vectors that cross the polynomial surface, generate a set of waypoints offset by an offset length from the polynomial surface, programmatically generate a motion path connecting the multiple waypoints, and move the robot's end effector based on the motion path, thereby causing the laser component to perform laser ablation on the surface of a portion of the workpiece.

[0007] This robotic laser ablation system offers potentially beneficial technical effects by programmatically generating motion paths based on scans of actual workpieces, whose shape may vary compared to the CAD design due to manufacturing variations or wear during use, and provides precise control of the offset distance from the laser to the actual workpiece surface during laser ablation to avoid over- or under-ablation, with the additional advantage of shorter cycle times compared to fully manual path preparation.

[0008] This summary is provided to introduce a simplified conceptual option, which will be further described in the detailed description below. This summary is not intended to identify any significant or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to any implementation that resolves any or all of the defects described in any part of this disclosure. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows a robotic laser ablation system according to one implementation of the present disclosure. [Figure 2] This is a flowchart showing the process flow of a laser ablation method according to one implementation of the present disclosure. [Figure 3] This figure shows a pictorial process flow example of the laser ablation method shown in Figure 2, implemented by the robotic laser ablation system shown in Figure 1. [Figure 4] This is a detailed view of a robot-operated laser ablating a workpiece while traveling along a motion path constructed by the robotic laser ablation system in Figure 1, according to the laser ablation method in Figure 2. [Figure 5] Figure 1 is a schematic diagram of an example of a computing system that may be used in the robotic laser ablation system. [Modes for carrying out the invention]

[0010] Figure 1 shows a robotic laser ablation system 100, which is an example of a system suitable for implementing the technology described herein. As shown in the figure, the robotic laser ablation system 100 includes a robot 102 that processes a workpiece 120. The robot 102 and the workpiece 120 may, in one example, be housed in an isolation chamber which consists of a filtering system for collecting airborne dust and debris. The robot includes a base 102A, an actuator 103 coupled to the base 102A, an end effector 104 attached to the end of the actuator 103, and a laser component 106 and a camera 108 attached to the end effector 104. The laser component 106 may include a laser light source 106A and laser guide optical elements 106B such as a scanner, mirror, lens, waveguide, etc. In some embodiments, it will be understood that the laser light source 106A and the laser guide optical elements 106B of the laser component 106 are together attached to the end effector 104. In other embodiments, the laser light source 106A of the laser component 106 may be mounted outside the robot, and the end effector may hold the laser guide optical element 106B of the laser component 106, which redirects the laser from the laser light source to a target area on the workpiece 120. The actuator 103 can be essentially mechanical, hydraulic, pneumatic, or electric, and may be configured to move the end effector 104 with multiple degrees of freedom. The associated computing device 110 may be communicatively coupled to the robot 102 via a wired or wireless connection 111, which may include a computer network.

[0011] The computing device 110 controls and directs the operation of the robot 102, which includes an end effector 104, a laser component 106, and a camera 108. The computing device 110 includes a processing circuit 112 and associated memory 114 for storing instructions for the processing circuit 112. Instructions can be executed to implement a laser ablation program 113, which includes a camera module 113A, a segmentation module 113B, a surface fitting module 113C, a path generation module 113D, and a driver module 113E, the functions of each of these modules will be described later. The laser ablation program 113 of the computing device 110 is configured to display a graphical user interface (GUI) 116 on a display 110A associated with the computing device 110. As shown in the figure, the GUI 116 displays an example image 118 taken by the camera 108, along with a control unit 116A for providing user input to the laser ablation program 113.

[0012] As shown in the figure, the robot 102 has an actuator 103 in the form of a repositionable arm to which the end effector 104 is attached at its distal end. Thus, the robot 102 of the robotic laser ablation system 100 is configured to move the end effector 104 with multiple degrees of freedom (DOF). The DOF of the end effector 104 refers to the independent ways in which the end effector can move in three-dimensional (3D) space.

[0013] In some implementations, the end effector 104 may have three DOFs for movement in position space (e.g., x (left / right), y (up / down), and z (forward / backward)). In other implementations, the end effector 104 may have three additional DOFs for changes in orientation (e.g., pitch, yaw, and roll). In other implementations, there may be multiple DOFs of different numbers to accommodate movement and / or changes in orientation in position space (e.g., positioning the robot on a straight rail).

[0014] The laser component 106 and the camera 108 can both be mounted on the end effector 104 and positioned to face the side surface 122 of the workpiece 120. The path of the concentrated beam 124 from the laser component 106 during ablation of the surface of the workpiece 120 is shown by a dashed line. The field of view 126 of the camera 108 is shown by a dotted line, along with the camera axis 130 that forms the center of the field of view of the camera 108. A predetermined three-dimensional offset between the camera axis 130 and the laser beam 124 is saved as a setting in the laser ablation program 113 and converts data between camera coordinates and laser coordinates as part of the laser ablation program 113. In addition or alternatively, the camera 108A can be mounted at a location other than on the end effector 104, such as the overhead mounting location shown in the figure, in an orientation that has a field of view including the workpiece 120. In addition to or instead of the cameras 108 and 108A, a displacement sensor 109 can be provided on the end effector 104 or at another mounting location that has a field of view of the workpiece. The displacement sensor 109 may be a laser displacement sensor or the like. The displacement sensor 109 is configured to perform a high-resolution depth scan of the surface 122 of the workpiece 120. In addition to, or instead of, the cameras 108, 108A and the displacement sensor 109, an advanced topometric optical scanner (ATOS) can be used.

[0015] The workpiece 120 can be a part, component, mechanism, hardware, section, member, or element that can be subjected to laser ablation. The side surface 122 of the workpiece 120 is subject to image acquisition by the camera 108 and ultimately to laser ablation by the laser component 106. More specifically, the side surface 122 of the workpiece 120 includes one or more surfaces. Although schematically shown as a cube, it will be understood that in practice the workpiece 120 can have a complex shape. Thus, each surface of the workpiece 120 can be a two-dimensional surface of a three-dimensional workpiece, and can be a flat or curved two-dimensional surface. Examples of aircraft parts that can be used as workpiece 120 include small parts such as doors, hinges, and fasteners, and larger parts such as fuselage sections and landing gear assemblies. Thus, as can be understood, the surface of the workpiece 120 can be a flat surface, a curved surface, or a complex shape having both flat and curved parts. In some implementations, the robot 102 is configured to position the laser component 106 to ablate from multiple sides 122 of the workpiece 120 by multiple degrees of freedom within the actuator 103 and / or via a rotating turntable on which the workpiece 120 is mounted, for example.

[0016] The laser component 106 is configured to perform laser ablation on the surface facing the side 122 of the workpiece 120. Laser ablation is a versatile technique used to precisely remove material from a surface. For example, laser ablation can be used to achieve, in some cases, recoating protective surface coatings such as paint, removing oxide layers, and removing material for cleaning, cutting, drilling, and engraving.

[0017] Camera 108 is configured to capture an input image 118 of the workpiece 120. The input image 118 contains color information and depth information for each of the multiple pixels in the input image 118. Camera 108 can be, for example, a 3D depth camera configured to capture both color information and depth information. The depth camera may emit invisible light with a structured pattern and determine the depth by measuring the pattern, or it may determine the depth using time-of-flight or other three-dimensional techniques. Furthermore, the depth camera can utilize LiDAR (Light Detection and Ranging) technology, which measures depth by emitting laser pulses and calculating the time it takes for the reflected light to return. A CMOS, CCD, or other sensor may be used to collect color information for each pixel of the captured image. Depth information can be collected on a pixel-by-pixel basis and used to create a three-dimensional map of the surface of the workpiece 120. Color information, along with depth information, can be used for image processing, such as segmenting the input image 118 to characterize the region of interest, including the workpiece, as described later.

[0018] When an instruction stored in memory 114 is executed by processing circuit 112, the instruction causes processing circuit 112 to implement the laser ablation program 113 using the program's constituent modules, which perform the operation as follows: Camera module 113A is configured to receive an input image 118, such as image 118, from camera 108. Camera module 113A can be configured to perform initial image processing on the input image 118, such as adjusting contrast, brightness, cropping, or other image processing techniques to facilitate downstream processing. Segmentation module 113B is configured to segment the input image 118 to generate image segments that depict at least a portion of the workpiece 120. It will be understood that an image segment is typically a smaller portion of the input image (e.g., a region of interest), and that in order to perform image segmentation, the segmentation module 113B is configured to identify a portion of the workpiece 120 in the input image 118 and define the boundary of the workpiece 120 in the input image 118 by, for example, defining a boundary around it that will be used to generate an image segment. Appropriate techniques for doing so will be described in detail below with reference to workpiece 320 in Figure 3, which is an example of workpiece 120 in Figure 1.

[0019] The surface fitting module 113C is configured to receive an image segment from the segmentation module 113B and apply a surface fitting algorithm to generate a polynomial surface that fits the surface within the image segment. It will be understood that the camera 108 may be the depth camera described above, configured to capture color and depth information on a pixel-by-pixel basis. Thus, the image segment includes a set of pixels defined not only by the RGB color values ​​and xy positions of the pixels in the input image pixel array, but also by the depth value of each pixel. From this x, y, and depth information, discrete parameterized values ​​of the surface of the workpiece 120 can be determined. The surface fitting module 113C is configured to generate a continuous polynomial surface that approximates the parameterized surface, as will be described in more detail below.

[0020] The path generation module 113D receives a polynomial surface from the surface fitting module 113C, calculates a set of surface normal vectors that cross the polynomial surface, and is configured to generate a set of waypoints (see 142 in Figure 4) that are offset from the polynomial surface by an offset length in the z direction of Figure 4, for example, at the xy locations in Figure 4. The path generation module 113D is further configured to programmatically generate a motion path (see 146 in Figure 4) that connects multiple waypoints. In this way, the motion path 146 is generated in a three-dimensional space that maintains a constant distance from the actual surface of the workpiece 120, ensuring that the laser beam is precisely focused on the part surface to more effectively control ablation, thereby suppressing excessive or insufficient ablation.

[0021] The driver module 113E receives the motion path 146 and outputs control commands to the robot 102 and the laser component 106, thereby configuring the laser component 106 to perform laser ablation on a part of the surface of the workpiece 120 based on the motion path 146. This has a dual technical effect of automating the process of generating the motion path for the laser and saving time and computational resources for manual path generation. Also, by basing the motion path on the actual shape of the scanned workpiece rather than the idealized CAD representation of the workpiece, it helps reduce the error in the distance between the laser component 106 and the ablated surface of the workpiece 120, thereby reducing cases of excessive or insufficient ablation of the material. This process will be further described below with reference to FIGS. 2 to 4.

[0022] FIG. 2 shows a flowchart of a computerized method example 200 according to an implementation example of the present disclosure. FIG. 3 shows a pictorial process flow example 300 of the computerized method 200 for ease of understanding. The laser ablation system 100 of FIG. 1 is configured to implement the method 200 shown diagrammatically. Therefore, the method 200 and the pictorial process flow 300 are described together as being implemented by the laser ablation system 100. However, it will be understood that the method 200 can be implemented with other suitable hardware and software.

[0023] In step 202, the method 200 includes the step of providing a workpiece such as the above-described workpiece 120. The pictorial representation 302 shows a workpiece 320 similar to the above-described workpiece 120. More specifically, the system 100 can be configured to provide the workpiece 320 such that the side surface 322 faces the camera and the laser component. The side surface 322 includes one or more surfaces that will be imaged by the camera 108 and ablated by the laser component 106.

[0024] System 100 can achieve this either actively or passively. Actively, the system can pick and place one or more workpieces 320 using robot 102. Passively, the system can achieve this by providing an appropriate location for an operator to manually place one or more workpieces 320 in an appropriate position and orientation for the robot 102 of the robotic laser ablation system 100 to operate effectively.

[0025] In step 204, method 200 includes receiving an input image, such as input image example 118 shown in FIG. 1. This step can be achieved by camera module 113A of system 100 as described above. Pictorial representation 304 of FIG. 3 shows the captured input image 324 of workpiece 320. As described above, a depth camera can be used as camera 108 for capturing the input image. Also, as described above, the input image includes both color information and depth information for each of a plurality of pixels within the input image.

[0026] In step 205, method 200 includes the step of segmenting the input image to generate segmented images of at least a portion of the workpiece. This step can be performed by the segmentation module 113B described above. Segmenting the input image in 205 may include identifying a seed point in the input image in step 206 and implementing a segmentation algorithm based on the seed point in 208, as will be further described below. The pictorial representation 306 in Figure 3 includes an example of a segmented version of the input image 324 of the workpiece 320. The seed point 328 is selected such that the seed point is located within a segment 326 of the input image 324 containing the workpiece 320. More specifically, segment 326 is a region of pixels in the input image 324 that depicts a portion of the surface of the workpiece 320. Thus, using the segmentation algorithm (described later), the seed point 328 effectively identifies a region of pixels in the input image that depicts a portion of the surface of the workpiece 320.

[0027] Seed point identification can be achieved in various ways. For example, seed point identification can be achieved at least partially by obtaining user selection of seed points via a GUI. By providing such manual selection, users (such as experts) can directly select points within the region of interest. Even with the use of such expert input, this technique offers time savings that outweigh conventional methods that required the manual generation of each complete motion path. One way in which manual selection can be performed is by obtaining user selection of seed points via a GUI. As shown in Figure 1, the computing device 110 uses a GUI 116 that displays an example image 118 captured by the camera 108. GUI 116 is an example of a GUI that can be used to receive manual selection of seed points from a user. In one example, mouse clicks may be used for selection. In another example, a virtual or physical four-way pad may be provided to move a cursor over the location of the desired seed point for selection. In yet another example, candidate seed points may be identified by a computer vision algorithm, and GUI 116 can provide affordances for switching between candidates to select the appropriate seed point.

[0028] As an alternative to the manual selection described above, the identification of seed points can be at least partially achieved by programmatically determining seed points based on criteria (e.g., thresholds). Examples of such programmatic techniques that may be used to select seed points for image segmentation include the shortest distance from the point to the camera, foreground / background separation, intensity-based methods, gradient-based methods, feature-based methods, and machine learning-based methods.

[0029] The shortest distance method from a point to the camera uses depth information to identify the point closest to the camera in the image as a seed, which is particularly effective in scenes where foreground objects are clearly closer. Another method requires separating the foreground from the background and then calculating the centroid of the foreground region to use as a seed point. Intensity-based methods such as histogram analysis and adaptive thresholding can also be used, as they utilize variations in pixel intensity to identify potential seed regions. Gradient-based methods can identify seed points by identifying edges or local changes in pixel values.

[0030] Feature-based methods identify unique local features as seed points using techniques such as SIFT / SURF (Scale Invariant Feature Transform / Speeded Up Robust Feature) keypoint detection or corner detection. Machine learning techniques can also be used, such as suggesting seed points or seed regions using trained neural network classifiers for semantic segmentation. For video processing, motion-based methods such as optical flow or background subtraction can effectively identify moving objects as potential seed points. Splendor detection techniques, which aim to identify areas that stand out more than their surroundings based on visual attention models, can also be used to identify seed points.

[0031] In step 208, the method includes implementing a segmentation algorithm using seed points to identify regions of pixels in an input image that depict a portion of the surface of a workpiece. It will be understood that each pixel in the region has a corresponding two-dimensional (2D) coordinate in the input image and a corresponding depth coordinate from the camera to the surface. Pictorial representation 308 depicts the result of the segmentation algorithm and shows only segment 326 of image 324 that has been segmented from the input image 324. Segment 326 is a region of pixels in the input image 324 that depicts a portion of the surface of interest of the workpiece 320.

[0032] In this specification, surfaces of interest can vary greatly in number and size. For example, there may be a single surface of interest (e.g., one fuselage section) or multiple surfaces of interest (multiple small parts on a tray). Surfaces of interest can be small (e.g., fasteners) or very large (e.g., landing gear assemblies). For even smaller parts, cycle time can be reduced by imaging and ablating multiple parts in a single ablation session.

[0033] The segmentation algorithms described herein refer to processes used by the segmentation module 113B to divide an input image into two or more segments or parts such that at least one of the segments depicts a workpiece 320. The segmentation algorithm assigns labels to pixels that share specific characteristics indicating a target segment (e.g., a workpiece or background). Examples of segmentation algorithms that can be used include thresholding methods, edge detection techniques, region-based segmentation, clustering-based segmentation, and neural network-based methods.

[0034] An example of a thresholding segmentation algorithm is shown below. • Load the input image • If the image is in color, convert it to grayscale. • Select a threshold (for example, 128 for a medium gray area). • Iterate through each pixel in the image: If the pixel value > threshold, set to white (255). If the pixel value is less than or equal to the threshold, set to black (0). • Output the resulting binary image.

[0035] This example segmentation algorithm segments an image into two regions: one region contains all pixels brighter than a threshold (set to white), and the other region contains all pixels darker than or equal to the threshold (set to black).

[0036] Another example of a segmentation algorithm uses K-means clustering. This method segments an image by grouping pixels with similar color characteristics. An example of the K-means clustering algorithm is as follows: • Load the input image • Convert the image to the desired color space. • Reshape the image into a list of pixels. Apply K-means clustering to cluster pixels into K color space clusters. • Assign each pixel to its corresponding cluster. • Create an image segmented into K image segments.

[0037] The algorithm described above segments an image into a specified number (K) of clusters based on color similarity. The K-means algorithm can, for example, segment an input image according to its pixel values, thereby generating a segmented image in which each segment represents the average color of its cluster.

[0038] Several implementations for segmenting the input image used by steps 206 and / or 208 can use image segmentation models. Examples of such suitable segmentation models include semantic segmentation models, instance segmentation models, panoptic segmentation models, edge detection models, region-based segmentation models, clustering-based segmentation models, attention-based segmentation models, real-time segmentation models, nearest neighbor segmentation models, segment-anything models (SAM), and others, as described below. • Semantic segmentation models: These models assign a class label to every pixel in an image. Examples include U-Net, FCN (Fully Convolutional Networks), and DeepLab. • Instance segmentation models: These models identify and segment individual object instances. Masked R-CNN is a notable example. • Panoptic segmentation models: Combining semantic segmentation and instance segmentation, these models assign both a class ID and an instance ID to each pixel. DETR with a panoptic head is one example. • Edge detection models: These models focus on finding boundaries between regions, such as Canny edge detection or HED (Holistically-Nested Edge Detection). • Domain-based segmentation models: These models extend the domain from a seed point, such as traditional domain augmentation algorithms or deep learning versions like DeepIGeoS. • Clustering-based segmentation models: These models group similar pixels and include K-means clustering and deep clustering techniques. • Attention-based segmentation models: These models use attention mechanisms, such as SETR (Segmentation Transformer), to focus on relevant parts of the image. • Real-time segmentation models: Optimized for speed, these models include ENet and SwiftNet. • Nearest Neighbor Segmentation Models: These models classify pixels based on the labels of their nearest neighbors in the feature space. While traditionally used with hand-drawn features, these models can also be applied to deep feature representations. One example is the K-Nearest Neighbors (KNN) algorithm adapted for segmentation tasks. • Segment Anything Model (SAM): Developed by Meta AI, this promptable segmentation model is designed to segment any object in an image based on various types of prompts (points, boxes, or text). This model is flexible and can often generalize to new objects and images without additional training.

[0039] In step 210, method 200 includes the step of applying a surface fitting algorithm to generate a polynomial surface (e.g., a cubic polynomial) that fits the surface in the segmented image. This step can be implemented, for example, by the surface fitting module 113C described above. The pictorial representation 310 in Figure 3 shows a depiction of the polynomial surface 330 that fits the surface of the segmented image. The cubic polynomial is fitted to the surface as an approximation of the surface in the segmented image. This process addresses noise in the scan and achieves a smooth estimation of the surface. Alternatively, since the technique is not limited to cubic polynomials, a polynomial of a different degree may be selected.

[0040] The surface fitting algorithm uses mathematical techniques to approximate a continuous polynomial surface from discrete data points in a multidimensional space, i.e., from x, y, and z (depth) values ​​in the input image received from camera 108. The surface fitting algorithm aims to find the polynomial function that best fits the parameterized image data within an acceptable margin of error. This process typically requires determining the coefficients of the polynomial. The degree of the polynomial can be chosen based on a trade-off between fitting accuracy and computational complexity. Low-degree polynomials, such as bilinear and biquadratic functions, offer simplicity and speed but may miss details, while higher-degree polynomials, such as bicubic or higher-degree functions, can capture more complex surface features at the cost of increased computational demands and potential overfitting.

[0041] The fitting process can utilize least-squares optimization, which minimizes the sum of squared differences between the polynomial surface and the actual data points. This technique can be extended to weighted least-squares to account for the reliability or importance of the data, if desired. Surface fitting algorithms can also incorporate techniques for handling outliers or noise in the data to ensure a smooth and representative surface. Several implementations can use adaptive-order selection or piecewise polynomial fitting to optimize the balance between accuracy and efficiency across different regions of the data space. The resulting polynomial surface provides a continuous mathematical representation of the original discrete data within segment 326 of the input image 324, enabling interpolation, analysis, and visualization of underlying patterns and trends in the dataset.

[0042] An example of the steps to implement a basic polynomial surface fitting algorithm is as follows: • Collect input data points (x, y, z coordinates from the input image). • Select the degree of the polynomial (for example, 2 for biquadratic polynomials). • Construct the design matrix using the input coordinates. • Formulate the normal equations • Solve simultaneous equations to find the polynomial coefficients. • Construct a suitable surface function using coefficients.

[0043] This example demonstrates fitting biquadratic or other degree polynomial surfaces to discrete data. The algorithm constructs a design matrix, calculates the coefficients (e.g., using least squares), and returns both the fitted surface function and its coefficients.

[0044] Another example of a polynomial surface fitting algorithm uses radial basis functions (RBFs) for surface fitting. RBF interpolation is particularly useful for scattered data points and can handle more complex surface geometries. The steps of such an algorithm are outlined below. • Collect input data points (x, y, z coordinates) Select a radial basis function (e.g., Gaussian basis function). • Construct an interpolation matrix. • Solve for the RBF coefficient • Create a suitable surface function

[0045] The RBF method creates a smooth surface that passes through all data points and can accommodate irregularly spaced data points to represent complex surface shapes. Various RBF types are available, including multi-order, Gaussian, linear, and cubic RBF.

[0046] As shown in step 212 in Figure 2, method 200 includes the step of calculating a set of surface normal vectors crossing a polynomial surface in order to generate a set of waypoints offset by a calculated offset length of the surface normal vector from the polynomial surface. The calculated offset length can be uniform in one implementation configuration. When uniform energy is applied to the laser from a uniform offset distance, a uniform ablation depth can be achieved, assuming that the workpiece material is uniform. In other implementation configurations, the calculated offset length can vary depending on a subset of waypoints. Various offset distances can be used to control the depth of laser ablation. One potential application example is as follows: A workpiece may have a thicker coating in one part than in another. In the thicker part, a larger removal depth can be achieved by calculating a smaller offset and therefore ablating from a closer location. In the thinner part, a larger offset can be calculated, resulting in a smaller ablation depth. In addition to adjusting the motion path, or as an alternative to adjusting the motion path for fine-tuning, a z-adjustment mechanism can be added to the laser component to be used to adjust the focus of the laser beam itself, thereby slightly increasing or decreasing the ablation depth.

[0047] The path generation module 113D of system 100 in Figure 1 is configured to perform this step. The pictorial representation 312 in Figure 3 shows that the set of illustrated waypoints 142 331 is a uniform offset length from the polynomial surface 330 (i.e., the length of the surface normal vector). Figure 4 shows in more detail an example of a polynomial surface 330 having a surface normal vector 140 that extends from an origin 144 on the polynomial surface 330 by a uniform distance in the normal direction to the polynomial surface 330 and terminates at waypoints 142. The waypoints 142 may be evenly spaced or may be spaced according to a fluctuating density function. To calculate the position of the waypoints, the path generation module 113D calculates the normal vector from the origin 144 on the polynomial surface 330, thereby establishing the position of each waypoint 142. The origin 144 may be determined, for example, by superimposing or projecting a grid onto the polynomial surface.

[0048] As shown in step 214 in Figure 2, the method includes the step of defining a two-dimensional (2D) frame containing each subset of waypoints. An example of a 2D frame is shown in pictorial representation 314 and 148 in Figure 4. The 2D frame 148 is a plane and is used to align the waypoints to the plane. A pair of 2D frames 148 are shown in Figure 4, but it will be understood that a number of 2D frames 148 are defined over the entire polynomial surface 330. The 2D frames 148 are parallel to each other and separated by a calculated distance (which may be uniform or vary depending on the function), slicing the polynomial surface 330 at intervals. Waypoints 142 are fitted so that they lie within the 2D frames 148. By fitting waypoints 142 to the 2D frames 148, a motion path 146 with the minimum reciprocal wave motion due to the planar alignment of waypoints can be generated in the next step discussed below.

[0049] In step 216 of Figure 2, method 200 further includes the step of programmatically generating a motion path that connects waypoints. This step is performed by the path generation module 113D in Figure 1. An example motion path 146 is shown in Figure 4. As shown in Figure 4, a subpath 146A of motion path 146 connects a subset of waypoints 142 within each two-dimensional frame 148. System 100 programmatically generates motion path 146 in camera space. Processing circuit 112 can be configured to convert motion path 146 into a reference frame for robot 102. Robot 102 can then be guided to move its actuators 103 and end effector 104 to move the laser component 106 along motion path 146. The pictorial representation 316 in Figure 3 shows a robot example 102 moving a laser component 106 along a transformed motion path 146 within a robot reference frame, and Figure 4 provides an additional detail view of the robot 102 moving the laser component 106 along the motion path 146.

[0050] As shown in Figure 4, the motion path 146 can be a meandering raster path. A meandering raster path, also known as a snake or meandering pattern, is a systematic motion path. This path covers an area of ​​surface 326 of the workpiece 320 in a reciprocating pattern, starting from one corner and moving in parallel lines across surface 326, alternating directions. This technique facilitates efficient and uniform coverage of rectangular or square areas, minimizes unproductive movement, and enables continuous motion. The motion path 146 can be easily adapted to different dimensions and resolutions, and the predictable nature of the motion path simplifies programming and operator situational awareness. In processes such as laser ablation, this technique enables consistent processing across the entire target surface and promotes uniform quality of material removal from the target surface.

[0051] Other implementations can use patterns of other motion paths.146. For circular regions or concentrated energy concentrations, spiral paths can be used. To maintain consistent edge quality, concentric paths that trace the shape from the outside to the inside, or vice versa, are effective. Vector paths are suitable for complex shapes or selective ablation that trace a given contour. Cross-hatch patterns combined with vertical raster movement are useful for generating uniform ablation depth or textured surfaces. Radial paths that move along a line from a center point can be used on circular or spherical surfaces. Adaptive systems can enable accuracy on irregular surfaces by using paths that adjust in real time based on sensor feedback.

[0052] The overall motion path 146 can be divided into subpaths 146A as described above, with each subpath concatenating a subset of waypoints 142 within a two-dimensional frame 148, suggesting that the motion is planned in a two-dimensional slice of three-dimensional space. Initially, system 100 generates this motion path 146 in camera space, relative to the viewpoint of a camera observing the environment. In some implementations, system 100 translates this camera space path into a reference frame for robot 102. This translation enables robot 102 to understand and execute motion relative to the position and orientation of the laser component 106 mounted on robot 102's end effector 104, rather than the viewpoint of a camera. By translating the motion path 146 into a robot coordinate system, the motion path 146 can be adapted to the robot driver and other robot-specific software.

[0053] In step 218 of Figure 2, Method 200 includes the steps of moving a robot to move a laser component mounted on an end effector along a converted motion path, and performing laser ablation of a portion of the workpiece surface by intermittently or continuously exciting the laser as the laser travels along the converted motion path. The driver module 113E in Figure 1 can be configured to perform this step. Figure 4 shows a laser component 106 traveling along a motion path 146 and continuously emitting a concentrated beam 124 excited at the surface 326 of the workpiece 320 as it moves. In some implementations, the laser component 106 can be moved to a stationary position by pausing at each waypoint 142 to allow laser ablation to be performed while the robot is stationary relative to the part surface (also known as tiling ablation). In other implementations, the laser source 106A of the laser component 106 can be emitted while the robot 102 moves the effector 104 relative to the part surface (also known as continuous ablation). This method may be configured to use a "bowtie" algorithm to generate parallel raster lines, as will be described later. In all of these scenarios, the part can be moved relative to a fixed laser component (e.g., a conveyor belt).

[0054] The bowtie algorithm for laser ablation is a special motion path strategy designed to generate parallel raster lines despite the relative motion between the laser component's scanner and the part surface. A simpler technique for continuous motion processing is to simply move the X-axis scanner mirror back and forth as the laser component's scanner moves in the Y direction. This yields a classic zigzag pattern. The zigzag pattern is undesirable for many applications due to the degree of spot overlap resulting from columns that have correct spacing in the center but a large degree of overlap at the edges. This can create hot spots or ridges rather than consistent material removal. To correct this, the "bowtie" algorithm, named after the shape it creates when viewed from above, moves both the X and Y axes in a programmed pattern while the laser component's scanner moves continuously in the Y direction. The laser component's scanner mirror moves backward across the X-axis of the scan field at a backward angle to reflect the forward motion of the head, then "jumps" forward on the Y-axis and moves backward across the scan field in the opposite direction across the X-axis. If the timing is set correctly, the algorithm compensates for the forward movement of the head, resulting in perfectly parallel raster lines.

[0055] The pictorial representation 318 shows the workpiece 320 after a mosaic-like path of a pattern has been laser ablated over it. The laser ablation pattern shown in the ablation zone 332 of the pictorial representation 318 is square. Other implementations may be hexagonal, triangular, rectangular (e.g., other than square), or other shapes. The square ablation zone 332 represents such a laser ablation pattern of the path to be performed. A mosaic-like arrangement of hexagonal patterns is known to have less overlap and is therefore more efficient. The hexagonal pattern allows for more efficient use of the circular laser optics, increasing the total area that can be processed per location. This minimizes the number of movements performed by the robot 102 to have complete coverage. This also minimizes overlap to a maximum of three times, which may be beneficial for specific applications. It will be understood that the shape of the ablation zones can be changed zone by zone as needed.

[0056] Step 218 removes material from the surface 326 of the workpiece 320 using laser ablation. This method uses a robot 102 equipped with a laser component 106 mounted on the end effector 104 of the robot 102, which is part of the robot 102 that directly interacts with the workpiece 320. The movement of the robot 102 is guided by a pre-planned and converted motion path 146, as shown in Figure 4, to ensure the precise positioning of the laser component 106 over the target area. As the robot 102 moves along this meticulously calculated trajectory, the laser source 106A of the laser component 106 is either continuously activated or intermittently pulsed. This controlled excitation of the laser allows for precise material removal from the surface 326 of the workpiece 320. Intermittent activation can be used to create a specific pattern or to control the depth of ablation, while continuous excitation can be used for more uniform material removal or cutting. The converted motion path 146 can take into account the kinematics of the robot 102 and specific parameters of the ablation process, such as velocity and approach angle. This robotic laser ablation method combines elements of robotics, motion control, laser technology, and material processing to achieve highly accurate and reproducible surface modification or material removal.

[0057] In some implementations, step 218 includes an ablation quality control process. That is, during or after laser ablation, the workpiece 320 is inspected to determine whether its quality meets predetermined criteria, such as the amount of material removed. Such a quality control process may further include the camera 108 being configured to capture confirmation images of a portion of the workpiece 326 during and / or after the laser ablation is performed on that portion of the workpiece 320. The quality control process may further include the processing circuit 112 being configured to determine, based on the confirmation images, that the quality of the laser ablation on a portion of the workpiece 326 is insufficient (based on some predetermined criteria), and in response to that determination, to perform further laser ablation on that portion of the workpiece 320.

[0058] In some implementations, the surface being processed (e.g., surface 326 of workpiece 320 in Figure 3) may include areas known as exclusion zones, which should not undergo the ablation process. In some implementations, these zones can be identified by the user through manual selection using a GUI, such as the GUI 116 of the computing device 110. In other implementations, these zones can be automatically determined based on criteria determined by a program.

[0059] These exclusion zones can represent various features or critical areas on the workpiece 320. For example, an exclusion zone could be an existing hole or opening that serves a functional purpose and must remain intact. Alternatively, an exclusion zone could be a sensitive area such as a seal, a precision-machined surface, or an area with a specific coating that should be preserved. The motion path 146 for the laser component 106 is carefully designed to navigate around these exclusion zones to ensure that the laser emitted by the laser component 106 does not interact with these protected areas. If necessary, exclusion zones can be formally designated within a safety-rated motion control system (e.g., Kuka SafeOperate, ABB SafeMove) to ensure further protection by preventing the robot from entering the exclusion zone or by preventing the laser from firing on a designated surface. This level of control over the motion program helps maintain the integrity of the workpiece 320 while still achieving the desired ablation results in the surrounding areas.

[0060] In one or more implementations described herein, a single input image is captured that allows for an object surface that is flat or has a gentle curve. The gentle curve can be in one direction, two directions, or three directions. However, in other implementations, multiple images of the workpiece 320 can be captured, and such images can be stitched together to create a motion path 146 that travels along a surface with complex curvature or is too large to be captured in a single image.

[0061] In one or more implementations, multiple regions of a complex surface 326 can be imaged separately, and these images can be stitched together to create a motion path 146 on the complex surface 326 (e.g., a ball or landing gear).

[0062] Figure 5 schematically illustrates a non-limiting embodiment of the computing system 400 capable of performing one or more of the methods and processes described above. The computing system 400 is shown in a simplified form. The computing system 400 can embody other embodiments of the computing system described above. The computing system 400 includes a processing circuit 402, a volatile memory 404, and a non-volatile storage device 406. The computing system 400 may optionally include a display subsystem 408, an input subsystem 410, a communication subsystem 412, and / or other components not shown.

[0063] A processing circuit typically includes one or more processors, which are physical devices that execute instructions. For example, a processing circuit can be configured to execute instructions that are part of one or more applications, programs, routines, libraries, objects, components, data structures, or other logical structures. Such instructions can be implemented to perform tasks, implement data types, transform the state of one or more components, achieve technical effects, or otherwise reach desired results.

[0064] The processing circuit may include one or more physical processors that execute software instructions. Additionally or alternatively, the processing circuit may include one or more hardware logic circuits or firmware devices that execute hardware implementation logic or firmware instructions. The processors of the processing circuit 402 may be single-core or multi-core, and the instructions executed therein may be configured for sequential processing, parallel processing, and / or distributed processing. Individual components of the processing circuit may optionally be distributed across two or more separate devices, which may be located remotely and / or configured for cooperative processing. For example, embodiments of the computing systems disclosed herein may be virtualized and executed by remotely accessible network computing devices configured in a cloud computing configuration. In such cases, it will be understood that these virtualized embodiments run on different physical processing circuits on different machines. These different physical processing circuits on different machines will be understood to be collectively encompassed by the processing circuit 402.

[0065] The non-volatile memory device 406 includes one or more physical devices that hold instructions executable by a processing circuit to implement the methods and processes described herein. When such methods and processes are implemented, the state of the non-volatile memory device 406 can be transformed, for example, to hold different data.

[0066] The non-volatile storage device 406 may include a removable and / or built-in physical device. The non-volatile storage device 406 may include optical memory, semiconductor memory, and / or magnetic memory, or other mass storage technology. The non-volatile storage device 406 may include non-volatile devices, dynamic devices, static devices, read / write devices, read-only devices, sequential access devices, position-addressable devices, file-addressable devices, and / or content-addressable devices. It will be understood that the non-volatile storage device 406 is configured to retain instructions even when power to the non-volatile storage device 406 is cut off.

[0067] The volatile memory 404 may include a physical device containing random access memory. The volatile memory 404 is typically used by the processing circuit 402 to temporarily store information during the processing of software instructions. It will be understood that the volatile memory 404 typically does not continue to store instructions when power to the volatile memory 404 is cut off.

[0068] The processing circuit 402, the volatile memory 404, and the non-volatile storage device 406 can be integrated into one or more hardware logic components. Such hardware logic components may include, for example, field-programmable gate arrays (FPGAs), programmable integrated circuits (PASICs) and application-specific integrated circuits (ASICs), programmable standard products (PSSPs) and application-specific standard products (ASSPs), systems on a chip (SOC), and complex programmable logic devices (CPLDs).

[0069] The terms “module,” “program,” and “engine” can be used to describe a mode of computing system 400 that is typically implemented in software by a processor to perform a specific function using a portion of volatile memory, the function including a transformation process that specifically configures the processor to perform that function. Thus, a module, program, or engine can be instantiated via a processing circuit 402 that executes instructions held by a non-volatile storage device 406 using a portion of volatile memory 404. It will be understood that different modules, programs, and / or engines can be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Similarly, the same module, program, and / or engine can be instantiated by different applications, services, code block, object, routine, API, function, etc. The terms “module,” “program,” and “engine” can encompass individuals or groups such as executable files, data files, libraries, drivers, scripts, database records, etc.

[0070] If included, the display subsystem 408 can be used to present a visual representation of the data held by the non-volatile memory 406. The visual representation may take the form of a graphical user interface (GUI). Since the methods and processes described herein modify the data held by the non-volatile memory and thus transform the state of the non-volatile memory, the state of the display subsystem 408 can also be transformed to visually represent the changes in the underlying data. The display subsystem 408 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with the processing circuitry 402, volatile memory 404, and / or non-volatile memory 406 in a shared enclosure, or such display devices may be peripheral display devices.

[0071] If included, the input subsystem 410 may include or interface with one or more user input devices such as a keyboard, mouse, touchscreen, camera, or microphone.

[0072] If included, the communication subsystem 412 may be configured to connect the various computing devices described herein to each other and to other devices in a communicative manner. The communication subsystem 412 may include wired and / or wireless communication devices compliant with one or more different communication protocols. In non-limiting examples, the communication subsystem may be configured for communication over a wired or wireless local or wide area network, a broadband cellular network, etc. In some embodiments, the communication subsystem may enable the computing system 400 to send and / or receive messages to and from other devices over a network such as the Internet.

[0073] Furthermore, this disclosure includes configurations as follows:

[0074] Example 1. A robot laser ablation system comprising: a robot having an end effector that moves with multiple degrees of freedom; a laser component attached to the end effector; a camera that captures light and depth information in an input image of a workpiece; and a processing circuit and associated memory that stores instructions, wherein when an instruction is executed, the processing circuit receives an input image, segments the input image to generate image segments that depict at least a portion of the workpiece, applies a surface fitting algorithm to generate a polynomial surface that fits the surface in the image segment, calculates a set of surface normal vectors that cross the polynomial surface, generates a set of waypoints offset by an offset length from the polynomial surface, programs a motion path connecting the multiple waypoints, and moves the robot's end effector based on the motion path, thereby causing the laser component to perform laser ablation on a portion of the workpiece's surface.

[0075] Example 2. The robotic laser ablation system according to Example 1, wherein segmentation is at least partially achieved by identifying seed points in an input image and implementing a segmentation algorithm using the seed points to identify regions of pixels in the input image that depict a portion of the surface of a workpiece, wherein each pixel in the region has a corresponding two-dimensional coordinate in the input image and a corresponding depth coordinate from the camera to the surface.

[0076] Example 3. The robotic laser ablation system described in Example 2, wherein the processing circuit is configured to display a confirmation selector on a graphical user interface (GUI) that receives user input to confirm a seed point and / or user input to confirm that the image segment contains a surface.

[0077] Example 4. The robotic laser ablation system according to Example 2, wherein the identification of the seed point is at least partially achieved by user selection of the seed point via a graphical user interface (GUI) or by programmatic determination of the seed point based on criteria.

[0078] Example 5. A robotic laser ablation system according to any one example of Examples 1 to 4, wherein the processing circuit is configured to define a two-dimensional frame containing each subset of waypoints, the subset of waypoints within each two-dimensional frame is connected by subpaths of motion paths, the motion paths are generated in camera space, and the processing circuit is further configured to convert the motion paths into a reference frame of the robot.

[0079] Example 6. A robotic laser ablation system according to any one of Examples 1 to 5, wherein the laser component includes a laser source and / or a laser guide optical element, and the laser ablation is performed by moving a robot to cause a laser component mounted on an end effector to travel along a motion path, and by intermittently or continuously exciting the laser source as the laser component travels along the motion path.

[0080] Example 7. A robotic laser ablation system according to any one of Examples 1 to 6, wherein the processing circuit is configured to segment an input image using a segmentation model selected from the group consisting of a semantic segmentation model, an instance segmentation model, a panoptic segmentation model, an edge detection model, a region-based segmentation model, a clustering-based segmentation model, an attention-based segmentation model, a real-time segmentation model, a nearest neighbor segmentation model, and a segment-anything model.

[0081] Example 8. A robotic laser ablation system according to any one example of Examples 1 to 7, wherein the workpiece is one of a plurality of workpieces configured to be simultaneously captured in the image by the camera, and segmentation by a processing circuit further segments the input image into a plurality of image segments, each image segment containing the plurality of workpieces.

[0082] Example 9. A robotic laser ablation system as described in any one of Examples 1 to 8, wherein laser ablation is applied in a square, rectangular, triangular, and / or hexagonal pattern.

[0083] Example 10. A robotic laser ablation system according to any one of Examples 1 to 9, wherein a camera is further configured to take confirmation images of a portion of the workpiece surface during and / or after performing laser ablation on that portion of the workpiece surface, and a processing circuit is further configured to determine, based on the confirmation images, that the quality of the laser ablation on the portion of the workpiece surface is insufficient, and in response to that determination, to perform further laser ablation on the portion of the workpiece surface.

[0084] Example 11. A method for facilitating robotic laser ablation, comprising: receiving an input image of a workpiece from a camera mounted on the end effector of a robot, wherein the end effector is configured to move in multiple degrees of freedom and the camera is configured to capture light and depth information in the input image of the workpiece; segmenting the input image to generate image segments depicting at least a portion of the workpiece; applying a surface fitting algorithm to generate a polynomial surface that fits the surface in the image segments; calculating a set of surface normal vectors crossing the polynomial surface and generating a set of waypoints offset by an offset length from the polynomial surface; programmatically generating a motion path connecting multiple waypoints from the set of waypoints; and performing laser ablation of a portion of the surface of the workpiece based on the motion path using a laser component mounted on the end effector of a robot.

[0085] Example 12. The method according to Example 11, wherein the segmentation step is at least partially realized by the step of identifying a seed point in an input image, and implementing a segmentation algorithm using the seed point to identify a region of pixels in the input image that depicts a portion of the surface of a workpiece, wherein each pixel in the region has a corresponding two-dimensional coordinate in the input image and a corresponding depth coordinate from the camera to the surface.

[0086] Example 13. The method of Example 12, further comprising the step of displaying a confirmation selector on a graphical user interface (GUI) that receives user input to confirm a seed point and / or user input to confirm that an image segment contains a surface.

[0087] Example 14. The method of Example 12, wherein the step of identifying a seed point is at least partially accomplished by the step of obtaining a user selection of a seed point via a graphical user interface (GUI), or by the step of programmatically determining a seed point based on criteria.

[0088] Example 15. The method according to any one example from Examples 11 to 14, further comprising the step of defining a two-dimensional frame containing each subset of waypoints, wherein the subset of waypoints in each two-dimensional frame is connected by subpaths of motion paths, the step of generating includes generating motion paths in camera space, and the processing circuit is further configured to convert motion paths into a reference frame of the robot.

[0089] Example 16. The method according to any one example of Examples 11 to 15, wherein the laser component includes a laser source and / or a laser guide optical element, and the step of performing laser ablation includes moving a robot to move a laser component attached to an end effector along a motion path, and intermittently or continuously exciting the laser source as the laser component moves along the motion path.

[0090] Example 17. The method according to any one example from Examples 11 to 16, wherein the step of segmenting the input image uses a segmentation model selected from the group consisting of semantic segmentation models, instance segmentation models, panoptic segmentation models, edge detection models, region-based segmentation models, clustering-based segmentation models, attention-based segmentation models, real-time segmentation models, nearest neighbor segmentation models, and segment-anything models.

[0091] Example 18. The method according to any one example of Examples 11 to 17, wherein the workpiece is one of a plurality of workpieces configured to be captured simultaneously in an image by a camera, and the segmentation step further comprises segmenting the input image into a plurality of image segments, each image segment comprising the plurality of workpieces.

[0092] Example 19. The method according to any one example from Examples 11 to 18, wherein laser ablation is applied in a square, rectangular, triangular, and / or hexagonal pattern.

[0093] Example 20. A robot configured to move an end effector with multiple degrees of freedom; a laser component attached to the end effector, comprising a laser source and / or a laser guide optical element; a camera attached to the end effector, configured to capture an input image of a workpiece, wherein the input image includes color information and depth information for each of a plurality of pixels in the input image; and associated memory for storing a processing circuit and instructions, wherein, when executed, the instructions cause the processing circuit to receive the input image and obtain a segmented image of at least a portion of the workpiece, comprising the steps of identifying seed points in the input image, the step of identifying seed points being at least partially realized by obtaining a user selection of seed points via a graphical user interface (GUI) or by programmatically determining seed points based on criteria; and implementing a segmentation algorithm using the seed points to identify a region of pixels in the input image that depicts a portion of the surface of the workpiece, wherein each pixel in the region However, the input image is segmented to have relevant 2D coordinates in the input image and relevant depth coordinates from the camera to the surface, at least partially by the steps of: applying a surface fitting algorithm to generate a polynomial surface that fits the surface in the segmented image; calculating a set of surface normal vectors crossing the polynomial surface; generating a set of waypoints offset from the polynomial surface by the offset length of the surface normal vectors; defining a 2D frame containing each subset of the waypoints; programmatically generating a motion path connecting the waypoints; the subsets of waypoints in each 2D frame being connected by subpaths of the motion path; the motion path being generated in camera space; the processing circuit being further configured to convert the motion path to a reference frame of the robot; and moving the robot to move a laser component attached to an end effector along the converted motion path to laser ablate a portion of the surface of the workpiece; and as the laser component moves along the converted motion path,A robotic laser ablation system comprising a processing circuit and associated memory for storing instructions, which are executed by intermittently or continuously exciting a laser source.

[0094] As used herein, "and / or" is defined as inclusive or ∨, as specified by the truth table below.

[0095] [Table 1]

[0096] The configurations and / or methods described herein are illustrative in nature, and it will be understood that these particular embodiments or examples should not be considered restrictive, as numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Thus, the various actions illustrated and / or described may be performed in the order illustrated and / or described, in other orders, in parallel, or in any order. Similarly, the order of the processes described above may be changed.

[0097] The subject matter of this disclosure includes all novel and non-obvious combinations and subcombinations of the various processes, systems, and configurations disclosed herein, as well as all other features, functions, actions, and / or characteristics, and all their equivalents. [Explanation of Symbols]

[0098] 100 Robot laser ablation system, 102 Robot, 102A Base, 103 Actuator, 104 End effector, 106 Laser components, 106A Laser light source, 106B Laser guide optical element, 108 Camera, 108A Camera, 109 Displacement sensor, 110 Related computing device, 110A Display, 111 Wireless connection, 112 Processing circuit, 113 Laser ablation program, 113A Camera module, 113B Segmentation module, 113C Surface fitting module, 113D Path generation module, 113E Driver module, 114 Memory, 116 GUI, 116A Control unit, 118 Image example, 120 Workpiece, 122 Side view, 124 Light, 126 Field of view, 130 Camera axis, 140 Surface normal vector, 142 Waypoint, 144 Origin, 146 Motion Path, 146A Subpath, 148 2D Frame, 200 Method, 120, 320 Workpiece, 322 Side View, 118, 324 Input Image, 326 Segment, 328 Seed Point, 330 Polynomial Surface, 331 Set, 332 Ablation Zone, 400 Computing System, 402 Processing Circuit, 404 Volatile Memory, 406 Non-Volatile Memory Device, 408 Display Subsystem, 410 Input Subsystem, 412 Communication Subsystem

Claims

1. A robot (102) having an end effector (104) that moves with multiple degrees of freedom, The laser component (106) attached to the end effector (104), A camera (108) that captures light and depth information within the input image (118, 324) of the workpiece, A processing circuit (112) and an associated memory (114) for storing instructions, wherein when an instruction is executed, the processing circuit (112) receives the instruction. The aforementioned input images (118, 324) are received, In order to generate image segments that depict at least a portion of the workpiece (120, 320), the input image (118, 324) is segmented. To generate a polynomial surface (330) that fits the surface within the image segment (326), a surface fitting algorithm is applied. The system calculates a set of surface normal vectors that cross the polynomial surface (330), and generates a set of waypoints (142) that are offset from the polynomial surface (330) by an offset length. The program generates a motion path (146) that connects multiple waypoints (142), and By moving the end effector (104) of the robot (102) based on the motion path (146), the laser component (106) is used to perform laser ablation on the surface of a portion of the workpiece (120, 320). Processing circuit (112) and associated memory (114) for storing instructions A robotic laser ablation system (100) is provided.

2. The aforementioned segmentation is Identifying the seed point (328) in the aforementioned input image (118, 324), and Implementing a segmentation algorithm using the seed point (328) to identify a region of pixels in the input image (118, 324) depicting a portion of the surface of the workpiece (120, 320), wherein each pixel in the region has a corresponding two-dimensional coordinate in the input image (118, 324) and a corresponding depth coordinate from the camera (108) to the surface. A robotic laser ablation system (100) according to claim 1, which is at least partially realized by...

3. The robotic laser ablation system (100) according to claim 2, wherein the processing circuit (112) is configured to display a confirmation selector on a graphical user interface (GUI) (116) that receives user input confirming the seed point (328) and / or user input confirming that the image segment (326) includes the surface.

4. Identifying the aforementioned seed point (328) User selection of the seed point (328) via a graphical user interface (GUI) (116), or Program determination of the seed point (328) based on the judgment criteria The robotic laser ablation system (100) according to claim 2, which is at least partially realized by...

5. The processing circuit (112) is configured to define a two-dimensional frame (148) containing each subset of the waypoints (142), and the subsets of the waypoints (142) within each two-dimensional frame (148) are connected by subpaths (146A) of the motion path (146). The motion path (146) is generated in the camera space, and the processing circuit (112) is further configured to convert the motion path (146) into a reference frame of the robot (102). The robotic laser ablation system (100) according to claim 1.

6. The laser component (106) includes a laser source (106A) and / or a laser guide optical element (106B), and the laser ablation is performed To move the robot (102) so that the laser component (106) attached to the end effector (104) travels along the motion path (142), and As the laser component (106) travels along the motion path (142), the laser source (106A) is excited intermittently or continuously. A robotic laser ablation system (100) according to claim 1, which is performed by...

7. The robotic laser ablation system (100) according to claim 1, wherein the processing circuit (112) is configured to segment the input image (118, 324) using a segmentation model selected from the group consisting of a semantic segmentation model, an instance segmentation model, a panoptic segmentation model, an edge detection model, a region-based segmentation model, a clustering-based segmentation model, an attention-based segmentation model, a real-time segmentation model, a nearest neighbor segmentation model, and a segment-anything model.

8. The robotic laser ablation system (100) according to claim 1, wherein the workpiece (120, 320) is one of a plurality of workpieces (120, 320) configured to be simultaneously captured in the image by the camera (108), and the segmentation by the processing circuit (112) further segments the input image (118, 324) into a plurality of image segments (326), each image segment (326) includes one of the plurality of workpieces (120, 320).

9. The robotic laser ablation system (100) according to claim 1, wherein the laser ablation is applied in a square, rectangular, triangular, and / or hexagonal pattern.

10. The camera (108) is further configured to capture confirmation images of the portion of the surface of the workpiece (120, 320) during and / or after the laser ablation of the portion of the surface of the workpiece (120, 320), The processing circuit (112) Based on the aforementioned confirmation image, it is determined that the quality of the laser ablation on a portion of the surface of the workpiece is insufficient. In response to the determination, perform further laser ablation on the surface of the portion of the workpiece. The robotic laser ablation system (100) according to claim 1, further configured as follows.

11. A method for facilitating robotic laser ablation (200), Step (204) of receiving an input image of a workpiece from a camera mounted on the end effector of a robot, wherein the end effector is configured to move with multiple degrees of freedom, and the camera is configured to capture light and depth information of the workpiece in the input image, The steps include (205) segmenting the input image to generate image segments that depict at least a portion of the workpiece, The steps include (210) applying a surface fitting algorithm to generate a polynomial surface that fits the surface within the image segment, The steps (212) include calculating a set of surface normal vectors that cross the polynomial surface and generating a set of waypoints offset by an offset length from the polynomial surface, The steps include (214) generating a programmatically generated motion path connecting multiple waypoints from the set of waypoints, (218) A step of performing laser ablation of the surface of the portion of the workpiece based on the motion path using a laser component attached to the end effector of the robot. Method (200), including.

12. The segmentation step is Step (206) of identifying a seed point in the input image, and Step (208) Implementing a segmentation algorithm using the seed points to identify a region of pixels in the input image that depicts the surface of the portion of the workpiece, wherein each pixel in the region has a corresponding two-dimensional coordinate in the input image and a corresponding depth coordinate from the camera to the surface. The method according to claim 11 (200), which is at least partially realized by...

13. The method according to claim 12 (200), further comprising the step of displaying a confirmation selector on a graphical user interface (GUI) that receives user input confirming the seed point and / or user input confirming that the image segment includes the surface.

14. The step (206) of identifying the seed point, A step of obtaining the user selection of the seed point via a graphical user interface (GUI), or A step in which the program determines the seed point based on the criteria. The method according to claim 12 (200), which is at least partially realized by...

15. The method according to claim 11 (200), further comprising the step (214) of defining a two-dimensional frame containing each subset of the waypoints, wherein the subset of waypoints in each two-dimensional frame is connected by subpaths of the motion path, the generating step includes the motion path being generated in camera space, and a processing circuit is further configured to convert the motion path into a reference frame of the robot.

16. The laser component includes a laser source and / or a laser guide optical element, and the step (218) of performing laser ablation is, The steps include: moving the robot to move the laser component mounted on the end effector along the motion path; The steps include intermittently or continuously exciting the laser source as the laser component travels along the motion path, and The method according to claim 11 (200), including the method according to claim 11.

17. The method according to claim 11 (200), wherein the step of segmenting the input image uses a segmentation model selected from the group consisting of a semantic segmentation model, an instance segmentation model, a panoptic segmentation model, an edge detection model, a region-based segmentation model, a clustering-based segmentation model, an attention-based segmentation model, a real-time segmentation model, a nearest neighbor segmentation model, and a segment-anything model.

18. The method according to claim 11 (200), wherein the workpiece is one of a plurality of workpieces configured to be simultaneously captured in the image by the camera, and the segmenting step further comprises segmenting the input image into a plurality of image segments, each image segment comprising one of the plurality of workpieces.

19. The method according to claim 11, wherein the laser ablation is applied in a square, rectangular, triangular, and / or hexagonal pattern.

20. A robot (102) configured to move the end effector (104) with multiple degrees of freedom, A laser component (106) attached to the end effector (104), the laser component (106) including a laser source (106A) and / or a laser guide optical element (106B), A camera (108) attached to the end effector (104) is configured to capture input images (118, 324) of the workpiece (120, 320), and the input images (118, 324) include color information and depth information for each of the multiple pixels in the input images (118, 324), and the camera (108) A processing circuit (114) and an associated memory for storing instructions, wherein when an instruction is executed, the processing circuit, The aforementioned input images (118, 324) are received, The segmented images of at least a portion of the workpiece are A step of identifying a seed point (328) in the input image (118, 324), wherein the step of identifying the seed point (328) is, Obtaining the user selection of the seed point (328) via a graphical user interface (GUI) (116), or The program determines the seed score (328) based on the judgment criteria. This is achieved, at least partially, by the steps, A step of implementing a segmentation algorithm using the seed point (328) to identify a region of pixels in the input image that depicts the surface of a portion of the workpiece, wherein each pixel in the region has a corresponding two-dimensional coordinate in the input image (118, 324) and a corresponding depth coordinate from the camera (108) to the surface. To generate at least partially by this, the input image (118, 324) is segmented, To generate a polynomial surface (330) that fits the surface in the segmented image, a surface fitting algorithm is applied. The set of surface normal vectors crossing the polynomial surface (330) is calculated, and a set of waypoints (142) offset from the polynomial surface (330) by the offset length of the surface normal vectors is generated. Define a two-dimensional frame (148) that includes each subset of the aforementioned waypoint (142). A program generates a motion path (146) connecting the waypoints (142), the subset of waypoints (142) in each 2D frame is connected by subpaths of the motion path (146), the motion path (146) is generated in camera space, and the processing circuit is further configured to convert the motion path (146) into a reference frame of the robot (102). Laser ablation of the surface of a portion of the workpiece (120, 320) To move the robot (102) along the converted motion path (146) to move the laser component (106) attached to the end effector (104), and As the laser component (106) moves along the converted motion path (146), the laser source (106A) is excited intermittently or continuously. The processing circuit (114) and associated memory that stores instructions are executed by this. A robotic laser ablation system (100) is provided.

Images