Planning and reactive execution of defect treatment with robots on the basis of low-cost inspection data

By integrating compact inspection sensors for real-time re-measurement and adaptive path planning, the method addresses the inaccuracies in defect processing on complex surfaces, ensuring precise and efficient defect repair while avoiding damage, thus improving the overall quality and efficiency of the process.

WO2024231476A1PCT designated stage expired Publication Date: 2024-11-14CONVERGENT INFORMATION TECH

Patent Information

Application Number
PCT/EP2024/062786
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-08
Filing Date
2024-05-08
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Current methods for defect processing by robots using low-cost inspection data are inaccurate, leading to increased cycle times, wear, and quality degradation due to position tolerances and incorrect defect classifications, especially on surfaces with curvatures or edges, resulting in potential damage and unnecessary processing or missed defects.

Method used

Equipping robots with compact inspection sensors for real-time defect re-measurement during processing, allowing for adaptive robot path planning based on precise defect data to ensure accurate positioning and processing of defects, while anticipating measurement errors and simulating alternative defect positions to avoid surface damage.

Benefits of technology

This approach enhances the accuracy of defect processing, reduces cycle time, and improves quality by ensuring precise robot movements and tool paths, minimizing errors and unnecessary processing, and effectively handling defects on complex surfaces without damaging them.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2024062786_14112024_PF_FP_ABST
    Figure EP2024062786_14112024_PF_FP_ABST
Patent Text Reader

Abstract

A method for robot path planning for the purpose of carrying out a robot-assisted repair of surface defects is described (spot repair). According to one example, the method comprises the following: carrying out a first visual inspection of a workpiece by means of one or more first sensors; detecting defects on the surface of the workpiece and determining first defect data, which characterise the detected defects, on the basis of measurement data that were picked up during the first visual inspection; carrying out – for at least one defect of the detected defects and on the basis of the associated first defect data – automated, computer-assisted first robot path planning in order to determine a robot path for a second inspection of the workpiece in the region around the at least one defect and in order to determine one or more robot paths for treating the at least one defect; carrying out the second visual inspection of the workpiece by means of one or more robot-guided, second sensors and determining second defect data, which characterise the at least one defect, on the basis of measurement data that were measured during the second visual inspection; checking – on the basis of the second defect data – whether the at least one defect can be treated with a first of the planned robot paths or – if present – another one of the planned robot paths, and if so, carrying out automatic, robot-assisted treatment of the at least one defect using the robot path in question; if the at least one defect cannot be treated with one of the planned robot paths: carrying out second robot path planning in order to determine a new robot path for treating the at least one defect on the basis of the second defect data in question, and carrying out automatic, robot-assisted treatment of the at least one defect using the new robot path.
Need to check novelty before this filing date? Find Prior Art

Description

PLANNING AND REACTIVE PROCESSING OF DEFECT REMOVAL WITH ROBOTS BASED ON LOW-COST INSPECTION DATA TECHNICAL FIELD

[0001] The present invention relates to the computer-aided (automatic and semi-automatic) programming of robots, for example robots for repairing surface defects or reworking workpiece surfaces according to input from sensors. BACKGROUND

[0002] One task of robots in production is the automated processing of workpiece surfaces to repair previously detected defects on the surface. To program such tasks, executable robot programs are created using reactive automatic programming, which can be implemented by a robot controller (see, for example, US 2003 / 0139836 A1). A robot program describes commands that can be processed by the robot controller, which the robot controller converts into corresponding control signals to the robot and the tool operated by the robot (e.g., grinding tool, polishing tool, spray gun for applying polishing agent), so that the robot performs the programmed movements and actions to achieve the desired processing of the workpiece surface.

[0003] For the actual programming, a list of previously found defects on the surface is interpreted, and based on this list, movements of one or more tools over or on the surface in the area of ​​these defects are planned. Commands linked to these movements (e.g., motors on / off, spray application, wait) are also planned, as are intermediate movements between the individual areas in which machining is to take place. For the planning of the movement on the surface, for example, "recipes" (i.e., a combination of tool movements and specific machining parameters (pressure, speed, . . .) and commands (motor on / off, spraying, ....) are selected. This selection is made depending on the type and severity of the defect to be machined, as well as the shape of the local workpiece surface. The selected recipe (tool path template) is applied to the relevant surface area (e.g.,by projecting onto the surface or rolling or. using other methods). In advantageous embodiments, the applied tool paths are modified and / or shifted depending on the nature or shape of the surface in order to avoid damaging edges or gaps, for example.

[0004] In the past, defects were detected and subsequently classified using robot-guided high-end sensors (such as reflectometry, deflectometry, 3D profiling sensors, or 3D cameras), which were precisely guided over the surface of the moving or stationary components (workpieces) (see US 11,110,611 B2). Such high-end sensors and the associated (computer) systems are complex and expensive. Therefore, attempts were made to replace defect detection using such high-end sensors, which were moved by robots depending on the component so that they were guided relatively close and in a preferred orientation over the component surface, with more compact or cost-effective methods. With such methods, the inspection or scanning of the components / workpieces is carried out, for example, with permanently mounted cameras (e.g.in a light tunnel) while the component / workpiece is moving, or while the sensors are moved over the component at a greater distance and in a non-optimal alignment.

[0005] With this new type of defect detection, position tolerances, tolerances in the opening angles of components (e.g., doors), less precise movement of components compared to the movements of sensors with robots, speed deviations, etc., have a greater impact. The detected defects are often determined with significantly less accuracy in their position, and the number of incorrect classifications of defects and errors in determining their parameters or characteristics increases.

[0006] One way to deal with the aforementioned inaccuracies and errors would be to process larger areas around each detected defect. However, this increases cycle time and wear and tear, and reduces the quality of error correction. This simple method can also mistakenly process defects that would not be processed at all if more precise information about the defect had been available, e.g., defects that are non-critical / negligible, too large, or too severe for automatic processing.

[0007] Another possibility is to use a reduced number of robots and sensors after the first (inaccurate) inspection to inspect only the areas around the previously found To inspect defects more closely (e.g., with robot-guided sensors) in order to correct their positional accuracy and classification, and to determine their characteristics and parameters more precisely. The inspection is planned based on the originally determined defect list. In an advantageous implementation, those defects that would not be processed according to the original inspection, but which are close to the threshold values ​​that lead to non-processing, are also inspected. The defect list, corrected with regard to the position and class of the defects, processing decision, etc., is then used to plan the actual defect repair in the subsequent processing cell (instead of the original, inaccurate and less reliable first measurement / inspection). However, this method reduces the cost and space advantages of the original, cheaper solutions (e.g., automated light tunnels).In an advantageous implementation, this subsequent inspection is performed using line tracking to reduce the additional space and hardware requirements. This is particularly possible because an inspection of only the relevant defects detected beforehand requires significantly less time than an inspection of the entire component or subsequent processing of the relevant defects.

[0008] The execution of a path correction (e.g. by means of offsets) during grinding, polishing, spraying, etc. to compensate for the original positional deviation, as supported by state-of-the-art robot controls, is only effective for simple surfaces without strong curvatures or edges. Such an offset correction, which is common in robots (correction of the originally planned robot path by an offset), is primarily intended for the correction of the measured position of a component or part of a component. If a path planned for a defect on the component is shifted / corrected via an offset in position, this can lead to damage to the surface in areas of the component with stronger curvatures or edges, since firstly a recipe is applied without appropriate adaptation to the changed surface shape and secondly shape features such asDifferent recipes would be applied to edges and gaps than at the location where the defect was originally measured / suspected - and vice versa. Due to the initial inaccuracy, it is also possible that the defect was detected on the wrong side of a feature (such as a gap or an edge), so that during machining a significantly different (incorrect) recipe is executed at a different (incorrect) location with opposite (incorrect) deformation or displacement in relation to the feature, or that the machining takes place on a different component of the part than where the defect actually lies.

[0009] Furthermore, a mere offset shift to correct the position can lead to a collision of the robot or tool with the component, with parts of the environment, or with other robots or tools. Furthermore, this approach does not prevent defects that were classified as just barely "machinable" due to less accurate inspection, e.g. in the light tunnel, but would be classified as "not machineable" upon closer inspection (as too large, too small, etc., wrong type) and therefore not machineable, from being unnecessarily machined. Furthermore, it does not prevent defects that were just barely classified as "not machineable" due to less accurate inspection, but would be classified as "machinable" upon closer inspection, from being incorrectly left unmachined. OVERVIEW

[0010] As a space- and cost-saving alternative, it is proposed to equip the robots that are responsible for processing the defects found with a compact inspection sensor in addition to the processing tool and to plan not only the repair of the defects but also a prior re-measurement / re-inspection of the defects based on the defect lists (e.g. based on the inspection in the light tunnel).

[0011] In the following, "online" refers to a process performed in the machining cell, while "inline" refers to a process performed between the inspection and the machining cell. A "subplan" is a section of a plan, e.g., solely for the repair of an individual defect. A "plan" is the link between subplans and intermediate movements / re-inspection (local, more detailed inspection of a defect found during a previously performed inspection). The (robot) program follows directly from a plan.

[0012] The example described here relates to a method for robot path planning for the purpose of performing a robot-assisted repair of surface defects (spot repair). According to one example, the method comprises the following: performing a first optical inspection of a workpiece by means of one or more first sensors; detecting defects on the surface of the workpiece and determining first defect data that characterize the detected defects based on measurement data measured during the first optical inspection; performing - for at least one of the detected defects and based on the associated first defect data - an automatic, computer-assisted first robot path planning for Determination of a robot path for a second inspection of the workpiece in the area around the at least one defect and for determining one or more robot paths for processing the at least one defect; carrying out the second optical inspection of the workpiece by means of one or more robot-guided, second sensors and determining second defect data that characterize the at least one defect, based on measurement data measured during the second optical inspection; checking - based on the second defect data - whether the at least one defect can be processed with a first of the planned robot paths or - if available - another of the planned robot paths, and, if so, automatically, robot-assisted processing of the at least one defect using the respective robot path;if the at least one defect cannot be processed with one of the planned robot paths: performing a second robot path planning to determine a new robot path for processing the at least one defect based on the respective second defect data, and automatically, robot-assisted processing the at least one defect using the new robot path;

[0013] In one embodiment, the following procedure is used to automatically detect defects on the surface of a workpiece and to process them using a robot (spot repair). First, a first optical inspection of a workpiece is carried out using one or more first sensors. These (first) sensors can have a comparatively low accuracy. This is followed by the detection of defects on the surface of the workpiece and the determination of first defect data that characterize the detected defects, based on measurement data that was measured during the first optical inspection. The defect data can include the position of a defect on the workpiece surface (defect coordinates) as well as the measured characteristics (e.g. size, type) of a defect.Subsequently, for at least one of the detected defects and based on the associated first defect data, an automatic, computer-assisted first robot path planning is carried out to determine a robot path for a second inspection of the workpiece in the area around the at least one defect, as well as to determine one or more robot paths for processing the at least one defect. This is followed by carrying out the second optical inspection of the workpiece using one or more robot-guided, second sensors, and determining second defect data that characterize the respective defect based on measurement data measured during the second optical inspection. The data for the second The sensors used for inspection have a comparatively high accuracy, which is why the second defect data have a smaller uncertainty. Based on the second Defect data is checked to determine whether the at least one defect can be processed using a first of the planned robot paths or, if available, another of the planned robot paths. If so, the automatic, robot-assisted processing of the at least one defect is carried out using the respective robot path. If the at least one defect cannot be processed using one of the planned robot paths, a second robot path planning is carried out to determine a new robot path for processing the respective defect based on the respective second defect data. The automatic, robot-assisted processing of the respective defect is then carried out using the new robot path.

[0014] The initial inspection to detect defects, conducted in a light tunnel or with less precise, space- and cost-saving methods, is expanded to include an expected uncertainty in the identified position of the defect on the component, either statically determined or individually determined during measurement. The description of the component's defects (referred to above as defect data), e.g., in a list, is expanded to include this deviation.

[0015] In one embodiment, the aforementioned first robot path planning comprises planning a robot path based on the first defect data (i.e., based on the measurement data from the first inspection) as well as planning further robot paths for various modified (first) defect data. In particular, modified defect data can contain modified defect positions. This means that the defect in question is "virtually" shifted (anticipating a measurement error during the first inspection) to obtain the modified defect data. In this way, a plurality of modified defect data can be calculated, each for a modified defect position, in particular within the range of expected uncertainty around the originally measured defect position.

[0016] In another embodiment, defects that were determined to be borderline "not to be treated" based on their class, severity, or parameters during the initial inspection are also included in the list as "to be treated." In a subsequent inspection, these areas could be inspected again, more closely, and a new decision can be made regarding these defects.

[0017] After the first (initial) inspection, the processing of this list of defects is initially planned. However, in the initial processing planning, a preliminary re-inspection is planned in addition to the actual repair. This is planned for the The area of ​​the component around the respective defect is planned, which includes at least the expected deviation or inaccuracy of the defect in its position and the largest expected size of the defect. In other words, the combined area is determined, which is formed from the possible positions of a defect and the resulting occupied areas of the respective defect depending on the optical properties (ray paths).

[0018] The initial (in-line) planning of a suitable repair strategy can, according to one embodiment, be based on the recipe that matches the initially detected and characterized defect, i.e., the recipe used in path planning results from the determined position, class, and severity of the respective defect. During the initial robot path planning, according to one embodiment, the surface area in which the true defect can actually be located can also be determined (e.g., via processing simulation, heuristics, or machine learning) in order to be satisfactorily processed with the recipe applied unchanged. This means that the deviations between the true defect (or precisely inspected defect) and the inaccurately detected defect are determined, at which the defect could be processed with the planned program without subsequent adjustment of the recipe. In one embodiment, this area, orThe deviations are verified by sampling the area around the initially identified defect, simulating the processing of a defect of each sample (i.e., with a virtually shifted defect position) using the processing planned for the initial defect. The samples for which processing was simulated as "OK" with the original partial plan are grouped into a "Group 1." In an advantageous implementation, the size of the sampled area is determined by the individually determined, expected uncertainty in the identified position of the defect.

[0019] The defect data can be viewed from the parameter set. Through the aforementioned sampling, possible measurement errors during the initial (inaccurate) inspection are anticipated. The measured parameter set is only one possible parameter set that describes the respective defect. Due to the inaccurate measurement, the measured parameter set that characterizes a specific defect is subject to uncertainty. This uncertainty range can be discretized (in various ways), and each point in the state space of all possible parameter sets is a "sample," i.e., a modified parameter set (modified defect data) that could also describe the real defect. In particular, the samples are formed by modifying the originally measured defect position.

[0020] In a further variant, alternatively or additionally, the initial planning determines (e.g., by means of simulation) how far a defect can be from the initially determined defect location, so that a simple transformation of the robot program, which was planned for errors at the original defect location, is still sufficient to repair the defect at its actual position or a position determined more precisely later, without damaging the surface of the component (defects in “Group 2”). This means that only an offset that can be called up later in the robot controller during processing (usually only in x, y, z coordinates) is simulated, without intelligent replanning of the robot path to repair the defect in question. In other words, the “replanning” of the robot path is reduced to the modification (e.g., by applying an offset to the tool center point, TCP) of an already planned robot path.For this purpose, the surface around the defect is preferably scanned again (sampling) to determine possible alternative defect positions, with each sample representing a different defect position (with correspondingly modified defect data). In an advantageous implementation, this check is performed by a collision check of the shifted robot movement and a simulation (process simulation) of the machined area. If the simulation shows that a defect at the sample location would be satisfactorily repaired (OK) with a partial plan or robot program shifted by an offset, then this sample is assigned to "Group 2."Alternatively, if very short planning times are required, this can be checked by a simple curvature analysis of the surface around the defect instead of the machining simulation and / or by using larger collision models for tools and robots and an additional restriction of the allowed axis ranges of the robot for the simulation of the movement at the original location instead of a new collision check at the possible real locations.

[0021] According to the concept presented here, an initial optical inspection is carried out using (one or more) comparatively inaccurate and inexpensive sensors, and potential measurement errors are anticipated during the initial robot path planning by not only planning a robot path for the actually detected defect (i.e., based on the measured defect data), but also considering modified defects (and defect data). All of these modified defects or the defect data associated with them could - theoretically, assuming a random measurement error – lead to the same measurement data. Path planning based on the modified defect data can be simplified by checking whether a previously planned robot path can be reused (Group 1), possibly applying a TCP offset (Group 2). During the second inspection, which takes place later with significantly more accurate, robot-guided sensors, each of the previously anticipated modified defects (or the associated modified defect data) is a possible measurement result.

[0022] Samples which could not be assigned to either group 1 or group 2, i.e. possible alternative defect positions (samples / modified defects) which, according to the simulation, cannot be satisfactorily repaired (OK) with either the initially planned (unchanged) robot program or the robot program executed with an offset, are assigned to “Group 3”. Defects which are assigned to this group during the later online re-inspection are, depending on the implementation, either skipped or a new path planning is carried out online for these defects in group 3 and then processed. In another embodiment, an additional robot path is planned anticipatorily (in advance, so to speak) for the defects in group 3, which in this case is not possible by simply applying an offset and is therefore correspondingly more complex (and can require more computing time (up to several minutes)).

[0023] In an advantageous implementation, the repair of a defect at the sample's position is initially planned for each sample in group 3. This means that the applicable recipe is individually determined for each sample in group 3, its application to the surface, deformation, or displacement, as well as the robot movements are planned.

[0024] In an alternative, more computationally intensive but better in terms of robustness and process results, not only is a possible repair planned for the defect at the originally detected location and for the originally detected defect class, and then changes in the defect position are analyzed, but the repair is also planned for various possible defect positions (samples) in parallel and saved for later online use. For the determination, the uncertainty area around the initially detected defect is again sampled, either uniformly or depending on the local curvature.

[0025] In an advantageous embodiment, this initial planning of the repair is not only planned for all possible variations of the position of the initially detected defect (i.e. for the geometrically scanned surface), but also for all possible or probable defect classes and manifestations of the defect according to the initial inspection. This means that the initial planning not only anticipates that a later re-inspection will correct the position of the defect on the surface, but also its class, manifestation and parameters. In this embodiment, a sample represents the combination of the possible variations in the position of the defect as well as in the variation of its class, manifestation and parameters. This means that several orMany alternative processing steps are initially planned and saved as alternative sub-plans with reference to the underlying samples, each of which represents a specific combination of position, class and characteristic of a variation of the initially detected defect.

[0026] Particularly close to edges and gaps, e.g. door gaps, etc. (generally referred to as "features"), it can happen that the samples (these alternatively scanned defects) are located on the other side of a feature and the repair is planned for the respective position of the sample. In particular, different recipes are selected depending on the distance to the features or the position (side) to the features. And if, for example, an alternative defect is on the opposite side of a feature to the one originally inspected, the machining patterns are modified in the opposite way or shifted towards / away from the surface feature or deformed, so that the machining is completely different. Likewise, alternative defects or their samples can lie outside or inside the area in which a feature is relevant for machining, so that different machining recipes are selected for the alternative sub-plans.Likewise, depending on the distance and relative position of the simulated / scanned defects from the feature(s) or depending on the surface shape of the component near the defect, there will be or no displacement or deformation along or towards / away from the feature, or there will be or a lesser or greater displacement or deformation of the tool path or path, or there will be or a displacement or deformation of the path in another direction.

[0027] For surfaces with stronger curvatures, at least the initially planned movement, the orientation of the tools during machining and the approach will be significantly different from the initial sub-plan for machining the initially measured defect, and in the different initial sub-plans for different samples, the Robots often adopt a completely different configuration, or possibly a different configuration of their additional axes. Such corrections to the information about the initially inspected defect would not be possible with a simple offset.

[0028] The sampling area and sampling frequency can be predefined, learned (using machine learning), or derived from the sensor information.

[0029] In an alternative, cycle time and memory-saving implementation, it is determined across all samples whether the sub-plans assigned to them for repair use the same recipes and similar tool paths and similar robot configurations and could be summarized, so that all possible variations of the defect could be processed with an "averaged" program (robot path) and the re-inspection can be omitted.

[0030] In an advantageous implementation, an intersection is determined across all samples from the possible robot configurations for the original and alternative sampled defects, for which the most or most likely defects can be reached with short and safe robot movements. In a variation, an average of the intersection of the configurations is used.

[0031] In an advantageous embodiment, the alternative, parallel planned machining operations start at the same initial situation (the refined inspection of the defect) and end with the same robot configuration or a similar configuration, which allows easy and safe planning or smoothing into the subsequent movement to the next defect or to the end.

[0032] To generate one or more plans for each robot from the individual subplans for repairing and, if necessary, prior re-inspecting the individual defects, a sequence is determined for these defects, and collision-free intermediate movements of the robots are planned. If more defects per robot need to be repaired than can be repaired within a specified maximum cycle time, the defects to be repaired or those not to be repaired are selected.

[0033] In a very simple implementation, where only the offset mentioned above is applied to the planned robot movement without the effects of possible In order to anticipate variations in defects in the initial planning, the defects that have not already been rejected as "unprocessable" are selected, e.g. based on the defect class or severity or the determined processing time of the defect including geometric criteria or accumulated processing time or by random number generator.

[0034] In one example, the areas around the defects are sampled, and then the repair is simulated for each sample using a (hypothetical) application of the original program (with offset if necessary), or an individual partial plan is planned for each sample. The results of the sampling are taken into account when selecting the defects to be repaired. To optimize cycle time, preference is given to defects that, according to the initial simulation, do not require re-inspection and can be processed with the program unchanged across the entire uncertainty range of the identified defect. Other positive criteria for selecting defects that must be re-inspected are: (1.) smallest number or percentage of samples that cannot be processed, i.e. the highest probability of later successful processing, (2.) Highest number of samples that can be processed either using the original program or using the original program with applied offset, respectively, with the lowest probability of requiring replanning. For all samples of a defect for which the repair was individually planned, the determined repair time per sample is weighted by the probability that this sample corresponds to the new measurement, e.g., assumed to be indirectly proportional to the distance of the sample from the initially measured defect position or according to a Gaussian distribution. The resulting average repair time of the defects can also be used as a (third) criterion.

[0035] The intermediate movements between the processing of a found defect or the parallel planned alternative processing of the defect and the next defects are planned as “air motion” until the processing and its previous more detailed inspection / re-measurement for all initially found defects has been carried out.

[0036] If it is determined during initial planning that, within the expected inaccuracy of the initial defect, the defect can always be safely repaired with the "unadjusted" original repair program, so that no alternatives are necessary, then there is no need for an inspection and this can be omitted from the planning. This means that during later execution, the subsequent inspection skipped in order to save cycle time without any significant loss of quality.

[0037] The robot programs are generated for the various robots using commands. Alternatively, only the plan can be saved and used later, with motion commands and commands for the sensors and tools, for example, being fed into the robots step by step / individually in real time.

[0038] During the execution of a robot program, the respective robot moves to the starting position of the respective defect processing according to the initially planned movement (of the robot program) in the event that no adjustment of the defect is necessary and processes it without modification. In this case, an inspection is not performed. If the initial planning reveals that the defect is not in an advantageous location, i.e., a modification of the originally planned defect processing may be necessary, the robot moves to the respective first inspection position and performs the inspection. The result of the re-measurement / inspection can be used as follows.

[0039] In those implementations where multiple alternatives were planned, the most suitable alternative is selected, and the corresponding plan / robot program is executed. This is done by comparing the properties of the inline-determined samples (position, defect class, severity, parameters, if applicable) with the properties of the newly inspected defect online, and determining the most similar sample. The partial plan or partial program for repair assigned to this sample is then executed with the robot.

[0040] In some embodiments, a check is performed to determine whether the newly inspected defect (or its deviation from the initially inspected defect) lies within the inline, stable, simulated area (location and class, severity) (Group 1). In this case, the original program is executed.

[0041] In the event that the defect lies outside the previously determined area (Group 3), the processing must be re-planned online for the exact position of the defect (in examples without pre-planning several alternatives). For this purpose, the planning is advantageously only carried out for the movement from the inspection position to the pre-determined end position of the originally planned processing of the originally determined Defect is performed. Alternatively, planning can be performed from the inspection position until the start of the inspection or processing of the next defect. If the defect is the last defect to be processed in the program or subprogram, planning can be performed until the robot reaches its final position in the program.

[0042] In a further implementation, it is possible to simulate how far the originally (inaccurately) detected defect can be shifted along with the original machining program (Group 2). This means that the entire machining program can be shifted and executed online using a simple offset without damaging the surface or surface features, no longer machining the defect with the required quality, or colliding with the part or other components in the robot cell. This means, for example, that it is determined for which surface samples in the uncertainty range around the inline-inspected defect do not require replanning or planned adjustment, or selection of an alternative plan (Group 2).

[0043] In an advantageous implementation, this is achieved by simulating the displacement and process, and by checking for collisions during initial planning. In an embodiment that is less computationally intensive, this is achieved by a simple analysis of the surface curvature: For almost flat surfaces and displacements of a few millimeters, few problems are to be expected. The collision-free state can be re-simulated. Alternatively, the original defect can be planned and simulated during inline planning with enlarged collision models and artificially further restricted ranges of motion for the robot axes, so that a later shifted / offset version with the original collision models and axis angles lies within the extended collision models and within the true axis restrictions and does not need to be checked separately.

[0044] By mounting the sensor on the tool, an alternative embodiment allows the defect to be re-inspected after machining and, if necessary, re-machined if the initial machining was not completely successful. If the machining was not precise, a new path planning can be performed or an alternative plan can be called and executed.

[0045] In an alternative embodiment, which requires less computing power and possibly Lead time between initial inspection and repair is required, but regarding If the execution time is less than optimal, only the sub-plan for the initially inspected defect is initially planned inline - without alternatives and without checking variations (samples). In this variant, a new inspection must be carried out online in every case. Using the result of the new inspection, the repair is reactively re-planned online using the updated information about the defect. During the planning time, the robot waits at the inspection point (or moves slowly towards the initially planned processing). In one variant of this implementation, in order to reduce the necessary computing time / waiting time, a simulation is carried out in parallel with the re-planning to determine whether the originally inline-planned sub-program is sufficient to repair the newly inspected defect, or whether an offset shift of the program is sufficient.For the first calculated / available result (without offset, with offset or newly planned) that predicts a sufficient result, the partial plan or partial program is used for processing with the robot.

[0046] BRIEF DESCRIPTION OF THE ILLUSTRATIONS

[0047] Various implementations are explained in more detail below using the examples shown in the figures. The illustrations are not necessarily to scale, and the invention is not limited to the aspects shown. Rather, emphasis is placed on illustrating the principles underlying the illustrated embodiments.

[0048] Fig. 1 illustrates the well-known concept of repairing defects with prior and sufficiently accurate inspection.

[0049] Fig. 2 illustrates the repair of defects with prior inaccurate inspection.

[0050] Fig. 3 illustrates the repair of defects with upstream inaccurate inspection and several sequentially arranged processing cells.

[0051] Fig. 4 provides an overview of all the different variants of inline planning of inspection and processing of defects using a flowchart that visualizes the data / signal flow.

[0052] Fig. 5 shows a simple implementation of computer-aided inline planning of a robot path for a defect.

[0053] Fig. 6 illustrates the planning of robot paths for reactive machining with anticipation of the reaction to later position deviations resulting from the subsequent re-inspection of the defects.

[0054] Fig. 7 shows the planning of reactive machining with anticipation of the reaction to later position deviations and changes in the classification or certain severity of a defect during the re-inspection.

[0055] Fig. 8 illustrates online adaptation, ie reactive replanning.

[0056] Fig. 9-12 concerns variants / modifications of the example from fig. 8

[0057] Fig. 13 is a flowchart relating to a general embodiment. DETAILED DESCRIPTION

[0058] Fig. 1 illustrates by way of example the known concept of repairing defects with prior and sufficiently precise inspection, by means of which defects on the surface of the component (workpiece) are detected (ie recognized and localized as such on the component) and characterized (ie assigned to a defect class, described by means of parameters, etc.).

[0059] In the first robot cell (left in Fig. 1), the component moves, for example, through a camera tunnel or is inspected by robot-guided sensors. Based on the results of this inspection, the repair (sport repair) of the detected defects (or a subset of the defects) is planned. Upon entering the processing cell, the position of the component (relative to the robot cell) is preferably determined, and the robot programs for automatically processing the defects are created and executed.

[0060] In Fig. 1, module I designates the inspection cell or inspection tunnel. Reference numeral 1.1 denotes the component to be inspected and subsequently repaired. Reference numeral 1.2 denotes sensors for inspecting the component.

[0061] Module II covers data processing and robot path planning. Reference number II.1 refers to the inspection software for detecting and classifying defects on the component. Reference number II.2 refers to robot path planning for repairing the detected defects. Reference number II.3 refers to the generation of robot programs.

[0062] Module III designates the robot cell for processing defects (spot repair). Reference symbol III.1 denotes the component to be processed. Reference symbol III.2 denotes the robots for processing, optionally with additional axes, and reference symbol III.3 denotes tools for processing the detected defects.

[0063] Fig. 1 illustrates the data flow (reference symbols II.a to II.d) between the individual modules. Reference symbol Ia denotes the component conveying direction, reference symbol II.a denotes the sensor data provided by the inspection cameras. Reference symbol II.b illustrates the list of detected defects, and II.c denotes the resulting path plans for the individual robots to repair the detected defects. Reference symbol II.d denotes the robot programs automatically generated from the path plans.

[0064] Fig. 2 illustrates, using a diagram, the concept described here for repairing defects with upstream inaccurate inspection. While defects can be detected (i.e., identified and localized as such), the detection may not meet the required accuracy. This means that the location of a detected defect is subject to a tolerance, and the classification and decisions based on it (e.g., "to be repaired" or "not to be repaired") of the defect may also be erroneous.

[0065] In the first inspection cell (left in Fig. 2), the component moves through a camera tunnel, for example, and is inspected; the inspection is subject to the aforementioned inaccuracy. Based on the result of this initial inspection, a precise re-inspection of the detected defects and their automated repair is planned. For this purpose, the robots in the processing cell are equipped, in addition to the tools for repairing the defects, with compact inspection sensors for the precise inspection of comparatively small areas around the previously detected defects. Upon entry into the processing cell (right in Fig. 2), the position of the component is preferably measured, and the programs for inspecting and reactively processing the defects are created and reactively adapted and executed depending on the result of the individual inspection.

[0066] In Fig. 2, Module I denotes the inspection cell or the inspection tunnel. Reference numeral 1.1 denotes the component / workpiece that is to be inspected and subsequently repaired. Reference numeral 1.2 denotes the sensors for the inspection of the component. Module II represents the data processing and the automatic, computer-aided robot path planning. Reference numeral II.1 denotes the Inspection software for detecting and classifying defects on the component, II.2 automatic, computer-aided planning of re-inspection and reactive repair of the defects found, and II.3 automatic generation of robot programs based on the planned paths and their reactive execution. Depending on the implementation, replanning and various methods of adapting the processing can be provided. This adaptation depends on the update (re-inspection) of the defects and is carried out with regard to the position, class, and severity of the defects. Reference symbol II.4 represents the inspection module for re-evaluating the defects.

[0067] In Fig. 3, Module III represents the machining cell. Reference symbol III.1 denotes the component to be machined, III.2 denotes the robot for machining, optionally with additional axes, III.3 denotes the tools mounted on the robot for machining the defects, and III.4 denotes the sensors for re-inspecting the defects to be repaired that were detected in the inspection cell (Module I).

[0068] In Fig. 2, the data flow between the individual modules is designated by the reference symbols II. a) to II. d). The reference symbol Ia denotes the direction of movement of the component (e.g., on a conveyor belt). The reference symbol II. a denotes the sensor data from the inspection cameras, II. b denotes the resulting list of defects found, II. c denotes the robot path plans determined based on the list for the re-inspection and adaptive repair for the individual robots, II. d denotes the resulting robot programs, and II. e denotes the sensor data relating to the newly inspected defect. The reference symbol II. f denotes the update of the defect information (including, for example, position, class, characteristic) due to the re-inspection and more detailed inspection.

[0069] Fig. 3 illustrates the principle of defect repair with upstream inaccurate inspection and several sequentially arranged processing cells which do not perform the same process on different defects, but rather successive processes on the same defects, which are inspected again with greater accuracy in the first processing cell. In Fig. 3, module I represents an inspection cell or an inspection tunnel in which the aforementioned optical inspection is carried out to detect or identify defects. This first inspection is carried out with comparatively low accuracy. Reference numeral 1.1 designates the component to be inspected and subsequently repaired. Reference numeral 1.2 designates the sensors for the first (inaccurate) inspection of the component.

[0070] Module II represents data processing and robot path planning based on the measurement data. Reference II.1 designates the inspection software for detecting and classifying defects on the component based on the measurement data. Reference II.2 designates the planning of the re-inspection and reactive repair of the defects found. Reference II.3 designates the generation of robot programs and their reactive execution, in various variants with or without replanning and with various methods of adapting the processing according to the update of the defects in terms of position, class, and severity. Reference II.4 designates the inspection module for re-evaluating defects (with greater accuracy). Reference II.5 concerns the planning or repair steps in subsequent cells based on the information about the newly inspected defects. Number II.6 designates the robot program generation.

[0071] Module III in Fig. 3 represents a machining cell. Reference symbol III.1 designates the component to be machined. Reference symbol III.2 designates the robots for machining and re-inspection with increased precision, optionally with additional axes. Reference symbol III.3 designates the tools mounted on the robots for machining the defects. Reference symbol III.4 designates the sensors for re-inspecting (more precisely) the defects to be repaired. Module IV represents another machining cell without robot-mounted inspection sensors. Reference symbol IV.1 shows the component to be machined. Reference symbol IV.2 designates the robots for machining, optionally with additional axes. Reference symbol IV.3 designates the tools for machining the defects found.

[0072] In Error! Reference source could not be found, the data flow between the individual modules is designated with reference symbols II. a to II. d and the reference symbol Ia denotes the conveying direction of the component. The reference symbol II. a denotes the sensor data from the inspection cameras, section II. b the list of defects found, section II. c the plans for re-inspection and adaptive repair for the individual robots, section II. d the resulting robot programs, section II. e the sensor data concerning the newly inspected defect, section II. f the updating of the defect information (position, class, characteristic) and section II. g the updating of the defect information (position, class, characteristic) for the planning of the following repair steps in the subsequent robot cells. The reference symbol II. h denotes the planned robot paths for subsequent repair and point II. i the robot programs determined from them.

[0073] Fig. 4 provides an overview of all the different variants of inline planning for defect inspection and processing presented using a flowchart that visualizes the data / signal flow. Module I represents the preprocessing and loop-by-loop processing of all defects. 1.1 denotes the function for merging neighboring defects, i.e., defects with a distance less than a threshold are combined and treated as one defect if they are located on the same component of a part and on the same side of a feature (e.g., an edge). Function 1.2 assigns the relevant fused defects to the robots (or the robot, if there is only one robot). Functions 1.3.a and 1.3.b represent the sequential processing of all defects.

[0074] Module II represents the actual planning of inspection and adaptive planning of defect processing. Module III represents the subsequent step of defect selection, provided the number of defects exceeds the number the robot(s) can or should process in the given time. Criteria include accessibility, processability, robustness (i.e., the number of defects that can be processed without subsequent inspection), and the proportion of possible changes due to variants that can be successfully processed versus not processed.

[0075] Module IV Plan / Program Linking presents the creation of complete plans / programs from the plans / programs of individual defects using (IV.1) sequence planning, and the planning of the connecting movement (IV.2) of the robots (from defect to defect), as well as the saving of the plans and the program generation (IV.3).

[0076] In Fig. 4, the data flow between the individual modules is labeled with letters (a) to (o). Letter (a) represents the list of all defects found (and assessed as relevant) during the inaccurate inspection, including the defect data (in particular, their position, severity, class, and advantageously also the associated uncertainties regarding position, severity, and class). The defects detected by the inspection sensors are transferred to the repair planning module preparation.

[0077] Letter (b) represents the list of all defects after defect fusion, in which defects (closer than a maximum distance, located on the same component, and not separated by features) can be grouped together. Letter (c) represents the list of all defects per robot. Letter (d) represents the next defect to be planned. Letter (e) represents the determined (sub-)plan for inspecting and processing this current defect, and letter (f) represents the "jump" to the next defect in the list if the list has not yet been processed (software loop). Letter (g) represents the data concerning all (sub-)plans for all processable defects, and letter (h) the data concerning all (sub-)plans after a possible reduction if the number of defects of individual robots exceeds the maximum number of repairable ones.Letter (i) represents the subplans in ideal order, (j) represents the plans as a link between subplans for inspection and processing of defects and the intermediate movements connecting them, and (k) represents the complete plans stored for each robot.

[0078] Fig. 5 shows (Module II, see also Fig. 4) a simple implementation of inline planning for each defect in detail, without explicit inline anticipation of changes in the defect properties during a subsequent re-inspection with a robot-guided sensor. In particular, this module carries out the planning of the re-inspection (Step II.1). Step II.2 refers to the planning of the processing based solely on the result of the first (inaccurate) inspection. In addition, the component in the area around the defect is analyzed, as well as the selection of recipes, adaptation of the recipes, planning of the robot movement and signals. If successful, the sub-plan / sub-program is saved. If the defect is not processable or plannable, it is saved as "unrepairable". Step II.3 refers to the saving of the combined plan for inspection with subsequent processing.In case of failure save that irreparable and remove from planning the entire plan.

[0079] In Fig. 5, the data flow is labeled with the letters (a) to (d). Letter (a) denotes the next defect (position, characteristic, etc.) for which a plan is to be created. Letter (b) denotes a case in which no inspection is feasible, which is why the defect is skipped. Letter (c) denotes the defect and the identified partial plan for its re-inspection with increased accuracy. Letter (d) denotes the defect and the partial plan for its inspection and repair (if plannable). If planning is not possible, the defect is marked with the note “not repairable." Letter (e) denotes the partial plan for the respective defect, compiled for inspection and processing if the planning was successful. Otherwise, the plan consists only of the instruction "Skip defect," and the defect is excluded from further planning.

[0080] In this example, no precise check is performed to determine which deviations from the measured data make the defect still treatable and which ones no longer treatable. Consequently, this example does not offer the possibility of quickly responding to changes in defect data due to a re-inspection. If the defect is repairable but not re-inspection cannot be planned, this leads to additional failures.

[0081] Fig. 6 shows (Module II, see also Fig. 4) the planning of robot planning for reactive processing with anticipation of the reaction to later position deviations from the new inspection, including sampling but without planning of program alternatives / planning programmes.

[0082] In the example shown in Fig. 6, function II.1 (process step) concerns the planning of a robot path for the repair of a defect based on the measurement data from the first, inaccurate inspection. This planning also includes the computer-assisted verification of whether the planned path is achievable and whether the defect can be repaired. Function II.2 concerns the evaluation of the variance determined from the inline inspection and the decision whether the expected variance of the defect is small enough to perform a repair using the plan / program for the original defect (i.e., based on the original measurement data) without further consideration. Function II.3 concerns the planning of the re-inspection of the defect using the more accurate, robot-mounted sensor. Function II.4 concerns the sampling of the surface in an uncertainty region around the detected defect.

[0083] Function II.5 concerns the planning of the subplan for skipping the repair. This anticipates a case in which, based on the subsequent re-inspection, the defect is assessed differently than in the original inspection (e.g., as no longer relevant or not repairable). Function II.6.a denotes the beginning of the loop over all alternative defect positions (samples) determined in II.4, and function II.6.b denotes the end of the loop over all alternative defect positions determined in II.4.

[0084] Within the loop, function II.7 simulates the repair of the alternative defect (sample) using the original, unchanged program and checks whether the alternative defect can also be processed satisfactorily (“OK”). If this is the case, the respective sample is assigned to “Group 1” (Group 1 - “unchanged”). Function II.8 simulates the repair of the alternative defect (sample) using the original program, shifted by an offset corresponding to the offset between the unchanged defect and the sampled defect. Based on this simulation, it can be checked whether the alternative defect (sample) can be processed “OK.” If this is the case, the respective sample is assigned to “Group 2” (Group 2 - “with offset”). Function II.9 adds the sample to “Group 3” (Group 3 - “replanning required”).In another embodiment, a partial plan can be planned for this sample. Alternatively, group 3 can be interpreted as "do not repair." Function II.10 concerns saving the determined plans and the associated samples.

[0085] In Fig. 6, the data flow is shown with the letters (a) to (r): (a) denotes the original (inaccurate) measurement data (position, characteristic, etc.) concerning a defect, (b) denotes the measurement data for a defect that is not repairable or for which path planning is not possible (c) denotes the forwarding of the description of the defect in an unchanged position, (d) denotes the case in which no adaptation of the repair program / plan is required for a defect and no sampling of the area around the defect is required.(e) denotes a defect with a plan without adaptation, (f) denotes the case in which the defect cannot be inspected online, (g) denotes a defect including a partial plan / program for its online inspection, (h) denotes an unchanged defect with a partial plan / program for its processing and sampling around the defect according to its determined variance, (i) denotes a partial plan for repairing the unchanged defect and for skipping its processing, (j) denotes the unchanged defect and the samples / sampling of the position around the defect to be currently checked, (k) denotes a case distinction, namely the case in which the processing of the sample with the original program without offset iO is possible.(1) denotes the unchanged defect and the samples currently being tested / sampling the position around the defect. (m) denotes, in the case distinction, the case in which processing the sample with the original program plus offset is possible. (n) denotes the unchanged defect and the samples currently being tested / sampling the position around the defect. (o) denotes the unchanged defect and the samples currently being tested. Samples / sampling of the position around the defect, (p) denotes the jump back to the next sample, (q) denotes the plan / program for the unchanged defect, samples assigned to groups (1 unchanged, 2 "with offset", 3 "replanning required"), (r) denotes the sub-plan / program for repairing the unchanged defect and for skipping it, as well as grouping of the samples, or reference for skipping the defect.

[0086] The illustrated embodiment allows for anticipating changes in the inspection result with respect to position or assessment (whether a defect is repairable or repairable), saving time during the actual execution of the repairs. Furthermore, safety is increased with regard to collisions or other problems. The increase in the required computing power for planning is moderate. Changes in the nature of the defects that require a different type of repair are not anticipated and must be replanned (if they occur later online), or the repair must be skipped.

[0087] Fig. 7 shows the planning (Module II) of reactive machining with anticipation of the reaction to subsequent position deviations and changes in the classification or determined severity of the defect during the re-inspection (including sampling and parallel planning of program alternatives / plan alternatives for each sample). In Fig. 7, we denote: Function II.1) evaluating the determined variance from the inline inspection, deciding whether the expected variance of the defect is small enough to perform a repair using the plan / program for the original defect without further consideration. Function II.2) planning the re-inspection using the robot-mounted sensor for the defect. Checking accessibility / inspectability. Function II.3) “scanning” the surface around the defect (samples) in the uncertainty range in the position and in the uncertainty in the class and severity of the defect. Function II.4) planning the sub-plan for skipping the repair.This anticipates the case that the defect is different in the subsequent re-inspection than determined in the original inspection (e.g. is classified as no longer relevant or not repairable). Function II.5 a) Start of the loop over all alternative defects (samples) determined in II.3. Function II.6) Re-planning of the repair for the current alternative defect (sample) according to varied position, class, and characteristic. Checking of the execution and, if necessary, marking the sample as not repairable. Function II.5 b) End of the loop over all alternative defect positions, classes, and characteristics (samples) determined. Function II.7) Evaluation of whether. The number of samples for which the defect can be repaired according to the plan is greater than a minimum value, or the percentage of all samples is above a percentage threshold. Function II.8 post-processing, namely checking whether the plans for all samples are similar in terms of recipe, tool path, robot configuration, and robot movement. In this case, these are combined if necessary, i.e., marked, so that when merging the partial plans, the "online re-inspection" is skipped and the original partial plan / part program is used to repair the defect without prior inspection.

[0088] In the example shown in Fig. 7, the data flow is represented by the letters a) to m). Here, denote: a) the defect to be planned, b) the variance of the defect as measured (minor or no adaptation of the repair program / plan required, no sampling of the range of variance required), c) the forwarding description of the defect in an unchanged position, d) the case where the defect cannot be inspected, e) a defect with a partial plan / program for re-inspection, f) a defect unchanged with a partial plan / program for its processing and sampling around the defect according to its determined variance in position, class, and severity, g) the current sample to be planned, i) the partial plan for the current sample, j) the testing of further samples, k) partial plans for all samples orsamples assessed as “non-repairable”, 1) the sub-plan for the defect composed of re-inspection and all alternative plans for all samples, unchanged defect and current sample to be inspected / sampling of the position around the defect, m) the sub-plan / program for inspection and alternative sub-plans for repairing the sample of the defect.

[0089] Fig. 8 illustrates online adaptation, i.e., reactive replanning. The following overview includes all versions listed in Figs. 9-12.

[0090] Module I shown in Fig. 8 concerns the sequence control. This includes the nested, loop-like processing across all plans / programs of a robot and the loop across all defects within these programs / plans. Function Lia): Processing of all plans / programs per robot. Start of the loop. Function 1.2): Start of execution of the current plan / program. Function 1.3 a): Processing of all defects per current plan / program. Start of the loop. Function 1.4): Execution of the partial plan and approaching the inspection position before / above the defect. Function 1.3b): Processing of all defects per current plan / program. End of the loop. Function 1.5): Execution of the last partial plan and approaching the end position of the program / plan. Function 1.lb): Processing of all Plans / programs per robot. End of the loop. Module II: Contains the reactive processing of each defect. Function II.1) Execution of the partial plan / program for inspecting the defect with the robot-mounted sensor, evaluation of the sensor data, and determination of the inspection result. Function II.2) Reactive execution of the processing of the newly measured defect with the robot.

[0091] In an alternative implementation, each defect can be re-inspected after processing with the robot-mounted inspection sensor and, if necessary, re-processed.

[0092] In Fig. 9, the data flow is represented by the letters a) to m). These denote: a) all plans / programs per robot, b) the next plan / robot program, c) the start of program execution, d) the next defect, e) reaching the next defect, f) skipping processing in the event that a deviation of the newly inspected defect from the original inspection is above a threshold, g) updating the inspection result in the event that a change is within the permitted range, h) execution of defect processing, i) jump label, case: further defects in the plan / program, j) all defects in the plan / program case processed, k) end point / position in the program reached, 1) case: further plans / programs for the robot, m) all plans / programs for the robot processed.

[0093] Module II, shown in Fig. 9 d, illustrates a simpler implementation of reactive processing of a defect, using only a measured offset. Function II.1) refers to the approach to the inspection point and execution of the defect inspection - determination of the new position / offset. In an advantageous implementation, initiation of skipping if the change in the defect position exceeds a predefined fixed limit. Function II.2) The untested adaptation of the zero program: Applying an offset to the program / simple coordinate transformation as a state-of-the-art function of a robot controller. Function II.3) Program execution, advantageously with collision monitoring of the robot as a state-of-the-art function of a robot controller. Function II.4) Function for documenting the execution of the processing.

[0094] In Fig. 9, the data flow is represented by the letters a) to f): where a) indicates the defect to be repaired, b) the case where the measured offset is greater than a threshold, c) update of the defect information for the defect that can be processed according to the new inspection: only the defect position, d) robot program with the offset applied, e) Information that defect is being processed f) Message about defect processing with status (cancelled or repaired).

[0095] Module II, shown in Fig. 10, illustrates the reactive processing of a defect as part of a program depending on the evaluation of the groups with a zero program, a zero program with a measured offset, or a new plan. Function II.1) Query whether a new inspection is required according to previous inline planning. (If yes) Function II.2) Approach the inspection point and carry out the inspection of the defect. II.3) Function for re-evaluating the defect class, if necessary initiating a skip of processing if the defect cannot be processed as re-measured or cannot be processed, or if the measured offset lies above a defined or determined limit. Function II.4) Evaluation of whether the defect is close to a value in group 1 and can therefore be repaired with a zero program. This evaluation can be accessed by calling the planning software from the robot, or the comparison is built into the robot program during program generation. Function II.5) Evaluation of whether the defect is close to a Group 2 value and can therefore be repaired with a zero program with a pure offset. Function II.6) Replanning of defect processing with measured new defect position, class, and severity, for defects close to a Group 3 value. Function II.7) Function: Execution of the machining program for all 3 cases / groups (1) zero program, (2) zero program with offset, (3) newly planned machining program. If the offset is too large, then abort. II.8) Function for documenting the execution or aborting / skipping of machining (reasons, target parameters versus actual parameters, etc.).

[0096] In Fig. 10, the data flow is represented by letters a) to m). Here, a) denotes the defect to be repaired, and b) denotes the case where, according to previous inline planning, no re-inspection is required.In this part we continue directly with the zero program, c) information about the next defect (in case a new measurement is necessary), d) the data of the new inspection, update of the position and characteristic / type of the defect, e) the case that the defect does not have to be or cannot be processed, or the measured offset is greater than a threshold value, f) update of the defect information for the defect that can be processed according to the new inspection: position, characteristic, class, g) the case that the change is minor enough to work with the zero program (original program applicable), h) the passing on of the update of the defect information in the case that the original program cannot be applied unchanged, i) the case that the change lies in the group that can be repaired by the zero program (original program) with a simple offset according to the previous inline planning. j) Passing on the update of the defect information for replanning, k) The newly planned plan / program for the adapted processing of the defect. 1) Information that the defect is being processed, with parameters, m) Notification of defect processing with status (aborted, repaired) with parameters.

[0097] Module II shown in Fig. 11 illustrates the reactive processing of a defect as part of a program depending on the comparison between the inline determined sample and the newly inspected defect (one plan per sample of the defect). The plan is processed from the set of all inline (parallel) planned plan variants whose sample is most similar to the newly measured defect according to its characteristic, class and position. In Module II: Function II.1) the query as to whether a new inspection is necessary according to previous inline planning. If yes, Function II.2) the approach to the inspection point and the execution of the inspection of the defect. Function II.3) the function for re-evaluating the defect class, if necessary initiating a skipping of processing if the defect as newly measured cannot be processed or cannot be processed due to its characteristic and parameters (not position).In an alternative embodiment, the decision as to whether a defect cannot be processed can be derived from the fact that no similar sample was tested in the inline planning. Function II.4) refers to the determination of the sample that is most similar to the defect in terms of position, class and characteristic. This evaluation can be achieved by calling the planning software from the robot, or the comparison is built into the robot program during program generation. Function II.5) refers to the assignment of the plan or program planned inline for the selected sample for processing or for skipping processing. Function II.6) refers to the function of executing the processing program or the program section for skipping processing. II.7) refers to the function for documenting the execution or aborting / skipped processing (reasons, target parameters versus actual parameters, etc.).

[0098] In Fig. 11, the data flow is represented by the letters a) to 1). Here, a) denotes the defect to be repaired, b) denotes the case in which, according to the previous inline planning, no re-inspection is required. In this case, the program continues directly with the zero program, c) denotes the information about the next defect - in case re-measurement is required, d) denotes the data of the re-inspection, update position and defect type, e) denotes the case in which the defect does not need to be or cannot be processed, f) denotes the update of the defect information for the defect that can be processed according to the re-inspection. Defect: position, characteristic, class, g) in case no similar sample is found - skipping the defect - alternatively, replanning would also be possible as shown in Fig. 12, h) Assigned sample with update of the defect information i) Assigned plan / program, j) Information that defect is being processed, with parameters, m) Message about defect processing with status (aborted, repaired) with parameters.

[0099] In Fig. 13 the entire process is summarized in a flow chart.

Claims

CLAIMS 1. Procedure comprising: Carrying out a first optical inspection of a workpiece using one or more first sensors; Detecting defects on the surface of the workpiece and determining initial defect data characterising the detected defects based on measurement data measured during the initial optical inspection, Carrying out - for at least one of the detected defects and based on the associated first defect data - an automatic, computer-aided first robot path planning to determine a robot path for a second inspection of the workpiece in the area around the at least one defect and to determine one or more robot paths for processing the at least one defect; Performing the second optical inspection of the workpiece by means of one or more robot-guided second sensors and determining second defect data characterizing the at least one defect based on measurement data measured during the second optical inspection; Checking - based on the second defect data - whether the at least one defect can be processed with a first of the planned robot paths or - if available - another of the planned robot paths, and, if so, automatic, robot-assisted processing of the at least one defect using the respective robot path; if the at least one defect cannot be processed with one of the planned robot paths: carrying out a second robot path planning to determine a new robot path for processing the at least one defect based on the respective second defect data, and automatic, robot-assisted processing of the at least one defect using the new robot path.

2. The method according to claim 1, wherein performing the second robot path planning comprises: Modify one of the planned robot paths based on the second defect data.

3. The method according to claim 1 or 2, wherein performing the first robot path planning comprises: Planning a robot path based on the first defect data; modifying the first defect data multiple times and, for each modification, planning another robot path based on the modified defect data.

4. The method according to one of claims 1 to 3, wherein the first defect data and the second defect data comprise at least the following: the position of the respective defect on the workpiece surface and data concerning the severity of the defect.

5. The method of claim 4 when dependent on claim 3, wherein modifying the first defect data comprises modifying the position of the respective defect within a specific range around the defect, the specific range depending on the accuracy of the first optical inspection.

6. The method according to claim 5, wherein modifying the first defect data comprises modifying the severity of the respective defect within a certain range that depends on the accuracy of the first optical inspection.

7. The method according to any one of claims 1 to 6, when dependent on claim 3, wherein the testing comprises: Determine whether the second defect data for the respective defect – taking into account a specified tolerance – approximately match the first defect data, whereby, if so, the first of the planned robot paths is used for the subsequent processing of the respective defect; or Determine whether, for the respective defect, the second defect data - taking into account the specified tolerance - approximately correspond to certain modified defect data, whereby, if so, the planned robot path based on the certain modified defect data is used for the subsequent processing of the respective defect.

Citation Information

Patent Citations

  • Automatic detection and robot-assisted machining of surface defects

    US11110611B2

  • AUTOMATIC DETECTION AND ROBOT-ASSISTED PROCESSING OF SURFACE DEFECTS

    DE102015119240B3

  • Paint defect automated seek and repair assembly and method

    US20030139836A1

  • Learning framework for robotic paint repair

    US20210323167A1

Cited By

  • Multi-task planning method and system for mechanical arm

    CN121290433A