Robot path correction method

Abstract

The invention relates to a method for detecting, preferably statically, the shape and / or position of a workpiece for robot path correction, the method comprising the following steps: a) generating at least one reference point cloud from a reference data source; b) generating at least one reference feature from the at least one reference point cloud; c) generating at least one actual point cloud of the workpiece; d) generating at least one actual feature from the at least one actual point cloud; e) determining at least one deviation between the at least one reference feature and the at least one actual feature; f) determining correction data from the at least one deviation between the at least one reference feature and the at least one actual feature; and g) transmitting the correction data to a robot system for adapting the robot path.

Description

[0001] Description of robot path correction method

[0002] Technical field

[0003] The present invention relates to a method for, preferably static, shape and / or position detection of a workpiece for robot path correction, a device for, preferably static, shape and / or position detection of workpieces for robot path correction, a system for robot path correction and a computer program for carrying out a method for, preferably static, shape and / or position detection of a workpiece for robot path correction.

[0004] background

[0005] In modern manufacturing, particularly in the automotive industry, a high degree of automation is sought to maximize quality, speed, and cost-effectiveness. Industrial robots play a central role in this, as their programmed motion paths offer high repeatability and efficiency. However, challenges arise from geometric deviations and positioning inaccuracies of the workpieces.

[0006] For example, modern automotive production demands the highest precision and efficiency, especially in the application of sealing materials such as PVC to body panels. The current state of the art includes automated robotic cells that apply PVC seams to car bodies. The position of the car body within the cell varies with a certain degree of uncertainty, requiring precise measurement and correction methods to ensure a uniform and flawless application.

[0007] Basically, two different robot path guidance methods can be distinguished for such applications: online and offline robot path guidance.

[0008] The main difference between these two types of processes lies in the way in which the position and geometry of the workpiece are detected and corrected.

[0009] With online robot path guidance, the workpiece's position and geometry are captured in real time during machining. Correction values ​​are calculated and applied immediately during machining. This method allows for real-time adjustments and immediate responses to unforeseen deviations. However, implementation is more complex and expensive and can lead to delays if the real-time calculations are not fast enough. Furthermore, while this type of correction is necessarily performed with some lead time, it may still not be possible to react in time to an unforeseen narrowing, an unexpected obstacle, or similar issues. This results in a significantly increased risk of crashes.

[0010] DE 102014104031 B4, for example, discloses a method for online path guidance for a robot and the monitoring of an application structure as well as a sensor for carrying out these methods.

[0011] In contrast, offline robot path guidance captures the workpiece's position and geometry before machining begins. Correction values ​​are calculated in advance and integrated into the robot path. This has the advantage of eliminating interruptions during machining and allowing corrections to be planned beforehand, resulting in high precision and smooth trajectories. However, this method requires an additional robot run to capture the workpiece's current state before the machining or application run. This can be particularly time-consuming when numerous measurements are required and / or complex geometries need to be traversed.

[0012] Both methods have their specific areas of application and advantages, depending on the requirements of production and the available technical possibilities.

[0013] Offline robot path guidance is preferred because the online methods currently have significant disadvantages, as shown.

[0014] Well-known offline methods include laser light sectioning and strip light sectioning for the sensor-based detection of workpieces. The laser light sectioning method offers high measurement accuracy and flexibility because the sensor is mounted on the robot end effector. However, the need for measurement runs leads to a significant loss of cycle time, as shown.

[0015] In the stripe-type light sectioning method, several parallel lines are projected onto the workpiece and their reflections are evaluated. Although several lines can be evaluated simultaneously, measurement runs are necessary, leading to a comparable cycle time loss as with the laser light sectioning method.

[0016] Document DE 102005051 533 B4 relates to a method for improving the positioning accuracy of a manipulator, in particular a robot, with respect to a structure or contour to be processed by the manipulator, such as edges, folds, welds or adhesive beads, of a serially produced workpiece. The aim is to improve the positioning accuracy of the manipulator and simultaneously increase the speed when traversing the path edge, without compromising the control system in terms of compensating for the positional accuracy.

[0017] However, the method has some disadvantages. It requires the use of a manipulator- or robot-guided sensor. Furthermore, at least one point or reference structure must be selected and determined on a reference workpiece before the method is applied. The reference points must be sensor-acquired before machining a workpiece, which negatively impacts the overall process time, consisting of the measurement run and machining of the production part. In addition, the points selected for the method, both for the reference and production parts, cannot be modified externally (offline).

[0018] Summary description of the invention

[0019] Therefore, one of the objectives of the present invention is to overcome the disadvantages of known prior art methods for robot path correction. In particular, one of the objectives of the present invention is to provide offline robot path correction methods that allow immediate correction of the robot path and eliminate the need for a measurement run.

[0020] These and other problems are solved by the subject matter of the attached independent claims.

[0021] Preferred embodiments can be found in the dependent claims and furthermore in the following description, in particular taking into account various embodiments as discussed and described in the attached claims.

[0022] The embodiments, features, and combinations of features described herein in connection with the invention, as well as the combination of features specified in the appended claims, and also any combination of features mentioned and described in connection with the embodiments, are deemed to be disclosed herein, or at least to be derivable by a person skilled in the art. In particular, each feature and each combination of features in the embodiments described herein may, for example, be claimed in a different combination, especially in a different claim category, at least because the person skilled in the art will recognize that each individual combination of the features mentioned herein is suitable for contributing to the solution of the underlying problem.

[0023] Furthermore, each feature and each combination of features in the claims and in the description below can be used and claimed separately, independently of the subject matter claimed, independent of claim dependencies and cross-references, and independent of the claim category in which the feature is claimed. For example, in an arbitrary combination selected from one or more claims, one or more embodiments according to the description below and / or the accompanying figures may be provided.

[0024] The above-described problems are solved according to the invention by a method for, preferably static, shape and / or position detection of a workpiece for robot path correction, comprising the steps:

[0025] a) Generating at least one reference point cloud from a reference data source,

[0026] b) Generating at least one reference feature from the at least one reference point cloud;

[0027] c) Generating at least one actual point cloud of the workpiece;

[0028] d) Generating at least one actual characteristic from the at least one actual point cloud;

[0029] e) Determining at least one deviation between at least one reference characteristic and at least one actual characteristic;

[0030] f) Determining correction data from the at least one deviation between the at least one reference characteristic and the at least one actual characteristic; g) Transferring the correction data to a robot system to adjust the robot path.

[0031] It should also be noted in connection with the method according to the invention that the steps given do not necessarily have to be carried out in the specified order. The steps given can be carried out in any other suitable order or even simultaneously.

[0032] However, the sequence given above may be advantageous for certain embodiments of the method according to the invention.

[0033] At the same time, for example, step b) of generating at least one reference feature from the at least one reference point cloud can be carried out after step c) of generating at least one actual point cloud of the workpiece, or even simultaneously.

[0034] At the heart of the invention, the method relates to the automatic correction of a pre-programmed robot path, for example, for applying a sealant and / or adhesive, particularly PVC, to a car body part. A set of actual data is acquired from an actual object by generating at least a set of three-dimensional points. This actual data is compared with a set of reference data from the object, which also represents at least a set of three-dimensional points. The reference and actual data can be geometrically aligned in such a way that a reference-actual comparison is possible, thus enabling precise robot path correction. The reference-actual comparison is based on features generated from the point sets. These features yield comparative values ​​that are used to correct the pre-programmed robot path.The newly determined robot path can then be used to process the actual object and / or apply a medium to the actual object using a robot.

[0035] Within the scope of the present invention, the term “workpiece” refers to an object that is processed in a manufacturing process. In this context, the term preferably refers to the workpiece currently being processed, which is located, for example, in the production cell, and whose shape and position are recorded in order to correct the robot path accordingly. It is therefore preferably the specific component that is currently the focus of processing and for which the actual data are recorded and compared with the reference data to ensure precise processing or application.

[0036] In a preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, a workpiece is preferably a body part in the automotive sector, preferably selected from the group comprising door, headlight housing, hood, fender, trunk lid, roof frame, side sill, bumper, wheel arch, A- and B-pillar, tailgate, cross member, underbody, front apron and side panel.

[0037] These workpieces are typical components that are machined and assembled in automotive production, and their precise positioning and shape are crucial for the quality and fit of the final product.

[0038] In the inventive method, at least one reference point cloud is generated from a reference data source in step a). The term "reference point cloud," as used herein, preferably means a three-dimensional point cloud of an ideal workpiece, which serves as a reference and is obtained from a specific data source. This reference point cloud represents the reference data of the ideal workpiece, i.e., the ideal geometric data that serve as a basis for comparison with the actual data of the currently machined workpiece.

[0039] The reference data source can take various forms, such as CAD models, geometric models, or other digital representations of the workpiece. This reference point cloud is crucial for later determining the deviations between the reference and actual data and correcting the robot path accordingly.

[0040] A reference point cloud can also be generated, for example, from the digitization of a template object, in particular a workpiece ideal, i.e., a template for the workpiece in its most ideal state. According to the invention, a single reference point cloud can be generated from the reference data source. Alternatively, multiple reference point clouds can be generated. A single reference data source can be used, or several reference data sources can be combined into a single overall reference data source.

[0041] The reference point clouds can be combined to form a complete reference point cloud or processed individually to generate at least one reference feature in each case.

[0042] In embodiments where multiple reference point clouds are generated, these can be generated using the same or different methods. It is not necessary for the reference point clouds to be sorted.

[0043] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, the reference data source comprises design data of an ideal workpiece and / or measurement data of at least one ideal workpiece.

[0044] In particular, design data of the workpiece, or more precisely, the ideal workpiece, such as a CAD model, a geometric model, or other digital representations of the workpiece, can be used as a reference data source. Additionally or alternatively, the reference data source includes measurement data of at least one ideal workpiece. Therefore, the reference data source can comprise, on the one hand, digital design data, such as a CAD model, a geometric model, or other digital representations of the workpiece, and / or, on the other hand, measurement data of one or more real workpieces recorded with appropriate sensors.It should be understood that the measurement of one or more workpieces as a reference data source preferably takes place on the most ideal workpieces possible, which then serve as the workpiece ideal, and also includes the possibility of recording several most ideal workpieces and using them together and / or with design data to generate a workpiece ideal.

[0045] In a preferred embodiment, design data of the workpiece and measurement data of the workpiece or workpiece ideal are combined to generate the reference point cloud. Those skilled in the art will recognize that values, e.g., sections, section bundles, or volume segments from virtual design data behave essentially logically like values, e.g., sections, section bundles, or volume segments from measurement data, acquired with a sensor, e.g., a line sensor. It is therefore possible to algorithmically combine values ​​from measurement data with virtual values ​​from design data.

[0046] In the inventive method, at least one actual point cloud of the workpiece is generated in step c). The term “actual point cloud,” as used herein, preferably refers to the three-dimensional digital representation of the current workpiece. This point cloud is generated by digitizing the workpiece and consists of a multitude of metric points that depict the actual shape and position of the current workpiece. The actual point cloud serves as the basis for comparison with the reference point cloud in order to determine deviations and calculate correction data for adjusting the robot path.

[0047] Like the reference point cloud, the actual point cloud can be composed of individual measurement points to form a planar point cloud.

[0048] In this context, it should also be understood that a reference point cloud and / or actual point cloud is created by combining the various points detected by a sensor into a planar point cloud. It should be understood that the points preferably have a planar distributed extent. The term "planar distributed" as used herein preferably means that the points are preferably not corrected to a single point or line, but are distributed planarly across the surface of a partial area or the entire surface of the current workpiece. A planar point cloud is preferably a planar extended point cloud. However, it should be understood that a volumetric point cloud is also a planar point cloud within the meaning of the present invention and can be used.The expert will immediately understand that this requires the material to be penetrated, at least partially, for example with an X-ray setup.

[0049] Preferably, a single actual point cloud is generated based on the current workpiece. Alternatively, multiple actual point clouds can be generated, particularly through repeated scanning with the sensor and / or the use of several identical or different sensors. For example, multiple scans of the current workpiece can be used as the data source for the actual point clouds, or data acquired with different sensors can be used, with the data then being combined into a single actual point cloud or processed individually to generate at least one actual feature.

[0050] If multiple actual point clouds are to be generated, they do not necessarily have to be generated using the same method or sensors. Furthermore, the sensors used to generate a reference point cloud using survey data and those used to generate at least one actual point cloud can be the same or different.

[0051] To generate a reference point cloud using survey data, essentially the same sensors can be used as in step c) when generating at least one actual point cloud of the workpiece. These sensors are used to generate point clouds of the current workpiece or the ideal workpiece. A person skilled in the art will immediately recognize that different sensors can also be used, as long as they have sufficient resolution. Different sensors would be advantageous, for example, if one wants to acquire the reference point cloud with high-quality sensors to reduce outliers or noise. Conversely, using the same sensors has the advantage that new sensor types can be trained much more easily, since the sensor can be used on-site.

[0052] Various sensors can be used. For example, planar 3D sensors can be employed, such as triangulation-based sensors like stereo cameras or structured light sensors. Other data generation methods can also be used, such as time-of-flight (TOF) sensors or white light interferometry. Similarly, moving line sensors can be used, such as laser sectioning sensors or chromatic confocal line sensors. The acquisition positions must be known in order to subsequently assemble the lines into a planar point cloud. This can be achieved, for example, using a linear axis with an encoder, a calibrated manipulator-guided sensor, or external tracking via a laser tracker or other suitable tracking methods. The laser movement can also be virtual, for example, using a mirror.

[0053] Static line sensors can also be used, particularly when the workpiece to be detected is moving. Examples include laser light section sensors, chromatic confocal line sensors, and all other methods for generating metric line data. The component positions at the time of each line acquisition must be known in order to subsequently assemble the lines into a planar point cloud. This can be achieved, for example, using a linear axis with an encoder, or external tracking via a laser tracker, camera-based tracking, or other suitable tracking methods. A reference point cloud and / or actual point cloud can also be generated by point-like sensors, such as laser radar, time-of-flight (TOF) distance sensors, tactile scanning, and all other suitable methods for acquiring metric point data.

[0054] A person skilled in the art will readily recognize that the individual points of a reference point cloud and an actual point cloud do not have to be identical. Rather, it is necessary that the areas of the workpiece or ideal workpiece relevant to the robot path are adequately covered. Likewise, it is not detrimental if the reference point cloud and / or the actual point cloud contain additional points.

[0055] In the inventive method, in step b) at least one reference feature is generated from the at least one reference point cloud, and in step d) at least one actual feature is generated from the at least one actual point cloud. This offers the advantage that precise and efficient comparisons between the reference and actual state of the workpiece or workpiece ideal with regard to its shape and / or position in space can be carried out, which enables an accurate evaluation and analysis of deviations.

[0056] The automated generation of reference or actual features can be achieved in various ways. First, values ​​such as intersection lines, intersection bundles, or volume segments can be preprocessed, as in the state-of-the-art path correction method. This includes, for example, the removal of disturbance points or the smoothing of the data. Circular probing can be performed, edges extracted, or maxima and minima determined. The detection of specific features, for example using the Iterative Closest Point (ICP) algorithm or machine learning methods, is also possible. For improved performance and simpler implementation, feature extraction is preferably performed in the (virtual) sensor coordinate system and not at the spatial position of the point cloud. If one of the approaches in the selection object coordinate system was used for the membership check, the data is advantageously already available in this coordinate system.

[0057] After extracting the features, they can optionally be transformed into space, i.e., to the corresponding position in the point cloud, using the selection object coordinate system. For path correction features, this corresponds, according to the state of the art, to the application of the TCP coordinate system.

[0058] In the inventive method, at least one deviation between the at least one reference feature and the at least one actual feature is determined in step e).

[0059] This can be achieved by directly comparing the two sets of characteristics, i.e., the at least one reference characteristic and the at least one actual characteristic. Here, the positions, shapes, and other relevant properties of the at least one reference characteristic are compared with the corresponding at least one actual characteristic. The deviation(s) can manifest in various forms, such as positional shifts, size differences, or shape deviations. These differences are quantitatively recorded and analyzed to determine whether the at least one actual characteristic is within acceptable tolerances or whether corrective action is required.

[0060] Based on this, in the inventive method, correction data are then determined in step f) from the at least one deviation between the at least one reference feature and the at least one actual feature.

[0061] The generation of correction data can be carried out similarly to path correction, according to the state of the art. For relative correction, comparisons between the reference and actual characteristics can be performed in the sensor coordinate system, while for absolute correction, comparisons can take place in the spatial coordinate system.

[0062] In the inventive method, in step g) the correction data are transferred to a robot system to adjust the robot path.

[0063] This means that the generated correction data, after being generated in step f), is transferred to the robot system and applied there. This can be done, for example, by a superimposed real-time control system in the robot controller, which allows the transfer of correction data within each interpolation cycle. Alternatively, the correction data can be transferred via a suitable bus system and applied there using coordinate system corrections. Based on the known geometric relationships, the calculated deviations can be transformed into suitable coordinate systems and sent to the robot system as corrections. The robot applies these corrections and thus corrects its path.

[0064] The problems described above are also solved according to the invention by a device according to the invention for, preferably static, shape and / or position detection of workpieces for robot path correction, preferably for a method according to the present invention. The device according to the invention initially comprises a sensor for detecting an actual point cloud of a workpiece.

[0065] The sensor is preferably an optical sensor, in particular selected from a planar 3D sensor, e.g. selected from a stereo camera, structured light sensor, time-of-flight (TOF) sensor; a moving line sensor, e.g. selected from a laser light section sensor, chromatic confocal line sensor; a static line sensor with a moving workpiece, e.g. selected from a laser light section sensor, chromatic confocal line sensor.

[0066] The device is designed to generate the actual point cloud of a workpiece, preferably in the form of three-dimensional points.

[0067] The device further comprises a processing unit for generating at least one actual feature from the at least one actual point cloud and / or for determining one or more deviations between the at least one reference feature and the at least one actual feature and for determining correction data from the at least one deviation between the at least one reference feature and the at least one actual feature.

[0068] The device further comprises a database or storage for storing the reference point cloud and / or reference data source and / or at least one reference feature. A database or storage for reference point clouds, reference data sources, and reference features can be designed in various ways. In a local environment, storage on a local hard drive in a suitable point cloud format could be sufficient. Additionally or alternatively, a relational database such as MySQL or PostgreSQL can be used to store the data in a structured manner. These databases offer the ability to efficiently manage large amounts of data and execute queries quickly. Alternatively, NoSQL databases such as MongoDB or Cassandra could be used, which are particularly well-suited for storing unstructured or semi-structured data, as is common in point clouds.

[0069] For storing large amounts of data, such as those generated by point clouds, specialized storage solutions like Hadoop or Apache Spark are preferred, as they are designed to process and analyze large amounts of data, thus offering high scalability and flexibility.

[0070] External data storage solutions, such as cloud storage, also offer a way to store and manage data, particularly for creating location-independent backups. A key advantage of cloud solutions is the ability to migrate part of the data analysis and processing, and thus potentially part of the processing unit, to the cloud. The expert will select the appropriate storage solution based on the specific requirements of the application, such as the required data volume, access speed, and security standards.

[0071] The device further comprises an interface for transmitting correction data to a robot system for adjusting the robot path. According to the present invention, the transmission of correction data to the robot system preferably occurs in real time. It is possible, but not mandatory, to respond within one IPO cycle. Alternatively, the interface can be configured as a simple bus transmission interface.

[0072] Such an interface for transmitting correction data to a robot system can be implemented in various ways. For example, using Ethernet / IP, wireless communication (especially Wi-Fi, Bluetooth, UHF radio, or similar technologies), or XML interfaces. Experts are familiar with numerous interfaces and their advantages and disadvantages, and will select the appropriate one based on the specific application requirements, such as data rate, distance, environment, and cost.

[0073] Advantageously, the device according to the invention for carrying out the method according to the invention is integrated into an overall system with a robot.

[0074] The tasks described above are also solved according to the invention by a robot path correction system. The robot path correction system according to the invention therefore comprises a device according to the present invention and a robot system that receives the correction data and adjusts the robot path accordingly.

[0075] The system is preferably configured to carry out a method according to the present invention.

[0076] In order for the inventive method to be carried out in an inventive device and / or an inventive system, it is advantageous to provide the individual process and calculation steps as a computer program.

[0077] The problems described above are also solved according to the invention by a computer program for carrying out a method, preferably a method according to the present invention, for the preferably static shape and / or position detection of a workpiece for robot path correction. The program according to the invention receives an actual point cloud of a workpiece from an optical sensor and generates at least one actual feature from the at least one actual point cloud; and / or extracts at least one reference feature from the at least one reference point cloud; and / or determines at least one deviation between the at least one actual feature and the at least one reference feature; and / or determines correction data from the at least one deviation between the at least one reference feature and the at least one actual feature; and / or transmits correction data to a robot system for adjusting the robot path.

[0078] The computer program according to the invention is preferably configured to control a device according to the present invention and / or a system according to the present invention.

[0079] It will also be recognized by a person skilled in the art that a feature, embodiment, effect or advantage as described here in connection with the inventive method and / or the inventive device and / or the inventive system and / or the inventive computer program may also be a feature, embodiment, effect or advantage of the inventive device and / or the inventive system and / or the inventive computer program and / or the inventive method, or vice versa.

[0080] The terms “a”, “an”, and “that”, and similar references used in the context of the description of the invention (particularly in the context of the claims) are to be interpreted as covering both the singular and the plural, unless otherwise specified herein or clearly contradicted by the context. The indication of value ranges serves only as a shorthand to avoid having to refer to each individual value within the range. Unless otherwise specified herein, each individual value is included in the specification as if it were listed individually.Within the present application, terms such as “side” or “lateral”, “rear”, “front”, “top”, “bottom”, “ground”, “opposite”, “inside”, “outside” or the like, which describe the position of a first object relative to another object, preferably refer to the relative position of each respective part or object in relation to its position when it is fully assembled for its intended use.

[0081] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, in step b) generating at least one reference feature from the at least one reference point cloud, a generation of virtual reference values, such as reference section lines, reference section bundles, or reference volume sections, and preferably of the at least one reference feature from the reference values, is included.

[0082] The term "reference value," as used herein, preferably refers to a value or group of values ​​that serve as a benchmark. These values ​​are generated from a reference point cloud and can take various forms, such as reference sections, which represent virtual cross-sections through the reference point cloud and depict specific features or contours of the workpiece. Furthermore, reference section bundles, i.e., groups of reference sections, can provide a more detailed representation of the workpiece and, in particular, help to average out defects. Similarly, reference volume sections, which represent virtual sections of a volume within the reference point cloud, can depict specific areas of the workpiece in three dimensions.

[0083] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, step d) of generating at least one actual feature from the at least one actual point cloud includes generating virtual actual values ​​and preferably actual features from the actual values.

[0084] The term "actual value" or "actual value," as used herein, preferably refers to a value or group of values ​​generated from an actual point cloud of the workpiece. These actual values ​​represent the actual features or contours of the workpiece in its current state. They can take various forms, such as actual sections, which represent virtual cross-sections through the actual point cloud and depict specific features of the workpiece. Similarly, actual section bundles, i.e., groups of actual sections, can provide a more detailed representation of the workpiece. Furthermore, actual volume sections, which represent virtual sections of a volume within the actual point cloud, can depict specific areas of the workpiece in three dimensions.

[0085] The use of reference values ​​and / or virtual actual values ​​allows for a more precise determination of characteristics and improves the agreement between reference and actual characteristics. Flexibility is increased because different section planes can be easily adjusted. Furthermore, it saves time and resources, as no additional physical measurements are required. The amount of data is reduced, which facilitates analysis and leads to faster calculations.

[0086] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, step e) of determining the at least one deviation between the at least one reference feature and the at least one actual feature includes determining at least one deviation between actual features generated from the actual values ​​and the at least one reference feature generated from the reference values.

[0087] This is particularly advantageous because determining at least one deviation between the actual features generated from the actual values ​​and the at least one reference feature generated from the reference values ​​enables higher precision and accuracy. By using specific section planes, the relevant features can be captured and compared more accurately, leading to better agreement and more precise correction of the robot path. Furthermore, the amount of data can be advantageously reduced, which facilitates and accelerates the analysis and processing of the point clouds. Those skilled in the art will immediately recognize that multiple reference and actual values ​​can be generated, and that multiple reference or actual features can optionally be derived from each value, in particular a section, section bundle, or volume segment. A unique assignment to a reference feature is advantageous for each actual feature.This mapping is therefore preferably bijective. Based on this bijective mapping, a deviation can be determined for each virtual measurement point. Optionally, path correction sections generated using a robot-guided line sensor can also be incorporated into the feature generation to supplement the point cloud data.

[0088] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, step c) of generating at least one actual point cloud of the workpiece comprises capturing the actual point cloud by means of a sensor, wherein the sensor is preferably an optical sensor, preferably a 3D sensor. Particularly preferably, a planar 3D sensor, in particular a stationary planar 3D sensor, as well as a stereo approach or pattern projection, can be used. A person skilled in the art will understand and acknowledge that when using a light section or laser light section sensor, as used in prior art path correction methods, relative movement between the component and the sensor can also be employed to generate a planar extended point cloud.In contrast, the classic path correction method typically only generates and evaluates a single line.

[0089] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, the sensor is a stationary, planar 3D point cloud sensor.

[0090] This is particularly advantageous because a stationary sensor offers a stable and fixed position, ensuring consistent and repeatable measurements. The fixed installation of the sensor increases the accuracy of data acquisition, as no movements or vibrations can affect the measurements. Furthermore, a stationary sensor enables regular monitoring and recording of the actual point cloud, which is especially beneficial in automated production processes. The fixed positioning also simplifies sensor calibration and maintenance, improving the system's reliability and longevity. The main advantage of a stationary, area-based 3D point cloud sensor, compared to a light section or laser light section sensor, is that no robot-guided measurement is required, resulting in significant time savings.

[0091] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, the sensor is a robot-guided sensor. This sensor can either be a planar 3D point cloud sensor that is maneuvered to various measurement locations and performs stationary measurements at these locations. Alternatively, a light section or laser light section sensor can be continuously moved to generate a planar 3D point cloud.

[0092] This is particularly advantageous because a robot-guided sensor can be flexibly positioned and moved to digitize different areas of the workpiece. This mobility enables more comprehensive capture of complex geometries and hard-to-reach areas that might not be accessible with a stationary sensor. Furthermore, the robot-guided sensor can dynamically respond to changes in the workpiece or its environment, increasing the adaptability and precision of the measurements. Integration into a robot system allows for automatic and precise control of the sensor, further improving the efficiency and accuracy of data acquisition.

[0093] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, the sensor for capturing the actual point cloud is arranged outside the application area of ​​the robot.

[0094] This is particularly advantageous because arranging the sensor outside the robot's application area prevents it from being affected by the robot's movements. This increases measurement accuracy, as no vibrations or disturbances occur due to robot operation. In a further preferred embodiment of the inventive method and / or device and / or system and / or computer program, step c) of generating at least one actual point cloud of the workpiece is achieved by moving a sensor and the workpiece relative to each other, preferably by moving the sensor relative to the workpiece.

[0095] This is particularly advantageous because the relative movement between the sensor and the workpiece enables a more comprehensive and detailed acquisition of the actual point cloud. Moving the sensor relative to the workpiece allows for the capture of different perspectives and areas of the workpiece, resulting in a more precise and complete three-dimensional point set of the actual data. This is especially useful when the actual data is generated by a light section sensor or a laser light section sensor, as these sensors can achieve higher accuracy and resolution of measurements through relative movement.

[0096] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, in step c) of generating at least one actual point cloud of the workpiece, the points of the actual point cloud are distributed over a surface.

[0097] This is particularly advantageous because the area-wide distribution of points in the actual point cloud enables a detailed and comprehensive capture of the workpiece surface. The uniform distribution of measurement points results in higher accuracy and precision in capturing the workpiece geometry. This leads to better agreement between the actual data and the reference data, improving the quality of form and position acquisition. Furthermore, the area-wide distribution of points facilitates the identification and correction of deviations, contributing to more efficient and accurate robot path correction. A key benefit is that a relatively large area of ​​the workpiece can be captured simultaneously, allowing for multiple cuts to be made within it.

[0098] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, the at least one reference feature is generated from the at least one reference point cloud, preferably the reference values, in particular reference sections, by placing a selection object in the reference point cloud, and / or the actual features are generated from the at least one actual point cloud, preferably the actual values, in particular actual sections, by placing a selection object in the actual point cloud. This is particularly advantageous because placing a selection object in the reference point cloud and the actual point cloud enables targeted and precise generation of the sections. This method allows specific points or areas within the point clouds to be selected and analyzed, which increases the accuracy and relevance of the captured features.Furthermore, this approach facilitates the identification and correction of deviations between the reference and actual characteristics, as the selection objects can be specifically focused on relevant areas.

[0099] In a further preferred embodiment of the inventive method, device, system, and computer program, the selection object is placed as a cuboid at the desired location in the point cloud in order to extract virtual sections. The use of other geometries is also possible. The extent of the selection object defines the set of points to be considered, while the orientation of the selection object determines the orientation of the virtual section lines. The cuboid shape has proven to be particularly advantageous in this context.

[0100] Within the scope of the present invention, the change in height is preferably defined in the positive Z direction, such that the line itself runs in the positive X direction and the Y direction is obtained via the cross product of the X and Z directions.

[0101] For individual virtual intersection lines, the Y-component is preferably always 0, since the virtual intersection lines should behave like those in a real line sensor. However, those skilled in the art will immediately understand that other values ​​for the Y-component can be assumed. For each virtual intersection line, the sampling rate in the X-direction can be specified, and the actual points are obtained by sampling the point cloud at the defined location in the defined direction.

[0102] Optionally, multiple intersection lines with adjustable spacing can be generated and combined, with various calculation methods such as averaging, median calculation, or selecting the maximum value. The selection of points to be considered in the point cloud can be done in various ways, for example, by comparing each point with the oriented selection object or by transforming the entire point cloud into the coordinate system of the selection object.

[0103] An axis-parallel cuboid enclosing the selected object can be defined to determine a rough relationship, followed by a transformation of the remaining points into the coordinate system of the selected object. Alternatively, the point cloud can be packaged into a suitable tree structure, such as a kd-tree, to efficiently find nearest neighbors. Multiple virtual intersection lines can also be generated from a single selected object, where the intersection lines must have different Y-values ​​to allow for unique transformations.

[0104] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, step a) of generating at least one reference point cloud from a reference data source comprises at least one preprocessing step. Additionally or alternatively, step c) of generating at least one actual point cloud of the workpiece comprises at least one preprocessing step. The at least one preprocessing step is preferably selected from the group comprising filtering, sampling, interpolation, combining, and cropping the point cloud.

[0105] The term "filtering," as used here, primarily refers to the removal of unwanted data points from the point cloud. This can be achieved by removing outliers, which may have resulted from measurement errors or noise. A filter can also be used to retain only specific areas or features of the point cloud.

[0106] The term "sampling," as used here, preferably refers to the selection of a subset of points from the entire point cloud. This serves to reduce the amount of data and increase processing speed. Sampling can be performed randomly or according to specific criteria to ensure a representative selection of points.

[0107] The term "interpolation," as used here, preferably refers to the process of generating new data points within the range of the existing point cloud. This is preferably done by creating a suitable mathematical model, in particular an nth-order polynomial, spline, or similar, which is then evaluated at the desired location. In the simplest case, linear interpolation can be performed between two points.

[0108] The term "combine," as used herein, primarily refers to the merging of multiple point clouds into a single point cloud. This is necessary when data originates from different measurements or sources and needs to be combined to create a comprehensive representation of the workpiece.

[0109] The term "clipping," as used here, primarily refers to the removal of specific areas of the point cloud, leaving only the areas of interest for analysis or processing. This serves to focus attention on the interesting or important parts of the point cloud and to reduce the amount of data.

[0110] After step a), the reference point clouds can optionally be preprocessed by filtering, sampling, interpolation, combination, or clipping. Similarly, after step c), the actual point clouds can optionally be preprocessed by filtering, sampling, interpolation, combination, or clipping. The point clouds can be processed with various preprocessing steps, and it is not necessary to apply the same steps to both reference and actual data.

[0111] For example, outliers can be removed by applying a statistical outlier remover. Point clouds can be sampled to increase processing speed, or areas of the point cloud can be clipped. Point clouds can be clustered, combined, or smoothed. Edges in point clouds can be highlighted, for example, by covariance-based sampling. Holes in point clouds can be closed or enlarged, and points below a quality threshold can be removed, provided a quality can be assigned to the points. For example, the intensity of the captured light would be suitable for defining such a quality threshold: the lower the intensity, the less reliable the signal, and in the limiting case, it may even be lost in the noise.

[0112] In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, the correction data are generated by comparing the at least one reference feature and the at least one actual feature in a sensor coordinate system and / or spatial coordinate system. This is preferably done on the basis of the sections and not the entire point cloud.

[0113] This is particularly advantageous because comparing the at least one reference feature and the at least one actual feature in a common coordinate system, especially a sensor coordinate system and / or spatial coordinate system, enables precise and consistent determination of the correction data. Using a uniform coordinate system allows the features to be positioned and compared accurately, increasing the accuracy of deviation determination. This leads to more precise correction data used for adjusting the robot path. Furthermore, using a standardized coordinate system facilitates integration and compatibility with various sensors and systems, improving the flexibility and efficiency of the entire process.Furthermore, the results can be consistently transferred to a coordinate system known to the robot, provided the method is not already being applied in such a coordinate system. In a further preferred embodiment of the inventive method and / or the inventive device and / or the inventive system and / or computer program, the robot path is a trajectory for applying a coating material to the workpiece, wherein the coating material is preferably a sealant and / or an adhesive, more preferably PVC. This is particularly advantageous because applying a coating material such as sealant or adhesive, especially PVC, to a workpiece regularly requires relatively high precision. A precise robot path ensures that the coating material is applied evenly and exactly at the intended locations, which is crucial for the functionality and quality of the final product.Inaccuracies in application can lead to inadequate sealing or bonding, which could impair the product's performance and durability. Precisely adapting the robot path to the workpiece's actual dimensions ensures optimal application of the coating material, thereby increasing the efficiency and reliability of the production process. Adhering to tighter path tolerances may even allow for narrower coatings, resulting in material savings.

[0114] The present invention, in light of the features and embodiments discussed herein, overcomes the disadvantages of known prior art methods for robot path correction in a particularly advantageous manner. In particular, the present invention provides an offline robot path correction method that allows for immediate correction of the robot path and eliminates the need for a measurement run.

[0115] Another advantage is that the measurement points can be trained either virtually or on a template component. The acquisition of actual data can be either guided or static, increasing the flexibility of the process. The measurement run of the classic path correction is replaced or supplemented by potentially stationary data acquisition, thus increasing efficiency. The achievable data quality is within the limits of the sensor resolution and can be further improved in certain dimensions through averaging.

[0116] Another significant advantage is that the application location, i.e., the robot's execution point, does not have to be the recording location where the reference data is acquired or generated. This particularly facilitates applications in line tracking and with regard to moving parts. In particular, the reference data can also be generated virtually.

[0117] Brief description of the characters

[0118] The present invention is explained in more detail below with reference to the drawings, from which further features, embodiments, and advantages can be derived. In the embodiments shown in the figures, elements having similar or identical functions are designated with the same reference numerals. It should be noted that the figures may not be to scale.

[0119] This shows:

[0120] FIG 1 a schematic representation of a system according to the present invention; FIG 2 a schematic representation of a sequence control for a computer program according to the present invention;

[0121] FIG 3A and FIG 3B schematic representations of the extraction of virtual lines in the method according to the invention.

[0122] Detailed description

[0123] FIG 1 shows a schematic representation of a robot path correction system according to the invention. FIG 2 schematically shows a sequence control for a computer program according to the present invention. This system also includes a device according to the present invention for, preferably static, shape and / or position detection of workpieces 1 for robot path correction, with which a method according to the present invention can be carried out. The device has a sensor 14, here an optical and stationary sensor, for detecting an actual point cloud 6 of a workpiece 1 in the form of three-dimensional points.A processing unit 38 is provided for generating at least one actual characteristic 7 from the at least one actual point cloud 6, for determining deviations 8 between the at least one reference characteristic 5 and the at least one actual characteristic 7, and for determining correction data 9 from the deviations 8 between the at least one reference characteristic 5 and the at least one actual characteristic 7. A database 40 or a memory 41 stores the reference point cloud 3 and the reference data source 4 with the at least one reference characteristic 5. The correction data 9 are transmitted via an interface 39 to a robot controller 2 for adjusting the robot path 11. The system further includes the robot arm 10, which follows the robot path 11 specified by the robot controller 2 and corrected by the received correction data 9.The system and the device are configured to carry out a method according to the present invention. A computer program can be advantageously used to carry out the method.

[0124] The process according to the invention is also directly illustrated by the sequence control of the computer method shown in FIG. 2:

[0125] To achieve static shape and / or position acquisition of the workpiece 1 for robot path correction, a reference point cloud 3 is generated from a reference data source 4, and at least one reference feature 5 is generated or extracted from the at least one reference point cloud 3. To generate the at least one reference feature from the at least one reference point cloud 3, virtual reference values, particularly in the form of reference sections, are created, and the at least one reference feature 5 is generated from the reference sections. The reference data source 4 comprises CAD design data 22 of the ideal of the workpiece 1 and / or measurement data 23 of a real ideal of the workpiece 1.

[0126] Then, by capturing the workpiece 1 using the sensor 14, an actual point cloud 6 is generated from the current workpiece 1, from which actual characteristics 7 are derived by generating virtual actual values, in particular actual sections.

[0127] To generate the reference point cloud 3 from the reference data source 4 and / or to generate the actual point cloud 6 of the workpiece 1, a preprocessing step 24 is performed on the data by filtering, sampling, interpolation, combination and / or cutting. It should be understood that individual preprocessing steps 24, as shown in the figures, can be the same or different from other preprocessing steps 24 shown in the figures.

[0128] As can be seen from FIG 1, the sensor 14 for capturing the actual point cloud 6 can be arranged outside the application area of ​​the robot 10, in this case above the robot arm.

[0129] Once at least one actual characteristic 7 and at least one reference characteristic 5 are available, the deviation 8 between the at least one reference characteristic 5 and the at least one actual characteristic 7 is determined, and correction data 9 is derived from the deviations 8. This correction data 9 is transmitted to the robot system 10 via interface 39 to adjust the robot path 11.

[0130] This allows the robot path 11 to be adapted to the exact shape and position of the current workpiece 1, and, for example, the application of a coating material to the workpiece 1, such as the application of PVC to a body part 1, can be carried out extremely precisely and quickly.

[0131] Figures 3A and 3B provide schematic representations of the extraction of virtual lines to better illustrate the method according to the invention. When generating the actual point cloud 6 of the workpiece 1, the points of the actual point cloud 6 are preferably distributed over a surface. The at least one reference feature 5 is generated from the reference point cloud 3 by creating a selection object 21, here a cuboid, in the reference point cloud 3. Analogously, the at least one actual feature 7 is generated from the at least one actual point cloud 6 by placing the selection object 21 in the actual point cloud 6.

[0132] For each virtual measurement point, a suitable selection object 21 is placed at the desired location in the point cloud 3 or 6. The extent of the selection object 21 defines the set of points to be considered, and the orientation of the selection object 21 defines the orientation of the virtual section line (x). Figure 3A illustrates how the coordinate system of section lines can be defined, for example: the change in height (or, in the case of actual line sensors, the distance) lies in the positive Z direction, and the line itself runs in the positive X direction. The Y direction is then obtained via the cross product of the X and Z directions.

[0133] For individual virtual intersection lines, the Y-component is preferably always 0, as the virtual intersection lines should behave like a real line sensor. However, other values ​​for the y-component are also acceptable.

[0134] This is shown in FIG 3B:

[0135] For each virtual section line, the sampling rate in the X direction can be specified. The actual points are then obtained by sampling the point cloud at the defined location in the defined direction, with a maximum sampling density in the X direction as specified.

[0136] Optionally, multiple intersection lines with adjustable spacing can be generated and combined. This combination can be performed in various ways, such as averaging, median calculation, selecting the maximum value, selecting the minimum value, selecting the best quality (if available), or weighted averaging (if available).

[0137] The selection of the points to be considered in the point cloud can be done in various ways.

[0138] For example, a naive comparison of each point with the oriented selection object can be performed. This results in an asymptotic complexity of O(n), where n is the number of all points.

[0139] For example, the entire point cloud can be placed in the coordinate system of the selection object and evaluated within that coordinate system. In the example case of a selection cuboid, the membership check is thus reduced to easily verifiable axis-parallel comparisons. In this case, too, the asymptotic complexity is O(n), but the operations to be performed are computed more quickly.

[0140] For example, an axis-parallel cuboid enclosing the selection object can be determined to establish a rough membership; this membership check has an asymptotic complexity of O(n). Subsequently, the remaining points can be transformed into the coordinate system of the selection object and, in the case of a cuboid, thus checked by simple axis-parallel comparisons. Here, the complexity is O(m), where m is the number of remaining points.

[0141] For example, the point cloud can be inserted into a suitable tree structure, such as a kd-tree. The construction of such a tree has an asymptotic complexity of d (■ Jog). In the case of the reference point cloud, this only needs to be calculated once. Subsequently, nearest neighbors can be found with an asymptotic complexity of CH (gfi). Thus, the radius of a sphere enclosing the search object can be calculated, and a radius search around the center of this sphere can be performed using the tree to make a preliminary selection. The selected points can then be transformed into the coordinate system of the selection object, and, for example, in the case of a selection cuboid, the final membership can be calculated with an asymptotic complexity of O(m), where m is the number of preselected points.

[0142] Any other method for checking membership in the selection object can also be used.

[0143] Multiple virtual intersection lines can be generated from a single selection object. In this case, the intersection lines have different Y-values ​​(however, every point within a line always has the same Y-value). This is necessary because the selection object has a transformation matrix, and the lines must therefore have different Y-values ​​to perform unambiguous transformations (otherwise, all lines would coincide after the transformation). Additionally or alternatively, each line can be assigned an index to calculate the Y-values ​​on-the-fly in conjunction with the sampling distance in the Y-direction. It is also possible to leave each line at Y=0 and save a pose that includes a suitable translation in the Y-direction. Reference sign list.

[0144] workpiece (1)

[0145] Robot control (2)

[0146] Reference point cloud (3)

[0147] Reference data source (4)

[0148] Reference feature(s) (5)

[0149] Current point cloud (6)

[0150] Virtual section line (61)

[0151] Actual characteristic (7)

[0152] Deviation or deviation determination (8) Correction data (9)

[0153] Robot system, in particular robot arm or interface, (10) Robot path (11)

[0154] Sensor (14)

[0155] Selection object (21)

[0156] Design data (22)

[0157] Survey data (23)

[0158] Preprocessing step (24)

[0159] Processing unit (38)

[0160] Interface (39)

[0161] Database (40)

[0162] Storage (41)

Claims

Claims 1. A method for, preferably static, shape and / or position detection of a workpiece (1) for robot path correction, comprising the steps: a) generating at least one reference point cloud (3) from a reference data source (4), b) Generating at least one reference feature (5) from the at least one reference point cloud (3); c) Generating at least one actual point cloud (6) of the workpiece (1); d) Generating at least one actual feature (7) from the at least one actual point cloud (6); e) Determine at least one deviation (8) between the at least one reference characteristic (5) and the at least one actual characteristic (7); f) Determining correction data (9) from the at least one deviation (8) between the at least one reference characteristic (5) and the at least one actual characteristic (7); g) Transferring the correction data (9) to a robot system (10) to adjust the robot path (11).

2. The method of claim 1, wherein a step b) of generating at least one reference feature (5) from the at least one reference point cloud (3), comprising generating virtual reference values, in particular reference sections, reference section bundles, or reference volume sections, and preferably generating the at least one reference feature (5) from the reference values; and / or a step d) of generating at least one actual feature (7) from the at least one actual point cloud (6), a generation of virtual actual values, in particular actual sections, actual section bundles, or actual volume sections, and preferably of at least one actual feature (7) from the actual values, comprises, and preferably a step e) of determining the at least one deviation (8) between the at least one reference characteristic (5) and the at least one actual characteristic (7) includes determining at least one deviation (8) between the at least one actual characteristic (7) generated from the actual values ​​and the at least one reference characteristic (5) generated from the reference values.

3. Method according to one of claims 1 to 2, wherein step c) of generating at least one actual point cloud (6) of the workpiece (1) comprises capturing the actual point cloud (6) by means of a sensor (14), wherein the sensor (14) is preferably an optical sensor, preferably a 3D sensor.

4. Method according to claim 3, wherein the sensor (14) is a stationary sensor and / or a robot-guided sensor.

5. Method according to one of claims 3 or 4, wherein the sensor (14) for capturing the actual point cloud (6) is arranged outside the application area of ​​the robot (10).

6. Method according to one of claims 1 to 5, wherein step c) of generating at least one actual point cloud (6) of the workpiece (1) is carried out by moving a sensor (14) and the workpiece (1) relative to each other, preferably moving the sensor (14) relative to the workpiece (1).

7. Method according to one of claims 1 to 6, wherein in step c) of generating at least one actual point cloud (6) of the workpiece (1) the points of the actual point cloud (6) are distributed over a surface.

8. Method according to one of claims 1 to 7, wherein the at least one reference feature (5) is generated from the at least one reference point cloud (3), preferably the reference values, by placing a virtual selection object (21) in the reference point cloud (3) and / or the at least one actual feature (7) is generated from the at least one actual point cloud (6), preferably the actual values, by placing a selection object (21) in the actual point cloud (6).

9. Method according to one of claims 1 to 8, characterized in that the reference data source (4) comprises design data (22) of a workpiece ideal (1) and / or measurement data (23) of at least one workpiece ideal (1).

10. Method according to one of the preceding claims, wherein a step a) of generating at least one reference point cloud (3) from a reference data source (4) and / or a step c) of generating at least one actual point cloud (6) of the workpiece (1) comprises at least one preprocessing step (24) which is preferably selected from the group comprising filtering, sampling, interpolating, combining and cutting out the reference point cloud (3) or actual point cloud (6).

11. Method according to one of the preceding claims, wherein the correction data (9) are generated by comparing the at least one reference feature (5) and the at least one actual feature (7) in a common coordinate system, in particular a sensor coordinate system and / or spatial coordinate system.

12. Method according to one of the preceding claims, wherein the robot path (11) is a trajectory for applying a coating material to the workpiece (1), wherein the coating material is preferably a sealant and / or an adhesive, more preferably PVC.

13. Device for, preferably static, shape and / or position detection of workpieces (1) for robot path correction, preferably for a method according to one of claims 1 to 12, comprising - a sensor (14), preferably an optical sensor, for capturing an actual point cloud (6) of a workpiece (1), preferably in the form of three-dimensional points, - a processing unit (38) for performing the generation of at least one actual characteristic (7) from the at least one actual- point cloud (6) and / or for carrying out the determination of at least one deviation (8) between the at least one reference characteristic (5) and the at least one actual characteristic (7) and for determining correction data (9) from the at least one deviation (8) between the at least one reference characteristic (5) and the at least one actual characteristic (7); - a database (40) or storage (41) for storing reference point cloud (3) and / or reference data source (4) and / or reference features (5); - an interface (39) for transmitting the correction data (9) to a robot system (10) for adjusting the robot path (11).

14. Robot path correction system, comprehensive: - A device according to claim 13, - a robot system (10) that receives the correction data (9) and adjusts the robot path (11) accordingly, - preferably configured for carrying out a method according to claims 1 to 12.

15. Computer program for carrying out a method, preferably a method according to one of claims 1 to 12, for, preferably static, shape and / or position detection of a workpiece (1) for robot path correction, wherein the program: - receives an actual point cloud (6) of a workpiece (1) from a sensor (14) and generates at least one actual feature (7) from the at least one actual point cloud (6); and / or at least one reference feature (5) extracted from the at least one reference point cloud (3); and / or - at least one deviation (8) between the at least one actual characteristic (7) and the at least one reference characteristic (5) is determined; and / or - Determines correction data (9) from the at least one deviation (8) between the at least one reference feature (5) and the at least one actual feature (7); and / or transmits correction data (9) to a robot system (10) for adjusting the robot path (11), - Whereby the computer program is preferably configured to control a device according to claim 13 and / or a system according to claim 14.

Images