Inspection of predetermined trajectory of robot
A method using 3D modeling and augmented reality visualization addresses the challenge of detecting robot trajectory collisions, ensuring safer and more efficient robot operation by providing real-time collision warnings and enabling trajectory modifications.
Patent Information
- Application Number
- CN202380040247.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-13
- Publication Date
- 2024-12-31
AI Technical Summary
Existing methods fail to reliably check if a robot's predefined trajectory will collide with its real environment, especially during commissioning, as simulations may differ from reality.
A method involving a computer-implemented three-dimensional model of the environment and robot, determining distances between models, and visualizing the trajectory in augmented reality to issue warnings or safety signals for potential collisions, using devices like AR glasses or tablets for inspection and modification.
Enables rapid and accurate detection of collision risks, allowing for safer and more efficient robot trajectory adjustments, reducing the likelihood of collisions and enhancing commissioning safety.
Smart Images
Figure CN119233879A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method and a system for checking a pre-set trajectory of a robot and a computer program or a computer program product for executing the method. Background Art
[0002] The robot trajectory can be predetermined, in particular with the aid of a simulation environment and / or by learning (“teaching”), in particular learning of trajectory points.
[0003] Especially for commissioning runs, it is advisable to check in advance whether or possibly the real robot will collide with its real environment when driving along the pre-defined trajectory, which may differ from the simulated environment or the real environment as during learning. Summary of the invention
[0004] The object of the invention is to improve the operation of a robot, in particular by checking a predefined trajectory.
[0005] The object of the invention is achieved by a method having the features of claim 1. Claims 14 and 15 claim a system or a computer program or a computer program product for carrying out the method described here. The dependent claims relate to preferred developments.
[0006] According to one embodiment of the present invention, a method for checking a predetermined trajectory of a robot ("robot trajectory") comprises the following steps:
[0007] -determining or providing a computer-implemented three-dimensional model of the environment;
[0008] - determining the distance between the computer implemented robot model and the environment model for different segments of the trajectory; and
[0009] - visualizing the virtual representation of the trajectory in augmented reality using a visualization device to check the trajectory, wherein during the visualization,
[0010] - if the distance determined for a section of the trajectory is within a predefined warning range, a warning is issued for this section of the trajectory, and
[0011] If (as an alternative thereto) a safety signal is emitted for a section of the trajectory, in particular a distance determined for the section, is within a predefined safety signal range.
[0012] Thus, in one embodiment, the inspector can (more) reliably and / or (more) quickly check (monitor) whether the pre-set trajectory is in danger of a possible collision with the real environment. In one embodiment, the trajectory is checked, preferably by the inspector, with the aid of or based on the visualized virtual representation and the issued warning and / or safety signal, and in an extended solution, one or more sections for which a warning is (correspondingly) issued are checked specifically, in particular (more) detailed, (more) accurately and / or individually, in particular for a possible collision of the robot with the environment, and the trajectory is modified, in particular at least the section(s) are modified, if necessary.
[0013] In one embodiment, the robot comprises: a robot arm having three or more, preferably at least six, in one embodiment at least seven joints, in an extended embodiment revolute joints, which connect the movable members of the robot to one another and can be moved by a drive, in particular a motor, of the robot; and / or a movable base, in particular a base that can be moved by means of at least one drive, in particular a motor, of the robot. For such a robot, the present invention is particularly advantageous, in particular because of the complex trajectories that may exist. In one embodiment, a robot-guided tool or workpiece forms a (distal) movable member of the robot in the sense of the present invention, or the model of the robot (also) comprises a model of a robot-guided tool or workpiece.
[0014] In one specific embodiment, the trajectory is predetermined programmatically or by a working program and in a further development is predetermined in particular by means of a simulation environment and / or by learning or teaching, in particular learning of trajectory points.
[0015] In one embodiment, the three-dimensional environment model includes, in particular, permanently or temporarily stored data which illustrate or describe one or more three-dimensional contours or geometries of the real environment of the robot, in particular a robot cell, a production hall or a storage warehouse or the like.
[0016] In one embodiment, the robot model comprises, in particular, long-term or temporarily stored data which illustrate or describe the three-dimensional contour or geometry of the (real) robot, in particular one or more of its movable members. As described above, in one embodiment, a robot-guided tool or workpiece forms a (distal) movable member of the robot in the sense of the present invention, or the model of the robot (also) comprises a model of a robot-guided tool or workpiece.
[0017] The section of the trajectory for which a warning or safety signal is issued can consist of (trajectory) points or can be (trajectory) points whose distances are determined.
[0018] In one embodiment, the track section for which the warning or safety signal is issued extends on one or both sides through a (track) point or has a plurality of (track) points, wherein the section can be continuous or discrete, in particular a continuous or discrete sequence of points.
[0019] In one embodiment, in order to determine the distance of a segment having a plurality of points, the distance of a point selected therefrom (e.g., the starting point, the end point or the middle point of the segment) is determined (as the distance of the segment), and in a further development, the segment having a plurality of points is colored corresponding to or in accordance with the distance of the determined point. Thus, in one embodiment, the inspection can be performed (more) quickly.
[0020] In one embodiment, in order to determine the distance of a segment having a plurality of points, the distances of two or more discrete, in particular selected, points of the segment are determined, such as the start and end point of the segment, the start, end point and a middle point of the segment, etc., and the minimum of these distances is determined as the distance of the segment. Thus, in one embodiment, the inspection can be performed equally quickly and precisely.
[0021] In one embodiment, in order to determine the distance of a segment with multiple points, in particular a continuous segment, the minimum distance or the distance to the point closest to the environment model is determined as the distance of the segment. Thus, in one embodiment, the check can be performed (more) accurately.
[0022] In an extended solution, a section, in particular a section consisting of one or more points, is colored in accordance with or based on the determined distance. In this way, in one embodiment, the accuracy of the inspection can be improved. For example, a point or a section, in particular a section having a single point or a plurality of points, is colored red to issue a warning, and / or is colored green to issue a safety signal. Similarly, the coloring can also be multi-stage and / or discretized in other ways, for example discretized into {red, yellow, green}, etc., in particular in other colors and / or more finely discretized, or continuously changed with a decisive, in particular minimum distance, for example from red for a (too) small distance to green for a (sufficiently) large distance, etc.
[0023] In one embodiment, the method comprises the following steps:
[0024] - collecting data of the real environment of the robot by means of a collection device, in one embodiment a mobile collection device, in an extension a portable collection device, in particular a collection device that can be carried by a person;
[0025] Wherein, an environmental model is determined based on the collected data.
[0026] In one embodiment, the real surroundings of the robot can thus advantageously be taken into account during the check, so that the risk of a collision therewith can be checked particularly reliably.
[0027] In one embodiment, the acquisition device is arranged on the visualization device, in an extension it is arranged integrated or detachably on the visualization device. Thus, in one embodiment, the environment model can advantageously be determined on the spot or immediately before the visualization, and is therefore particularly up-to-date, thereby making the check particularly reliable or convincing.
[0028] In one embodiment, the acquisition device for acquiring data is moved translationally and / or rotationally relative to the real environment and / or is moved manually, in particular by an inspector operating the visualization device. As a result, in one embodiment, larger areas and / or environments of the environment can be acquired (more) accurately, thus making the inspection particularly reliable or convincing.
[0029] In one embodiment, the acquisition device has one or more contactless measuring distance meters, in an extension, one or more radar distance meters, one or more ultrasonic distance meters and / or one or more lidar distance meters. In this way, in one embodiment, the environment model can be determined (more) precisely, thus making the check particularly reliable or convincing. In this case, lidar distance meters are particularly preferred because of their compact design and precise measurement.
[0030] Additionally or alternatively, in one embodiment, the acquisition device has: one or more cameras, in one embodiment a 3D camera system, which in one embodiment has at least two or more stereo cameras, a triangulation system, at least one TOF camera, at least one interferometric camera, at least one light field camera, etc., in which at least one light source images a defined pattern on the environment and at least one camera preferably records the pattern from another perspective; and / or an image processing device. In one embodiment, the environment can thereby be acquired (more) quickly and the inspection can therefore be carried out particularly quickly and / or a larger environment can be considered.
[0031] Alternatively or particularly preferably in addition to the determination based on the acquired real environment data, in one embodiment, the environment model is determined based on predefined target data of the environment, in an extension based on CAD data of the environment. In one embodiment, the environment model can be determined (more) quickly and / or (more) accurately by taking into account such data.
[0032] In one embodiment, the model of the robot is determined based on predefined target data, in an extended version based on a predefined trajectory of the robot and / or based on CAD data of the robot and / or based on measurements of the robot. In one embodiment, by taking into account such data, the robot model can be determined (more) quickly and / or (more) accurately. In one embodiment, the robot model has, for different sections of the trajectory, in particular points, respectively, postures of the robot limbs relative to each other and / or relative to a reference system fixed to the environment that are predefined according to the predefined trajectory or describes such postures. In one embodiment, a posture or pose in the sense of the invention comprises a one-dimensional, two-dimensional or three-dimensional position and / or a one-dimensional, two-dimensional or three-dimensional orientation.
[0033] As described above, in one embodiment, a robot-guided tool or workpiece forms a movable member of the robot in the sense of the invention. In one embodiment, the robot model accordingly has a computer-implemented model of the robot-guided tool or workpiece as a movable member of the robot. Thus, in one embodiment, the risk of a robot-guided tool or workpiece colliding with the environment can be advantageously checked. Accordingly, in one embodiment, the robot model is determined based on predefined target data of the tool or workpiece, in an extended design CAD data, and / or based on measurements of the tool or workpiece.
[0034] In one embodiment, the environment model has one or more geometric primitives, which correspond to a real environment obstacle, especially to multiple real environment obstacles, according to a predetermined relationship, especially a spatial posture, and accordingly, at least the geometric primitives correspond to the real environment obstacles according to a predetermined relationship, especially a spatial posture. In addition or alternatively, in one embodiment, the robot model has one or more geometric primitives, which correspond to a robot limb, especially to multiple robot limbs, preferably at least one end effector, according to a predetermined relationship, especially a spatial posture, and accordingly, at least the geometric primitives correspond to the robot limbs according to a predetermined relationship, especially a spatial posture. Thus, in one embodiment, one or more distances can be determined (more) quickly. In one embodiment, the geometric primitive in the sense of the present invention is a polyhedron, especially a prism, especially a cuboid, or a cylinder, a cone, an ellipsoid, especially a sphere, etc. Thus, in one embodiment, one or more distances can be determined particularly quickly.
[0035] In one embodiment, the environment model is determined by at least one approximation of features, in particular points, particularly preferably a point cloud acquired by the acquisition device, in an extended solution by means of one or more grids and / or one or more approximation surfaces, in particular flat approximation surfaces and / or one or more curved approximation surfaces. In one embodiment, such grids or approximation surfaces are determined by means of compensation functions, in particular interpolation or extrapolation functions, smoothing functions and / or other fitting functions of the points acquired by the acquisition device. In particular, the environment model can have such approximations. In one embodiment, by means of such approximations, the environment can be modeled particularly advantageously, particularly quickly and / or accurately.
[0036] In one embodiment, the environment model is determined based on the robot, in particular based on data of the robot collected by a collection device, and / or based on a robot model. In an extended solution, the robot that may be collected along with the real environment data of the robot when the collection device is used to collect the real environment data is at least partially eliminated or hidden. In this way, in one embodiment, the environment model can be improved.
[0037] In one embodiment, the environment model is determined based on the selection of the environment region by the inspector. In an extension, the environment model does not take into account the environment region selected by the inspector, which in an extension is not already acquired by means of the acquisition device, and / or the environment model only takes into account the environment region selected by the inspector, which in an extension is only acquired by means of the acquisition device. As a result, in one embodiment, the environment model can be determined improved and / or (more) quickly.
[0038] In one embodiment, the distances between the robot model and the environment model for one or more different segments of the trajectory are based on
[0039] - an imaginary shield of the selected movable member of the robot, in one embodiment an imaginary shield of a tool or workpiece guided by the robot,
[0040] or
[0041] a virtual shield of a plurality of movable members of the robot, in one embodiment a virtual shield of all movable members of the robot, in one embodiment also a virtual shield of a tool or workpiece guided by the robot,
[0042] and
[0043] - a hypothetical enclosure of the robot's entire environment described by the environment model, or
[0044] - a hypothetical shield for a selected sub-area of the environment or shield
[0045] The minimum distance between them can be determined, in particular, by the corresponding distance.
[0046] In one embodiment, such a virtual shield is at least partially determined by the robot model or the environment model, in one embodiment by one or more geometrical primitives of the robot model and / or one or more geometrical primitives of the environment model and / or an approximation of the points collected by the acquisition device for determining the environment model or the points possessed by the environment model, in particular, it can be at least partially formed by geometrical primitives or approximations, in particular surfaces, corners, edges, nodes, coordinate lines, etc. of one or more grids or approximation surfaces, and / or has a predetermined distance, in particular an average distance, a minimum distance and / or a maximum distance relative to the surface of the robot (limb) or the environment or the environment area, and / or has a predetermined distance, in particular an average distance, a minimum distance and / or a maximum distance relative to one or more geometrical primitives, grids or approximation surfaces, or is defined or predetermined accordingly. The virtual shield can in particular be continuous or discrete. Thus, the distance between the robot model and the environment model can be determined using, for example, the minimum distance between the corners, edges and / or surfaces of geometrical primitives in the form of a cuboid of the robot model and the grid or approximation surface of the environment model.
[0047] In one embodiment, by determining the distance based on a minimum distance of an imaginary shield for a selected movable limb of the robot or based on a minimum distance of an imaginary shield for a selected sub-area of the robot's entire environment described by the environment model, the determination can be made (more) quickly, in particular limited to areas of primary relevance.
[0048] In one embodiment, by determining the distance based on a minimum distance of an imaginary shield of multiple, in one embodiment all movable limbs of the robot or a minimum distance of an imaginary shield of the robot's entire environment described by an environmental model, the check can be performed (more) reliably, in particular by taking into account a variety of different collision possibilities.
[0049] In one embodiment, the visualization device is a mobile visualization device, in particular a visualization device that can be carried (preferably by one hand of a person); in one embodiment, the visualization device comprises: a handheld device, preferably a handheld device, a tablet, a smartphone, a notebook, etc., and / or glasses, in particular AR (augmented reality) glasses. Thus, in one embodiment, the inspection can be carried out on site or on site and thus improved. In one embodiment, the visualization device is designed (in terms of hardware and / or software technology) for controlling the robot or is (also) used for this purpose. Thus, in one embodiment, the commissioning operation can be carried out (more) quickly and / or (more) safely.
[0050] In one embodiment, the (visualized) virtual representation of the trajectory comprises: a path fixed to a reference point of the robot, preferably an end effector of the robot, in one embodiment a continuous path; and / or a description of one or more, in particular all, movable members of the robot, in one embodiment with the aid of or through geometrical primitives of the robot model. In one embodiment, the description of one or more members of the robot changes during the visualization according to the predefined trajectory, and accordingly, the virtual representation can in particular comprise a virtual simulation of the robot or a description of the movement of one or more members of the robot when traversing the trajectory.
[0051] In one embodiment, by visualizing the path, the inspection can be performed (more) quickly and / or (more) reliably, in particular the inspector can inspect or judge the trajectory (more) simply, (more) intuitively and / or (more) quickly. In one embodiment, issuing a warning or a safety signal comprises: highlighting, preferably coloring and / or illuminating the corresponding track segment or path segment according to the distance determined for this purpose, wherein different distances can be assigned different highlights, preferably coloring or illuminating, for example: all distances within the warning range are red, all distances within the safety signal range are green; or all distances within the warning range are red, smaller distances within the safety signal range are yellow, and larger distances within the safety signal range are green; or the color changes continuously or in a plurality of discrete stages with the determined distance; and / or, the track segment or path segment with a distance within the warning range is displayed in an illuminated manner, while the track segment or path segment with a distance within the safety signal range is displayed without being illuminated, or its illumination varies with the distance.
[0052] Additionally or alternatively, issuing a warning or safety signal may include highlighting, preferably coloring and / or illuminating the display of one or more, in particular all, movable limbs of the robot according to the distance determined for the respective segment, wherein different distances may be assigned different highlighting, preferably coloring and / or illuminating as described above. For example, the description for or along the segment(s) of (too) small distance is colored red, and the description for or along the segment(s) of (sufficiently) large distance is colored green; likewise, the color of the description with the determined distance may change continuously or in a plurality of discrete stages, and / or the description of the distance may be shown illuminated in the warning range and not illuminated in the safety signal range, or its illumination may vary with the distance.
[0053] Likewise, in one embodiment, highlighting or issuing a warning or safety signal may include a corresponding trajectory segment or path segment or a style type described according to the distance determined therefor, wherein different distances may correspond to different styles, for example, all distances within the warning range may be represented by thick lines and / or solid lines, and all distances within the safety signal range may be represented by thin lines and / or dashed lines, and of course many other styles may also be used, such as line widths. It can vary continuously with distance or in multiple discrete stages, etc.
[0054] In one embodiment, by highlighting, particularly coloring and / or lighting, the inspector can inspect or judge the trajectory (more) quickly; by highlighting, particularly coloring and / or lighting, the description of one or more robot limbs can be (more) intuitive; by changing the style type, the visualization can be simplified and / or its recognizability can be improved.
[0055] In one embodiment, when the distance determined for a segment of the trajectory is within a warning range, the segment of the trajectory is visualized using a first virtual representation; and when the distance determined for the segment is within a safety signal range, the segment of the trajectory is visualized using a second virtual representation different from the first virtual representation. For example, as explained elsewhere, if the distance determined for a segment of the trajectory is within the warning range, the segment of the trajectory can be visualized using a first virtual representation that includes a path segment that is fixed to a reference point of the robot and is colored red; and if the distance determined for the segment is within the safety signal range, the segment of the trajectory can be visualized using a second virtual representation that is different from the first virtual representation that includes a path segment that is fixed to a reference point of the robot and is colored green.
[0056] In one embodiment, if the distance determined for a section of the trajectory is within a part of the warning range, the section of the trajectory is visualized using a first virtual representation, and if the distance determined for the section of the trajectory is within another part of the warning range, the section of the trajectory is visualized using another first virtual representation. Additionally or alternatively, in one embodiment, if the distance determined for a section of the trajectory is within a part of the safety signal range, the section of the trajectory is visualized using a second virtual representation, and if the distance determined for the section of the trajectory is within another part of the safety signal range, the section of the trajectory is visualized using another second virtual representation.
[0057] For example, in order to visualize a segment of a trajectory, if the distance determined for the segment is within the warning range, a first virtual representation is used, which includes a thicker and / or red-colored path segment fixed to a reference point of the robot; if the distance determined for the segment is larger but still within the warning range, another first virtual representation is used, which includes a thinner and / or orange-colored path segment of the reference point; if the distance determined for the segment is within the safety signal range, a second virtual representation is used, which includes a thicker and / or yellow-colored path segment of the reference point; and if the distance determined for the segment is within the warning range but larger, another second virtual representation is used, which includes a thinner and / or green-colored path segment of the reference point.
[0058] Of course, the above-mentioned coloring, lighting or style types and divisions are merely exemplary, and a large number of other discretizations and / or warning or safety signals may exist here.
[0059] In one embodiment, during the visualization of a virtual representation of a trajectory, one or more parameters of the trajectory are output for at least one segment, in particular a point, of the trajectory, in particular selected by an inspector. In one embodiment, a speed and / or at least one parameter, such as a speed, is output; and / or at least one parameter, such as a speed, is output for a segment, in particular a point, of the trajectory that has just been or is simulated to be traveled past or traveled towards during the visualization process.
[0060] Additionally or alternatively, in one embodiment, during the visualization process, a determined distance, in particular a value of a global minimum distance, is output for at least one segment of the trajectory and / or a segment of the trajectory, in particular a point, selected by the inspector, and / or a segment of the trajectory that has just been or is immediately simulated to be traveled through during the visualization process, in particular a point that is simulated to be traveled towards.
[0061] In one embodiment, the output of the parameter and / or distance value is carried out in the form of numbers, sounds and / or symbols. For example, the direction of travel can be output by an arrow; the TCP speed can be output by a corresponding digital specification; the distance value can be output symbolically by a corresponding line, in particular a measuring line with end points symbolized, for example by arrows, horizontal lines, etc., and / or digitally by a corresponding digital specification; and / or the parameter or distance value is output in the form of sound, in particular by a pitch that is different from a reference tone, which corresponds to the zero value of the parameter or distance value.
[0062] In one embodiment, by using one or more of the above features, an inspector may inspect or judge the trajectory (more) quickly and / or (more) reliably.
[0063] In one embodiment, the trajectory is modified according to the results of the inspection, in an extended solution, the modification is performed by (input or setting of) the inspector or automatically. Then, the method described herein can be re-executed for or with the modified trajectory in order to inspect the modified trajectory in a similar manner, wherein, in an extended solution, data of the real environment are re-collected and the environment model is re-determined based on the data, which can advantageously take into account changes in the real environment; in another extended solution, the previously used environment model is instead continued to be used or (re)provided, which can advantageously reduce the expenditure and time requirements.
[0064] The invention can be carried out particularly advantageously during commissioning of a robot or for commissioning a robot to travel a predefined trajectory, since safety can be particularly advantageously increased and / or effort and / or time can be reduced here, but the invention is not restricted thereto.
[0065] According to one embodiment of the present invention, a system is provided, which is particularly designed by hardware technology and / or software technology, particularly programming technology, to execute the method described herein, and / or comprises:
[0066] - means for determining or providing a computer-implemented three-dimensional model of the environment;
[0067] - means for determining the distance between the computer implemented robot model and the environment model for different sections of the trajectory; and
[0068] - a visualization device for visualizing a virtual representation of the trajectory in augmented reality for checking the trajectory, wherein during the visualization a warning is issued for a section of the trajectory if the distance determined for this section of the trajectory lies within a predefined warning range; and
[0069] And if the distance determined for a section of the trajectory is within a preset safety signal range, a safety signal is issued for the section of the trajectory.
[0070] In one embodiment, the system or apparatus thereof comprises:
[0071] - a collection device for collecting data of the robot's real environment, in particular a mobile collection device, in particular a portable collection device; and a device for determining an environment model based on the collected data, in particular by at least one approximation of the points collected by the collection device, wherein, in one embodiment, the collection device is arranged on the visualization device and / or has at least one non-contact measurement rangefinder, in particular at least one lidar rangefinder, radar rangefinder or ultrasonic rangefinder, and / or has at least one camera, in particular a 3D camera system, and / or an image processing device; and / or
[0072] - means for determining an environment model based on predefined target data of the environment, in particular CAD data, and / or based on the robot, in particular data of the robot acquired by means of an acquisition device, in particular by said acquisition device, and / or based on a robot model, and / or based on a selection of an environment area by an inspector; and / or
[0073] - for use based on predefined target data, in particular predefined trajectories of robots and
[0074] and / or CAD data of the robot and / or means for measuring the robot to determine the robot model; and / or
[0075] - means for determining the distance between the robot model and the environment model for at least one of the different sections of the trajectory based on the minimum distance between an imaginary shield of a selected movable limb of the robot or an imaginary shield of multiple, in particular all movable limbs of the robot and an imaginary shield of the entire environment of the robot described by the environment model or a sub-area selected therefrom; and / or
[0076] - means for visualizing a section of the trajectory using a first virtual representation when the distance determined for the section is within a warning range, in particular within a part of the warning range, and for visualizing the section of the trajectory using a second virtual representation different from the first virtual representation when the distance determined for the section is within a safety signal range, in particular within a part of the safety signal range, in particular visualizing the section of the trajectory using another first virtual representation when the distance determined for the section is within another part of the warning range, and / or visualizing the section of the trajectory using another second virtual representation when the distance determined for the section is within another part of the safety signal range; and / or
[0077] - for outputting, during the visualization of the virtual representation of the trajectory, at least one parameter of the trajectory, in particular the speed, and / or for at least one selected section of the trajectory and / or for a section, in particular a point, of the trajectory that is simulated to be traveled over or to be traveled towards during the visualization, and / or for at least one section of the trajectory and
[0078] and / or a selected section of the trajectory and / or a section, in particular a point, of the trajectory which is simulated to be travelled over or towards during the visualization, for outputting the value of the determined distance, in particular the global minimum distance, in particular in numerical and / or symbolic form; and / or
[0079] - Means for modifying the trajectory.
[0080] The system and / or device within the meaning of the present invention can be designed in hardware technology and / or software technology, in particular, it has: at least one processing unit, in particular a digital processing unit, in particular a microprocessor unit (CPU), a graphics card (GPU), etc., which is preferably connected to the storage system and / or bus system for data or signals, and / or one or more programs or program modules. The processing unit can be designed to: process the instructions of the program implemented as a program stored in the storage system; to collect input signals from the data bus; and / or to send output signals to the data bus. The storage system can have one or more, in particular different storage media, in particular optical, magnetic, solid and / or other non-volatile media. The program can be provided as follows: that is, it can embody or implement the method described herein, so that the processing unit can implement the steps of this method and thereby can in particular visualize the virtual representation of the trajectory or issue a warning or safety signal. In one embodiment, the computer program product can have, can in particular be a computer-readable, non-volatile storage medium for storing programs or instructions or a storage medium on which a program is stored or on which instructions are stored. In one embodiment, the implementation of the program or the instruction causes a system or a controller, in particular a computer or an arrangement of multiple computers, to execute the program or the instruction, so that the system or the controller, in particular the one or more computers, performs the method described herein or one or more steps thereof, or the program or the instruction is designed for this purpose.
[0081] In one embodiment, one or more, in particular all, steps of the method are performed completely or partially automatically, in particular by the system or a device thereof.
[0082] In one embodiment, the system has a robot. In one embodiment, a warning range is pre-set so that a collision between the robot and the environment exists or is (very) likely to occur within the warning range; and / or a safety signal range is pre-set so that a collision between the robot and the environment does not exist or is (very) unlikely to occur within the safety signal range.
[0083] In one embodiment, the method comprises the following steps:
[0084] - The robot drives over the track to be inspected.
[0085] In one embodiment, the trajectory can thus be advantageously checked with the real robot, wherein such a driving past after the check in augmented reality advantageously increases the safety of the robot during the driving past.
[0086] Accordingly, in one embodiment, the system or its device comprises:
[0087] - Means for driving the robot over the track to be inspected.
[0088] In one embodiment, the method also relates to operating a robot, including driving the robot through the inspected track, and thus can be particularly a method for operating a robot. Accordingly, in one embodiment, the system can also be a system for operating a robot. BRIEF DESCRIPTION OF THE DRAWINGS
[0089] Further advantages and features are given by the dependent claims and the embodiments. To this end, a partial schematic diagram is shown:
[0090] Figure 1 A system for checking a preset trajectory of a robot according to an embodiment of the present invention; and
[0091] Figure 2 A method for checking a preset trajectory of a robot according to an embodiment of the present invention. DETAILED DESCRIPTION
[0092] Figure 1 The system is shown in which an inspector 4 checks a pre-set trajectory of a robot 1 using a visualization device in the form of AR glasses 2 or a tablet computer 3 .
[0093] In step S10 (see Figure 2), the real environment 6 of the robot is collected by a collection device 5A or 5B, such as a 3D camera system, a lidar sensor, etc., which is preferably arranged integrally or detachably on the visualization device 2 or 3, and the computer-implemented environment model is determined in step S20 using these data. In a variant not shown, the environment model can be created additionally or alternatively based on the target data of the environment, or can be provided only from a memory, for example.
[0094] In step S30 , the distance between the computer-implemented robot model and the environment model is determined for individual sections, in one embodiment points, of a trajectory, wherein the trajectory is predetermined, for example, by means of a simulated environment or by learning or teaching.
[0095] For example, the robot model can have geometrical primitives in the form of cuboids, cylinders, etc., which correspond to a movable limb of the robot, and whose posture or pose changes accordingly according to the corresponding segment or trajectory point or when the robot simulates driving over the trajectory. The environment model can, for example, have a grid or an approximate surface, which approximates the point cloud collected when collecting the real environment. Then, as the distance between the robot model and the environment model, for example, the minimum distance between all these geometrical primitives and the grid or the approximate surface is determined. Then, for example, the minimum distance between all these geometrical primitives and the grid or the approximate surface is determined as the distance between the robot model and the environment model.
[0096] In step S40 , the virtual representation of the trajectory is visualized in augmented reality by the visualization device 2 or 3 in order to check the trajectory, for example whether the path of the TCP is a straight line and / or geometrical primitives when simulating driving over the trajectory.
[0097] During the visualization process, if the distance determined for a portion of the trajectory is within a pre-set warning range, a warning is issued for the portion of the trajectory, and if the distance determined for a portion of the trajectory is within a pre-set safety signal range, a safety signal is issued for the portion of the trajectory, for example by highlighting the corresponding portion as described above.
[0098] By means of the visualized virtual representation and the issuing of a warning or safety signal, the inspector 4 can check in step S40 whether there is a risk of collision with the environment when the robot 1 travels along the predefined trajectory, and to what extent.
[0099] In this case, the inspector can advantageously limit or focus on the section in which the warning is issued and inspect it (more) accurately and, if necessary, modify the trajectory in step S50, particularly in this section, and can then perform steps S30, S40 again and, if necessary, S50.
[0100] The checked trajectory can be traveled by the real robot in step S60 .
[0101] Although exemplary embodiments have been described in the foregoing description, it should be noted that many variations are possible. It should also be noted that the exemplary embodiments are merely examples and should not constitute any limitation on the scope of protection, application and configuration. On the contrary, the foregoing description can provide a person skilled in the art with a teaching for converting at least one exemplary embodiment, wherein various changes, in particular with respect to the functions and arrangements of the components, can be achieved without departing from the scope of protection of the present invention, such as can be obtained according to the claims and their equivalent feature combinations.
[0102] Reference numerals list
[0103] 1. Robot
[0104] 2 AR glasses
[0105] 3. Tablet
[0106] 4. Inspectors
[0107] 5A; 5B Collection device
[0108] 6 Environment
[0109] TCP Tool Center Point.
Claims
1. A method for checking a predetermined trajectory of a robot (1), comprising the following steps: - determining (S20) or providing a computer-implemented three-dimensional environment model; - for each segment of the trajectory, determining (S30) the distance between the computer-implemented robot model and the environment model; as well as - Visualizing (S40) a virtual representation of the trajectory in augmented reality with the aid of a visualization device (2; 3) in order to check the trajectory, wherein, during the visualization, if the distance determined for a section of the trajectory is within a pre-set warning range, a warning is issued for this section of the trajectory; and if the distance determined for a section of the trajectory is within a pre-set safety signal range, a safety signal is issued for this section of the trajectory.
2. The method according to the preceding claim, characterized in that The following steps are involved: - collecting (S10) data of the real environment of the robot by means of a collection device, in particular a mobile collection device, in particular a portable collection device; Therein, the environment model is determined based on the acquired data, in particular by at least one approximation of features, in particular points, acquired by means of the acquisition device.
3. The method according to the preceding claim, characterized in that The acquisition device is arranged on the visualization device, and / or the acquisition device moves translationally and / or rotationally and / or manually relative to the real environment in order to acquire data, and / or the acquisition device has: at least one non-contact measurement rangefinder, in particular at least one lidar rangefinder, radar rangefinder or ultrasonic rangefinder; and / or at least one camera, in particular a 3D camera system; and / or an image processing device.
4. The method according to any one of the preceding claims, characterized in that The environmental model is determined based on predetermined target data of the environment, in particular CAD data, and / or has at least one three-dimensional geometric primitive corresponding to real environmental obstacles in a predetermined relationship; and / or the robot model is determined based on predetermined target data of the robot, in particular a pre-set trajectory and / or CAD data of the robot and / or measurements of the robot, and / or has at least one three-dimensional geometric primitive corresponding to a limb of the robot in a predetermined relationship; and / or the robot model has a computer-implemented model of a robot-guided tool or workpiece serving as a movable limb of the robot.
5. The method according to any one of the preceding claims, characterized in that The environment model is determined based on data of the robot, in particular data of the robot acquired by means of the acquisition device, and / or based on the robot model, and / or based on a selection of an environment area by an inspector.
6. The method according to any one of the preceding claims, characterized in that For at least one segment of the trajectory, the distance between the robot model and the environment model is determined based on the minimum distance between an imaginary shield of a selected movable limb of the robot or an imaginary shield of multiple, in particular all, movable limbs of the robot and an imaginary shield of the entire environment of the robot described by the environment model or a selected sub-area thereof.
7. The method according to any one of the preceding claims, characterized in that The visualization device is a mobile visualization device, in particular a portable visualization device, in particular with a handheld device, in particular a smartphone and / or glasses, and / or the visualization device is designed to control the robot.
8. The method according to any one of the preceding claims, characterized in that The virtual representation of the trajectory comprises a reference point fixed to the robot, in particular a path of an end effector, in particular a continuous path, and / or a description of one or more, in particular all, movable limbs of the robot, in particular comprising a simulation of the robot as it travels along the trajectory.
9. The method according to any one of the preceding claims, characterized in that If the distance determined for a segment of the trajectory is within the warning range, in particular within a part of the warning range, the segment is visualized using a first virtual representation; and if the distance determined for the segment is within the safety signal range, in particular within a part of the safety signal range, the segment is visualized using a second virtual representation different from the first virtual representation.
10. The method according to the preceding claim, characterized in that If the distance determined for the segment of the trajectory is in another part of the warning range, visualizing the segment using another first virtual representation; and / or if the distance determined for the segment of the trajectory is in another part of the safety signal range, visualizing the segment using another second virtual representation.
11. The method according to any one of the preceding claims, characterized in that During the visualization of the virtual representation of the trajectory, at least one parameter of the trajectory, in particular a speed, is outputted and / or for at least one selected section of the trajectory and / or a section, in particular a point, of the trajectory which is simulated to be travelled over or travelled towards during the visualization, at least one parameter of the trajectory, in particular a speed, is outputted; And / or for at least one segment of the trajectory and / or a selected segment of the trajectory and / or a segment, in particular a point, of the trajectory that is simulated being traveled over or traveled toward during the visualization process, output a determined distance, in particular a value of a global minimum distance, in particular in the form of a number and / or sound and / or symbol.
12. The method according to any one of the preceding claims, characterized in that With the aid of a visualized virtual representation and issued warnings and / or safety signals, the trajectory is checked, in particular the section for which the warning is issued, and / or the trajectory is modified (S50) if necessary, in particular at least the section, and / or the robot is driven over (S60) the checked trajectory.
13. The method according to the preceding claim, characterized in that The virtual representation of the modified trajectory is visualized in augmented reality using the visualization device (S40) to check the modified trajectory, wherein during the visualization, if the distance determined for a portion of the modified trajectory is within a preset warning range, a warning is issued for the portion of the modified trajectory, and if the distance determined for a portion of the modified trajectory is within a preset safety signal range, a safety signal is issued for the portion of the modified trajectory.
14. A system for checking a predefined trajectory of a robot (1), the system being designed to carry out the method according to any one of the preceding claims and / or comprising: - means for determining or providing a computer-implemented three-dimensional model of the environment; - means for determining, for each segment of said trajectory, a distance between a computer-implemented robot model and said environment model; as well as - Visualization means (2; 3) for visualizing a virtual representation of the trajectory in augmented reality for checking the trajectory, wherein, during the visualization, a warning is issued for a section of the trajectory if the distance determined for this section of the trajectory is within a predefined warning range, and a safety signal is issued for this section of the trajectory if the distance determined for this section of the trajectory is within a predefined safety signal range.
15. A computer program or a computer program product, wherein: The computer program or computer program product comprises instructions, in particular stored on a computer-readable and / or non-volatile storage medium, which instructions, when executed by one or more computers or a system according to claim 14, cause the one or more computers or the system to perform a method according to any one of claims 1 to 13.