Inspection of predetermined trajectory of robot
By virtually representing the intersection of robot trajectory and environment in augmented reality, the problem of difficulty in checking trajectory collisions in real environments in the prior art is solved, faster and more reliable detection and modification are achieved, and the security of robot debugging is improved and efficiency.
Patent Information
- Application Number
- CN202380028212.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-15
- Filing Date
- 2023-03-13
- Publication Date
- 2024-11-15
AI Technical Summary
The prior art is difficult to effectively check whether the robot trajectory collides with the environment in a real environment, especially when the trajectory is pre-set, the real environment is different from the simulated environment.
By creating a three-dimensional environmental model and a robot model, we determine the intersection of the convex shield of the trajectory and the environmental model, and use a visual device to represent it virtually in augmented reality, send out warnings or safety signals, check whether the trajectory has intersection with the environment, and then modify it Track to avoid collisions.
It improves the reliability and speed of inspections, can more accurately detect the collision risks between the trajectory and the environment, and reduces the safety risks and expenses during the debugging process.
Smart Images

Figure CN118974781A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method and a system for checking a pre-set trajectory of a robot and a computer program or a computer program product for executing the method. Background Art
[0002] The robot trajectory can be predetermined, in particular with the aid of a simulation environment and / or by learning (“teaching”), in particular learning of trajectory points.
[0003] Especially for commissioning runs, it is advisable to check in advance whether or possibly the real robot will collide with its real environment when driving along the pre-defined trajectory, which may differ from the simulated environment or the real environment as during learning. Summary of the invention
[0004] The object of the invention is to improve the operation of a robot, in particular by checking a predefined trajectory.
[0005] The object of the invention is achieved by a method having the features of claim 1. Claims 12 and 13 claim a system or a computer program or a computer program product for carrying out the method described here. The dependent claims relate to preferred developments.
[0006] According to one embodiment of the present invention, a method for checking a predetermined trajectory of a robot ("robot trajectory") comprises the following steps:
[0007] -determining or providing a computer-implemented three-dimensional model of the environment;
[0008] - a convex shield defining at least one section of the trajectory;
[0009] - determining the intersection of the convex shield and the environment model; and
[0010] - Visualizing a virtual representation of the convex shield in augmented reality using a visualization device to check the trajectory, wherein during the visualization, if there is at least one intersection in a section of the trajectory, a warning is issued for this section of the trajectory, if there is no intersection, a safety signal (entwarnung) is issued (alternatively).
[0011] Thus, in one embodiment, the inspector can (more) reliably and / or (more) quickly check (monitor) whether the pre-set trajectory is in danger of a possible collision with the real environment. In one embodiment, the trajectory is checked, preferably by the inspector, with the aid of or based on a visualized virtual representation and the issued warning and / or safety signal, and in an extended solution, one or more sections of the convex shield of the trajectory for which a warning is (correspondingly) issued are checked specifically, in particular (more) detailed, (more) accurately and / or individually, in particular for a possible collision of the robot with the environment, and the trajectory is modified, in particular at least the section(s) are modified, if necessary.
[0012] In one embodiment, the robot comprises: a robot arm having three or more, preferably at least six, in one embodiment at least seven joints, in an extended embodiment revolute joints, which connect the movable members of the robot to one another and can be moved by a drive, in particular a motor, of the robot; and / or a movable base, in particular a base that can be moved by means of at least one drive, in particular a motor, of the robot. For such a robot, the present invention is particularly advantageous, in particular because of the complex trajectories that may exist. In one embodiment, a robot-guided tool or workpiece forms a (distal) movable member of the robot in the sense of the present invention, or the model of the robot (also) comprises a model of a robot-guided tool or workpiece.
[0013] In one specific embodiment, the trajectory is predetermined programmatically or by a working program and in a further development is predetermined in particular by means of a simulation environment and / or by learning or teaching, in particular learning of trajectory points.
[0014] In one embodiment, the three-dimensional environment model includes, in particular, permanently or temporarily stored data which illustrate or describe one or more three-dimensional contours or geometries of the real environment of the robot, in particular a robot cell, a manufacturing plant or a storage warehouse or the like.
[0015] In one embodiment, the model of the robot includes, in particular, long-term or temporarily stored data that illustrate or describe the three-dimensional contour or geometry of the (real) robot, in particular one or more of its movable members. As described above, in one embodiment, a robot-guided tool or workpiece forms a (distal) movable member of the robot in the sense of the present invention, or the model of the robot (also) includes a model of a robot-guided tool or workpiece.
[0016] The section of the trajectory for which a warning or safety signal is issued can consist of (trajectory) points or define the (trajectory) points of the intersection of the convex shroud.
[0017] In this context, a convex shield is preferably understood as a convex set, in particular a minimal convex set, which includes at least a part of the robot, in particular at least a part of one or more movable limbs of the robot, in a posture to which at least one (trajectory) point is assigned or can be assigned. If the segment of the trajectory includes two or more (trajectory) points, a convex shield is preferably understood as a convex set, in particular a minimal convex set, which includes at least a part of the robot, in particular at least a part of one or more movable limbs of the robot, in a posture to which the two or more (trajectory) points are assigned.
[0018] In one embodiment, the track section for which the warning or safety signal is issued extends on one or both sides through a (track) point or has a plurality of (track) points, wherein the section can be continuous or discrete, in particular a continuous or discrete sequence of points.
[0019] In one embodiment, in order to determine the intersection of the convex shield of the segment with the plurality of points and the environment model, it is determined whether at least a portion of the convex shield and at least a portion of the environment model share common points, common surfaces and / or jointly occupy a volume, and in an extended solution it is determined whether parts of the convex shield of the segment with the plurality of points are colored accordingly. Thus, in one embodiment, the check can be performed (more) quickly.
[0020] In one embodiment, in order to determine the intersection of a segment, in particular a continuous segment, with a plurality of (trajectory) points, the minimum distance or the distance to the point closest to the environment model is determined as the distance of the segment. In a further development, if the distance is within a predetermined and / or predefined range, the segment, in particular a segment of the convex cap, can be colored accordingly. In this way, in one embodiment, the check can be performed (more) precisely.
[0021] In an extended solution, when the determined distance is within a predetermined range, a segment of the robot, in particular a segment consisting of one or more points, in particular a segment of a trajectory, a segment of a convex shield and / or at least a part of the robot or a segment of at least one limb is colored corresponding to or based on the distance. In this way, in one embodiment, the accuracy of the inspection can be improved. For example, a segment, in particular a segment with a single point or multiple points, is colored red to issue a warning, and / or is colored green to issue a safety signal. Similarly, the coloring can also be multi-stage and / or discretized in other ways, for example, discretized into {red, yellow, green}, etc., in particular with other colors and / or more finely discretized, or continuously changed with a decisive, in particular minimum distance, for example from red for a (too) small distance to green for a (sufficiently) large distance, etc.
[0022] In one embodiment, the method comprises the following steps:
[0023] - collecting data of the real environment of the robot by means of a collection device, in one embodiment a mobile collection device, in an extension a portable collection device, in particular a collection device that can be carried by a person;
[0024] Wherein, an environmental model is determined based on the collected data.
[0025] In one embodiment, the real surroundings of the robot can thus advantageously be taken into account during the check, so that the risk of a collision therewith can be checked particularly reliably.
[0026] In one embodiment, the acquisition device is arranged on the visualization device, in an extension it is arranged integrated or detachably on the visualization device. Thus, in one embodiment, the environment model can advantageously be determined before the visualization or immediately, and is therefore particularly up-to-date, which makes the check particularly reliable or convincing.
[0027] In one embodiment, the acquisition device for acquiring data moves translationally and / or rotationally relative to the real environment. Thus, in one embodiment, a larger area and / or environment of the environment can be acquired (more) accurately, and thus the inspection is particularly reliable or convincing.
[0028] In one embodiment, the acquisition device has one or more contactless measuring distance meters, in an extension, one or more radar distance meters, one or more ultrasonic distance meters and / or one or more lidar distance meters. Thus, in one embodiment, the environment model can be acquired (more) precisely, thus making the check particularly reliable or convincing. In this case, lidar distance meters are particularly preferred because of their compact design and precise measurement.
[0029] Additionally or alternatively, in one embodiment, the acquisition device has: one or more cameras, in one embodiment a 3D camera system, which in one embodiment has at least two or more stereo cameras, a triangulation system, at least one TOF camera, at least one interferometric camera, at least one light field camera, etc., in which at least one light source images a defined pattern on the environment and at least one camera preferably records the pattern from another perspective; and / or an image processing device. In one embodiment, the environment model can thus be acquired (more) quickly and the inspection can therefore be carried out particularly quickly and / or a larger environment can be taken into account.
[0030] Alternatively or particularly preferably in addition to the determination based on the acquired real environment data, in one embodiment, the environment model is determined based on predefined target data of the environment, in an extension based on CAD data of the environment. In one embodiment, the environment model can be determined (more) quickly and / or (more) accurately by taking into account such data.
[0031] In one embodiment, the model of the robot is determined based on predefined target data, in an extension, based on a predefined trajectory of the robot and / or based on CAD data of the robot and / or based on measurements of the robot. In one embodiment, by taking into account such data, the robot model can be determined (more) quickly and / or (more) accurately. In one embodiment, the robot model has, for different segments, in particular points, a posture of the robot members relative to each other and / or to a reference system fixed to the environment, which is predefined according to the predefined trajectory, or describes such a posture. In one embodiment, a posture or pose in the sense of the invention comprises a one-dimensional, two-dimensional or three-dimensional position and / or a one-dimensional, two-dimensional or three-dimensional orientation. In one embodiment, the convex shield is determined based on discrete or continuous postures of the robot model, in particular based on interpolated discrete postures, in an extension, in particular the time-discrete postures of the robot model, in particular with the robot movement or independently of the robot movement, are used to determine the (trajectory) points with respect to the segments of the trajectory and / or the convex shield with respect to time.
[0032] As described above, in one embodiment, the robot-guided tool or workpiece forms a movable member of the robot in the sense of the present invention. In one embodiment, the robot model accordingly has a computer-implemented model of the robot-guided tool or workpiece as a movable member of the robot. Thus, in one embodiment, the risk of a collision of the robot-guided tool or workpiece with the environment can be advantageously checked, in particular the risk of a convex shield comprising a convex shield of the robot-guided tool or workpiece with the environment. Accordingly, in one embodiment, the robot model is determined based on pre-set target data of the tool or workpiece, in an extended design CAD data, and / or based on measurements of the tool or workpiece.
[0033] In one embodiment, the environment model has one or more geometric primitives, which correspond to a real environment obstacle, especially to multiple real environment obstacles, according to a predetermined relationship, especially a spatial posture, and accordingly, at least the geometric primitives correspond to the real environment obstacles according to a predetermined relationship, especially a spatial posture. In addition or alternatively, in one embodiment, the robot model has one or more geometric primitives, which correspond to a robot limb, especially to multiple robot limbs, preferably at least one end effector, according to a predetermined relationship, especially a spatial posture, and accordingly, at least the geometric primitives correspond to the robot limbs according to a predetermined relationship, especially a spatial posture. Thus, in one embodiment, the existence of one or more intersections can be determined (more) quickly. In one embodiment, the geometric primitive in the sense of the present invention is a polyhedron, especially a prism, especially a cuboid, or a cylinder, a cone, an ellipsoid, especially a sphere, etc. Thus, in one embodiment, the existence of an intersection can be determined particularly quickly.
[0034] In one embodiment, the environment model is determined by at least one approximation of features, in particular points, particularly preferably a point cloud acquired by the acquisition device, in an extended solution by means of one or more grids and / or one or more approximation surfaces, in particular flat approximation surfaces and / or one or more curved approximation surfaces. In one embodiment, such grids or approximation surfaces are determined by means of compensation functions, in particular interpolation or extrapolation functions, smoothing functions and / or other fitting functions of the points acquired by the acquisition device. In particular, the environment model can have such approximations. In one embodiment, by means of such approximations, the environment can be modeled particularly advantageously, particularly quickly and / or accurately.
[0035] In one embodiment, the environment model is determined based on the robot, in particular based on data of the robot collected by a collection device, and / or based on a robot model. In an extended solution, the robot that may be collected along with the real environment data of the robot when the collection device is used to collect the real environment data is at least partially eliminated or hidden. In this way, in one embodiment, the environment model can be improved.
[0036] In one embodiment, the environment model is determined based on the selection of the environment region by the inspector. In an extension, the environment model does not take into account the environment region selected by the inspector, which in an extension is not already acquired by means of the acquisition device, and / or the environment model only takes into account the environment region selected by the inspector, which in an extension is only acquired by means of the acquisition device. As a result, in one embodiment, the environment model can be improved and / or (more) quickly determined.
[0037] In one embodiment, the intersection between the convex shield, in particular the robot model, and the environment model of one or more different sections of the trajectory is based on
[0038] - an imaginary shield of the movable member of the selected robot, in one embodiment an imaginary shield of a tool or workpiece guided by the robot, which comprises or has a convex shield, or
[0039] - an imaginary shield of a plurality of movable members of the robot, in one embodiment an imaginary shield of all movable members of the robot, in one embodiment also an imaginary shield of a tool or workpiece guided by the robot, which includes or has a convex shield,
[0040] - a hypothetical enclosure of the robot's entire environment described by the environment model, or
[0041] - a hypothetical shield for a selected sub-area of the environment or shield
[0042] The intersection between them can be determined, and in particular the corresponding intersection.
[0043] In one embodiment, such a virtual shield is at least partially determined by a robot model or an environment model, in one embodiment, by one or more geometric primitives of the robot model and / or one or more geometric primitives of the environment model and / or an approximation of points collected by a collection device for determining the environment model or points possessed by the environment model, in particular, it can be at least partially formed by geometric primitives or approximations, in particular surfaces, corners, edges, nodes, coordinate lines, etc. of one or more grids or approximation surfaces, and / or has a predetermined distance, in particular an average distance, a minimum distance and / or a maximum distance relative to the surface of the robot (limb) or the environment or the environment area, and / or has a predetermined distance, in particular an average distance, a minimum distance and / or a maximum distance relative to one or more geometric primitives, grids or approximation surfaces, or is defined or pre-set accordingly. The virtual shield can in particular be continuous or discrete. Therefore, the intersection between convex shields can be determined based on the virtual shield of the robot model and the environment model, for example, using the intersection between corners, edges and / or surfaces of geometric primitives of the robot model and grids or approximation surfaces of the environment model.
[0044] In one embodiment, by determining the intersection of imaginary shields of selected movable limbs of the robot or of selected sub-areas of the robot's entire environment described by the environment model, which imaginary shields are in particular spaced apart from convex shields, the determination can be made (more) quickly, in particular restricted to areas of primary relevance.
[0045] In one embodiment, by determining the intersection of multiple, in one embodiment all movable limbs of the robot or imaginary shields of the robot's entire environment described by the environment model, the check can be performed (more) reliably, in particular also taking into account multiple different collision possibilities.
[0046] In one embodiment, the visualization device is a mobile visualization device, in particular a visualization device that can be carried (by a person, preferably with one hand), and in one embodiment, the visualization device comprises: a handheld device, preferably a handheld device, a tablet, a smartphone, a notebook, etc.; and / or glasses, in particular AR (augmented reality) glasses or VR (virtual reality) glasses. Thus, in one embodiment, the inspection can be carried out on the spot or on site and thus improved. In one embodiment, the visualization device is designed (in terms of hardware technology and / or software technology) for controlling the robot or is (also) used for this purpose. Thus, in one embodiment, the commissioning operation can be carried out (more) quickly and / or (more) safely.
[0047] In one embodiment, the (visualized) virtual representation of the convex shield of the trajectory includes a reference point fixed to the robot, preferably a path of the end effector of the robot, in one embodiment a continuous path, and / or a description of one or more, in particular all, movable members of the robot, in one embodiment with the aid of or by geometrical primitives of the robot model. In one embodiment, the description of one or more members of the robot changes during the visualization according to the predefined trajectory, and accordingly, the virtual representation can in particular include a virtual simulation of the robot or a description of the movement of one or more members of the robot when traversing the trajectory, in particular with the aid of the convex shield.
[0048] In one embodiment, by visualizing the path, and in particular the shield of the path, as a volume moving along the path, the inspection can be performed (more) quickly and / or (more) reliably, and in particular the inspector can inspect or judge the trajectory (more) simply, (more) intuitively and / or (more) quickly. In one embodiment, issuing a warning or safety signal includes: highlighting, preferably coloring and / or illuminating the corresponding trajectory segment or path segment based on the determined existence of an intersection, wherein the existence or non-existence of an intersection can be assigned different highlighting, preferably coloring or illumination, for example: the existence of an intersection is red, and the non-existence of an intersection is green; or the determined intersection is red, the smaller distance within the safety signal range (especially the imaginary shield or convex shield) is yellow, and the larger distance within the safety signal range is green; or the color changes continuously or in multiple discrete stages with the determined distance between the imaginary shield or convex shield and the environmental model; and / or, trajectory segments or path segments with intersections and / or distances within the warning range are displayed illuminated, while trajectory segments or path segments with distances within the safety signal range, especially without intersections, are displayed without illumination, or their illumination varies with distance.
[0049] Additionally or alternatively, issuing a warning or safety signal may include highlighting, preferably coloring and / or illuminating the description of one or more, in particular all, movable limbs of the robot according to the distance of the imaginary shield or convex shield determined for the corresponding segment from the environment model, wherein different distances can be assigned different highlights, preferably colors and / or illuminations as described above. For example, the description for or along the (too) small distance (s) is colored red, and the description for or along the (sufficiently) large distance (s) is colored green; similarly, the color of the description with the determined distance can be changed continuously or in a plurality of discrete stages, and / or the description of the distance is shown illuminated in the warning range and illuminated in the safety signal range, or its illumination varies with the distance.
[0050] In one embodiment, by highlighting, in particular coloring and / or lighting, the inspector can (more) quickly inspect or judge the trajectory, and by highlighting, in particular coloring and / or lighting, the description of one or more robot limbs can be (more) intuitively simplified and / or its recognizability improved.
[0051] In one embodiment, when the distance determined for a portion of a trajectory is within a warning range or there is an intersection, a first virtual representation is used to visualize an imaginary shield or a convex shield for the trajectory segment; and when the distance determined for the trajectory segment is within a safety signal range or there is no intersection, a second virtual representation different from the first virtual representation is used to visualize the trajectory segment. For example, as explained elsewhere, if the distance determined for a portion of a trajectory is within a warning range, a first virtual representation can be used to visualize the portion of the trajectory, the first virtual representation including a reference point fixed to the robot, in particular multiple reference points fixed to the robot, more particularly a path segment colored red including (multiple) reference points fixed to the robot in an imaginary shield or a convex shield; and when the distance determined for the portion of the trajectory is within a safety signal range, a second virtual representation different from the first virtual representation is used to visualize the portion of the trajectory, the second virtual representation including a reference point fixed to the robot, in particular multiple reference points fixed to the robot, more particularly a path segment colored green including (multiple) reference points fixed to the robot in an imaginary shield or a convex shield.
[0052] In one embodiment, when the distance determined for a section of the trajectory is within a part of the warning range or there is an intersection, a first virtual representation is used to visualize an imaginary shield or convex shield of the section, and when the distance determined for the section of the trajectory is within another part of the warning range, in particular there is no intersection, another first virtual representation is used to visualize the section. Additionally or alternatively, in one embodiment, when the distance determined for a section of the trajectory is within a part of the safety signal range, a second virtual representation is used to visualize the section of the trajectory, and when the distance determined for the section of the trajectory is within another part of the safety signal range, another second virtual representation is used to visualize the section.
[0053] For example, in order to visualize a segment of a trajectory, if the distance determined for the segment is within the warning range, a first virtual representation is used, which includes a thicker and / or red-colored path segment fixed to a reference point of the robot; if the distance determined for the segment is larger but still within the warning range, another first virtual representation is used, which includes a thinner and / or orange-colored path segment of the reference point; if the distance determined for the segment is within the safety signal range, a second virtual representation is used, which includes a thicker and / or yellow-colored path segment of the reference point; and if the distance determined for the segment is within the warning range but larger, another second virtual representation is used, which includes a thinner and / or green-colored path segment of the reference point. In one embodiment, this visualization with the aid of a reference point fixed to the robot, in particular a plurality of reference points fixed to the robot, and more particularly all reference points fixed to the robot can form a visualization of a convex shield. In an extended embodiment, points spaced apart from the reference points fixed to the robot in the normal direction are used to form an imaginary shield.
[0054] Thus, in one embodiment, a virtual shield or a convex shield can be visualized, in particular allowing an inspector to check a pre-set trajectory (more) quickly.
[0055] Furthermore, in one embodiment, the visualization of the imaginary shield or the convex shield can include a transparent or semi-transparent depiction in augmented reality. Thus, in one embodiment, intersections can be detected (more) quickly, in particular by inspectors.
[0056] Of course, the above-mentioned coloring, lighting or style types, transparency and divisions are merely exemplary, and a large number of other discretizations and / or warning or safety signals may also be present.
[0057] In one embodiment, during the visualization of a virtual representation of an imaginary shield or a convex shield of the trajectory, one or more parameters of the trajectory are output for at least one segment of the trajectory, in particular a point or a posture, in particular selected by the inspector. In one embodiment, a speed and / or at least one parameter, such as a speed, is output; and / or at least one parameter, such as a speed, is output for a segment of the trajectory, in particular a point or a posture, that has just been or is simulated to be driven past or driven towards during the visualization process.
[0058] Additionally or alternatively, in one embodiment, during the visualization process, for at least one segment of the trajectory and / or a segment of the trajectory, in particular a point selected by the inspector and / or a segment of the trajectory that has just been or instantly simulated to be traveled through during the visualization process, in particular a point that is simulated to be traveled towards, the determined distance of the imaginary shield or convex shield relative to the environmental model, in particular the value of the global minimum distance, is output.
[0059] In one embodiment, the output of the parameters and / or distance values is carried out in digital, acoustic and / or symbolic form. For example, the direction of travel can be output by an arrow; the TCP speed can be output by a corresponding numerical specification; the distance value can be output symbolically by a corresponding line, in particular a measuring line with end points symbolized, for example by arrows, horizontal lines, etc., and / or digitally by a corresponding numerical specification. In one embodiment, the output of the parameters can be visualized outside the imaginary shield or convex shield.
[0060] In one embodiment, by using one or more of the above features, an inspector may inspect or judge the trajectory (more) quickly and / or (more) reliably.
[0061] In one embodiment, the trajectory is modified according to the results of the inspection, in an extended solution, the modification is performed by (input or setting of) the inspector or automatically. Then, the method described herein can be re-executed for or with the modified trajectory in order to inspect the modified trajectory in a similar manner, wherein, in an extended solution, data of the real environment are re-collected and the environment model is re-determined based on the data, which can advantageously take into account changes in the real environment; in another extended solution, the previously used environment model is instead continued to be used or (re)provided, which can advantageously reduce the expenditure and time requirements.
[0062] The invention can be carried out particularly advantageously during commissioning of a robot or for commissioning a robot to travel a predefined trajectory, since safety can be particularly advantageously increased and / or effort and / or time can be reduced here, but the invention is not restricted thereto.
[0063] According to one embodiment of the present invention, a system is provided, which is particularly designed by hardware technology and / or software technology, particularly programming technology, to perform the method described herein, and / or comprises:
[0064] - means for determining or providing a computer-implemented three-dimensional model of the environment;
[0065] - means for determining a convex shield of at least one section of the trajectory;
[0066] - means for determining the intersection of the convex shroud and the environment model, in particular for different sections of the trajectory; and
[0067] - A visualization device for visualizing a virtual representation of a convex shield of the trajectory in augmented reality for checking the trajectory, wherein during the visualization, a warning is issued for a segment of the trajectory if at least one intersection exists for the segment, in particular if the distance determined for the segment is within a predefined warning range. Additionally or alternatively, in one embodiment, a safety signal can be issued for a segment of the trajectory if no intersection exists for the segment, in particular if the distance determined for the segment is within a predefined safety signal range.
[0068] In one embodiment, the system or apparatus thereof comprises:
[0069] - a collection device for collecting data of the real environment of the robot, in particular a mobile collection device, in particular a portable collection device; and a device for determining an environment model based on the collected data, in particular by means of at least one approximation of the points collected by the collection device; and / or
[0070] - at least one contactless measuring distance meter, in particular at least one lidar distance meter, radar distance meter or ultrasonic distance meter; and / or at least one camera, in particular a 3D camera system; and / or an image processing device; and / or
[0071] - means for determining an environment model based on predefined target data of the environment, in particular CAD data, and / or based on the robot, in particular data of the robot acquired by means of an acquisition device, in particular said acquisition device, and / or based on a robot model, and / or based on a selection of an environment area by an inspector; and / or - means for determining a robot model based on predefined target data, in particular a predetermined trajectory of the robot and / or CAD data of the robot and / or measurements of the robot; and / or
[0072] - a device for determining the intersection of a convex shield (and / or a virtual shield) between the robot model and the environment model for at least one of the different segments of the trajectory based on the intersection between a virtual shield of a selected movable limb of the robot or a plurality of, in particular all, movable limbs of the robot and a virtual shield of the entire environment of the robot described by the environment model or a sub-area selected therefrom; and / or - a device for determining the intersection of a convex shield (and / or a virtual shield) between the robot model and the environment model for at least one of the different segments of the trajectory when there is an intersection for a segment of the trajectory, in particular when the distance of the convex shield or the virtual shield determined for the segment is within a warning range, in particular within a part of the warning range. Means for visualizing the convex shield or imaginary shield of the segment using a first virtual representation and, in particular, using a second virtual representation different from the first virtual representation when the segment has no intersection, in particular, when the distance of the convex shield or imaginary shield determined for the segment is within the safety signal range, in particular, in a part of the safety signal range, using another first virtual representation, in particular, when the distance determined for the segment is in another part of the warning range, and / or using a second virtual representation different from the other first virtual representation when the distance determined for the segment is in another part of the safety signal range; and / or
[0073] - A device for outputting at least one parameter of the trajectory, in particular a speed, for the trajectory and / or at least one selected segment of the trajectory and / or a segment of the trajectory that is visualized, in particular simulated to be driven over or driven towards during the visualization, in particular a point or a gesture during the visualization, during the visualization of a virtual representation of the trajectory; and / or a device for outputting a distance, in particular a value of a global minimum distance, for at least one segment of the trajectory and / or a selected segment of the trajectory and / or a segment of the trajectory that is visualized, in particular simulated to be driven over or driven towards during the visualization, in particular a point or a gesture, in particular in digital, acoustic and / or symbolic form.
[0074] The system and / or device within the meaning of the present invention can be designed in hardware technology and / or software technology, in particular, it has: at least one processing unit, in particular a digital processing unit, in particular a microprocessor unit (CPU), a graphics card (GPU), etc., which is preferably connected to the storage system and / or bus system for data or signals, and / or one or more programs or program modules. The processing unit can be designed to: process the instructions of the program implemented as a program stored in the storage system; to collect input signals from the data bus; and / or to send output signals to the data bus. The storage system can have one or more, in particular different storage media, in particular optical, magnetic, solid and / or other non-volatile media. The program can be provided as follows: that is, it can embody or implement the method described herein, so that the processing unit can implement the steps of this method and thereby can in particular visualize the virtual representation of the trajectory or issue a warning or safety signal. In one embodiment, the computer program product can have, can in particular be a computer-readable, non-volatile storage medium for storing programs or instructions or a storage medium on which a program is stored or on which instructions are stored. In one embodiment, the implementation of the program or the instruction causes a system or a controller, in particular a computer or an arrangement of multiple computers, to execute the program or the instruction, so that the system or the controller, in particular the one or more computers, performs the method described herein or one or more steps thereof, or the program or the instruction is designed for this purpose.
[0075] In one embodiment, one or more, in particular all, steps of the method are performed completely or partially automatically, in particular by the system or a device thereof.
[0076] In one embodiment, the system has a robot. In one embodiment, a warning range is pre-set so that a collision between the robot and the environment exists or is (very) likely to occur within the warning range; and / or a safety signal range is pre-set so that a collision between the robot and the environment does not exist or is (very) unlikely to occur within the safety signal range.
[0077] In one embodiment, the method comprises the following steps:
[0078] - The robot drives over the track to be inspected.
[0079] In one embodiment, the trajectory can thus be advantageously checked with the real robot, wherein such a driving past after the check in augmented reality advantageously increases the safety of the robot during the driving past.
[0080] Accordingly, in one embodiment, the system or its device comprises:
[0081] - Means for driving the robot over the track to be inspected. BRIEF DESCRIPTION OF THE DRAWINGS
[0082] Further advantages and features are given by the dependent claims and the embodiments. To this end, a partial schematic diagram is shown:
[0083] Figure 1 A system for checking a preset trajectory of a robot according to an embodiment of the present invention; and
[0084] Figure 2 A convex shield for a segment of the trajectory of the robot according to one embodiment of the present invention; and
[0085] Figure 3 A method for checking a preset trajectory of a robot according to an embodiment of the present invention. DETAILED DESCRIPTION
[0086] Figure 1 The system is shown in which an inspector 4 checks a pre-set trajectory of a robot 1 using a visualization device in the form of AR glasses 2 or a tablet computer 3 .
[0087] Figure 2 A robot 1 is shown which performs a movement along a segment of a predetermined trajectory with a portion of its limbs. The movement is discretely shown in temporally successive postures which together form the movement of the segment along the predetermined trajectory. The movement is represented as a convex shield 8 and an imaginary shield 9 spaced apart from the convex shield. Both the convex shield 8 and the imaginary shield 9 have an intersection with the environment 6. The presence of this intersection can be indicated by issuing a warning, in particular a visual warning. The inspector can, for example, modify the pre-set trajectory based on the warning until a trajectory is found which does not intersect with the environment 6 and accordingly does not issue a warning.
[0088] In step S10 (see Figure 3 ), the real environment 6 of the robot is collected by a collection device 5A or 5B, such as a 3D camera system, a lidar sensor, etc., which is preferably arranged integrally or detachably on the visualization device 2 or 3, and the computer-implemented environment model is determined in step S20 using these data. In a variant not shown, the environment model can be created additionally or alternatively based on the target data of the environment, or can be provided only from a memory, for example.
[0089] In step S30, the intersection between the computer-implemented robot model and the environment model is determined for each segment, in one embodiment, point, of the convex shield of the trajectory, in particular the imaginary shield, wherein the trajectory is predefined, for example, by means of a simulated environment or by learning or teaching.
[0090] For example, the robot model may have geometric primitives in the form of cuboids, cylinders, etc., which correspond to a movable limb of the robot, and whose posture or pose will change accordingly according to the corresponding segment or trajectory point or when the robot simulates driving over the trajectory. The environment model may, for example, have a grid or an approximate surface, which approximates the point cloud collected when collecting the real environment. Then, as the distance between the robot model and the environment model, for example, the minimum distance between all these geometric primitives and the grid or the approximate surface is determined. Then, for example, the minimum distance between all these geometric primitives and the grid or the approximate surface is determined as the distance between the robot model and the environment model. Then, for example, the intersection between all these geometric primitives and the grid or the approximate surface or the points, in particular the volume, occupied by the robot model and the environment model at the same time is determined as the intersection between the robot model and the environment model.
[0091] In step S40, a virtual representation of a convex shield of the trajectory is visualized in augmented reality by means of a visualization device 2 or 3 to examine the trajectory, such as a shield surrounding the posture of the robot or its model (changing over time) when simulating driving through the trajectory, wherein the shield may be a convex shield; or the shield may be an imaginary shield, which is in particular spaced apart from the convex shield.
[0092] During the visualization process, if there is an intersection with a section of the trajectory, in particular if the distance determined for the section between the convex shield, in particular the imaginary shield and the environmental model is within a predetermined warning range, a warning is issued for the section of the trajectory, and if there is no intersection with the section, in particular if the distance determined for the section is within a predetermined safety signal range, a safety signal is issued for the section of the trajectory, for example by highlighting the corresponding section as described above.
[0093] By means of the visualized virtual representation and the issuing of a warning or safety signal, the inspector 4 can check in step S40 whether there is a risk of collision with the environment when the robot 1 travels along the predefined trajectory, and to what extent.
[0094] In this case, the inspector can advantageously limit or focus on the section in which the warning is issued and inspect it (more) accurately and, if necessary, modify the trajectory in step S50, particularly in this section, and can then perform steps S30, S40 again and, if necessary, S50.
[0095] The checked trajectory can be traveled by the real robot in step S60 .
[0096] Although exemplary embodiments have been described in the foregoing description, it should be noted that many variations are possible. It should also be noted that the exemplary embodiments are merely examples and should not constitute any limitation on the scope of protection, application and configuration. On the contrary, the foregoing description can provide a person skilled in the art with a teaching for converting at least one exemplary embodiment, wherein various changes, in particular with respect to the functions and arrangements of the components, can be achieved without departing from the scope of protection of the present invention, such as can be obtained according to the claims and their equivalent feature combinations.
[0097] Reference numerals list
[0098] 1. Robot
[0099] 2 AR glasses
[0100] 3. Tablet
[0101] 4. Inspectors
[0102] 5A; 5B Collection device
[0103] 6 Environment
[0104] 7 Convex shroud
[0105] 8 Imaginary Shield
[0106] TCP Tool Center Point.
Claims
1. A method for checking a predetermined trajectory of a robot (1), comprising the following steps: - determining (S20) or providing a computer-implemented three-dimensional environment model; - determining (S30) a convex shroud of at least one section of said trajectory; - determining the intersection of the convex shield and the environment model; as well as - Visualizing (S40) a virtual representation of the convex shield of the trajectory in augmented reality with the aid of a visualization device (2; 3) to check the trajectory, wherein, during the visualization, if at least one intersection exists for a section of the trajectory, a warning is issued for this section; if no intersection exists for a section of the trajectory, a safety signal is issued for this section.
2. The method according to the preceding claim, characterized in that The following steps are involved: - collecting (S10) data of the real environment of the robot by means of a collection device, in particular a mobile collection device, in particular a portable collection device; Therein, the environment model is determined based on the acquired data, in particular by at least one approximation of features, in particular points, acquired by means of the acquisition device.
3. The method according to the preceding claim, characterized in that The acquisition device is arranged on the visualization device, and / or the acquisition device moves translationally and / or rotationally relative to the real environment in order to acquire data, and / or the acquisition device has: at least one non-contact measurement rangefinder, in particular at least one lidar rangefinder, radar rangefinder or ultrasonic rangefinder; and / or at least one camera, in particular a 3D camera system; and / or an image processing device.
4. The method according to any one of the preceding claims, characterized in that The environment model is determined based on predetermined target data of the environment, in particular CAD data, and / or has at least one three-dimensional geometric primitive corresponding to real environmental obstacles in a predetermined relationship; and / or the robot model is determined based on predetermined target data of the robot, in particular a pre-set trajectory and / or CAD data of the robot and / or measurements of the robot, and / or has at least one three-dimensional geometric primitive corresponding to a limb of the robot in a predetermined relationship; and / or the robot model has a computer-implemented model of a robot-guided tool or workpiece serving as a movable limb of the robot.
5. The method according to any one of the preceding claims, characterized in that The environment model is determined based on data of the robot, in particular data of the robot acquired by means of the acquisition device, and / or based on the robot model, and / or based on a selection of an environment area by an inspector.
6. The method according to any one of the preceding claims, characterized in that For at least one segment of the trajectory, the intersection between the convex shield and the environmental model is determined based on the intersection between an imaginary shield of a selected movable limb of the robot or an imaginary shield of multiple, in particular all, movable limbs of the robot and an imaginary shield of the entire environment of the robot described by the environmental model or a sub-area selected therefrom.
7. The method according to any one of the preceding claims, characterized in that The visualization device is a mobile visualization device, in particular a portable visualization device, in particular with a handheld device, in particular a smartphone and / or glasses, and / or the visualization device is designed to control the robot.
8. The method according to any one of the preceding claims, characterized in that The virtual representation of the convex shield of the trajectory includes a description, in particular a continuous description, in particular a simulation of the robot, of one or more, in particular all, movable members of the robot during traversal of the trajectory.
9. The method according to any one of the preceding claims, characterized in that During visualization of a virtual representation of the convex shield, at least one parameter of the trajectory, in particular the speed, is output; and / or for at least one selected segment of the trajectory, and / or for a segment, in particular a point, of the trajectory that is simulated to be traveled over or traveled to during visualization, and / or for a segment, in particular a point, of the trajectory that is simulated to be traveled over or traveled to during visualization, at least one parameter of the trajectory, in particular the speed, is output, in particular in digital, acoustic and / or symbolic form.
10. The method according to any one of the preceding claims, characterized in that The trajectory is checked with the aid of a visualized virtual representation and the issued warnings, in particular the section of the virtual representation for which the warning was issued is specifically checked, and / or the trajectory, in particular at least the section, is modified (S50) if necessary, and / or the robot travels over (S60) the checked trajectory.
11. The method according to the preceding claim, characterized in that A virtual representation of the convex shield of the modified trajectory is visualized in augmented reality by the visualization device (S40) to check the modified trajectory, wherein during the visualization process, if, for a segment of the modified trajectory, the convex shield of the modified trajectory has at least one intersection with the environment model, a warning is issued for the segment of the modified trajectory.
12. A system for checking a predefined trajectory of a robot (1), the system being designed to carry out the method according to any one of the preceding claims and / or comprising: - means for determining or providing a computer-implemented three-dimensional model of the environment; - means for determining a convex shroud for at least one section of said trajectory; - means for determining the intersection of said convex shield and said environment model; as well as - visualization means (2; 3) for visualizing (S40) in augmented reality a virtual representation of a convex shield of the trajectory in order to check the trajectory, wherein during the visualization a warning is issued for a section of the trajectory if at least one intersection exists in this section.
13. A computer program or a computer program product, wherein: The computer program or computer program product comprises instructions, in particular stored on a computer-readable and / or non-volatile storage medium, which instructions, when executed by one or more computers or a system according to claim 12 , cause the one or more computers or the system to perform a method according to any one of claims 1 to 11 .
