Robot system for user guidance

Abstract

A robot system comprises a robot having an arm with a work unit arranged at its end, a sensor unit arranged around the arm for detecting, in particular optically, the actual working position of a user in a work area surrounding the actual position of the work unit, a display unit arranged around the arm for displaying a target working position of the user in the work area, and a control unit in signal communication with the sensor unit and the display unit for controlling the display unit based on the actual position of the work unit and / or the detected actual working position.

Description

[0001] The invention relates to a robot system comprising a robot, a sensor unit, a display unit and a control unit.

[0002] Robotic systems for user guidance are used in processes where users, i.e., humans, interact with robots. Such processes are also referred to as "interaction processes." The outcome of such interaction processes can generally be improved by providing the user with specific instructions on where, how, and when to interact with the robotic system. This helps avoid or reduce user errors that could affect the quality of the interaction process. For example, an image or video sequence could be displayed on a screen to guide the user. To avoid interfering with the user's work, such screens are positioned outside the user's and the robot's workspace where the interaction takes place.

[0003] Robotic systems for user guidance are typically complex and expensive, and can only check the user's work steps for accuracy after the fact. Furthermore, such systems are usually stationary and located outside the work area. Therefore, robotic systems cannot react directly to user errors.

[0004] DE 10 2023 116 771 B3 describes a method and a collaborative robot that uses a camera and a projector to improve interaction with humans. The robot projects context-related information directly into the human's workspace to provide visual cues about upcoming robot actions, necessary human actions, or safety-related warnings. This allows the user to focus their attention on the actual work process and react proactively.

[0005] DE 10 2020 106 161 B3 presents a system with distance sensors for monitoring a hazardous area around a tool, aiming to improve safety in direct human-robot collaboration. The sensors form a protective field that reduces the robot's speed or stops it when a person approaches, preventing injuries from sharp, rotating, or hot tools. Furthermore, the sensors' disc-shaped housing is rounded and edgeless to minimize the risk of injury in the event of a collision.

[0006] German patent application DE 10 2019 110 882 B4 describes a method for ensuring the safety of a robot arm in human-robot collaboration by using dynamically adjusted distance thresholds. These thresholds are adjusted in discrete steps, depending on the position of the machine part during a work step, to enable close and safe cooperation without unnecessary shutdowns. The system can learn the switching point for the work step and the adjusted thresholds, and cyclically check the functionality of the sensors at test points to ensure reliability.

[0007] From DE 10 2015 001 575 A1, a method and a device for visualizing the movement of a robot by means of optical display elements that are attached directly to the movable robot limbs and / or to the tool are known. These elements are controlled and activated according to the direction of movement of the robot, for example with warning colors (red) or release colors (green) to signal impending or ongoing movements to the worker.

[0008] German patent application DE 10 2010 005 708 B4 discloses a production system that enables overlapping work areas for humans and robots to ensure the simultaneous and collaborative execution of work steps. Human safety is ensured by limiting the robot's maximum speed in collaborative areas and defining "no-entry" zones for the robot near static objects to prevent entrapment. The system has various operating modes that can be dynamically switched to optimize productivity and safety depending on the presence of humans or robots in the respective work areas.

[0009] JP 2013-086223 A discloses a robot designed to improve operator safety. The robot is equipped with a robotic arm, a control section that regulates the movement of the robotic arm, and a motion indicator section located at the arm tip that displays the direction and speed of movement of the arm tip at the next given time. The motion indicator area is an array of pixels that light up or flash along the direction of movement of the arm tip.

[0010] The present invention is based on the objective of providing a robot system that enables improved user guidance.

[0011] This task is solved by a robot system with the features of claim 1. Advantageous further developments are the subject of the dependent claims and will become apparent from the description and the drawings.

[0012] The robot system according to the invention comprises a robot having an arm with a work unit arranged at its end, a sensor unit arranged around the arm, in particular in a ring shape, for detecting, in particular optically, the actual working position of a user in a working area surrounding the actual position of the work unit, a display unit arranged around the arm, in particular in a ring shape, for displaying a target working position of the user in the working area, and a control unit connected to the sensor unit and the display unit for controlling the display unit based on the actual position of the work unit and / or the detected actual working position.

[0013] A "robot" is understood to be a device that uses a movable arm with a work unit to perform specific mechanical tasks. Depending on the specific design of the work unit, it can, for example, pick up and put down objects. It can also perform tasks such as welding, screwing, riveting, gluing, or painting.

[0014] The sensor unit and the display unit are each arranged in a ring shape around the robot's movable arm, meaning they move with it. "Ring-shaped" can refer to a ring that is either closed or open in the circumferential direction. The ring shape can also be round, although "ring-shaped" generally includes angular configurations as well. The sensor unit and the display unit can, for example, share a housing, thus forming a single, connected unit. However, it is also conceivable that the sensor unit and the display unit are two physically separate units.

[0015] The sensor unit can include, for example, distance sensors for detecting the actual working position within the work area. The sensor unit detects any intervention in the work area, whereby the work area is defined or covered by the distance sensors; that is, the work area is preferably completely covered by the distance sensors.

[0016] Distance sensors can be either single-distance sensors or multi-sensors that generate distance values ​​for several adjacent measuring points. In the case of an optical measurement principle, so-called SPADs (single-photon avalanche diodes) can be used as sensors.

[0017] The sensor unit can detect the current working position optically, for example using the SPAD mentioned above or a camera. Electromagnetic detection, for example using radar, or the use of an acoustic measurement principle, such as ultrasound, are also conceivable.

[0018] The sensor unit's detection of the actual working position also makes it possible to identify when the user or another obstacle gets too close to the work unit, i.e., when a predetermined safety distance is breached. For this purpose, a protective volume can be defined around the work unit, which may differ from the work area and defines the safety distance in all directions. Both the shape and dimensions of the protective volume can vary, for example, depending on the work unit's speed or position, or they can be modified or programmed by adjusting the control unit. As soon as the control unit detects that the user or an obstacle is entering the protective volume, a safety response can be triggered that slows down or stops the work unit's movement and / or halts its operation.

[0019] The work area surrounding the work unit includes, in particular, an area in which the work unit and / or the user are scheduled to perform activities. The size of the work area depends on the specific application of the robot.

[0020] The target work position basically refers to the desired work position of the user, i.e., the exact location within the work area where the user is to perform a specific activity to interact with the robot or its work unit.

[0021] The control unit evaluates the sensor unit's data regarding the user's current working position and controls the display unit based on this data. For example, the control unit can activate the display unit to show a target working position for the user within the work area, such as when the actual working position deviates from this target position, or when the user is scheduled to change the current working position in the next step. It is also conceivable that the display unit might not show a target working position in the work area in certain situations, such as when the actual working position already matches the target working position.

[0022] The user receives direct instructions via the display within the workspace, enabling them to optimally execute the interaction process and its steps. This provides the user with immediate feedback on their interaction with the robot, both in terms of time and location. The feedback is displayed directly during the interaction process, guiding the user "live." Furthermore, the instructions are shown directly within the workspace, eliminating the need for the user to look at a separate screen located outside the workspace. This results in improved user guidance, as the system interacts directly with the user, preventing the need for the user to mentally reorient themselves by switching between a screen and the workspace.Furthermore, additional components, such as a separate screen, status indicators or an HMI ("Human Machine Interface"), can be omitted.

[0023] Preferably, the control unit is designed to monitor that the user correctly performs successive work steps of an interaction procedure in which the user interacts with the robot's work unit, and to guide the user through the interaction procedure by controlling the display unit to show the respective target work position.

[0024] This allows the user to be guided not only through individual work steps, but through the entire interaction process. The control unit knows several or even all of the steps of the interaction process and can guide the user through the correct procedure for each step. After completion of a step, the control unit can switch to the next step and guide the user accordingly.

[0025] The control unit thus knows the respective work steps, each of which includes at least one (target) action by the user defined by position, speed, or time. For example, a work step could be defined by the user moving their hand to a specific location in the work area and placing an object there, on which the work unit then performs actions.

[0026] Within these work steps, the control unit can compare the user's actual working position and / or its change over time, i.e., the speed of movement, known through evaluation of the recording data, with the target position known to the control unit and, if necessary, controls the display unit to show the user that the respective work step is being carried out correctly.

[0027] The correct execution of a work step includes aspects such as the position, but also the sequence and duration of the work steps. For example, the correct duration can be particularly important when gluing.

[0028] The control unit can be trained to determine that a step in the interaction process is complete based on a detected current work position and / or a detected change in the current work position over time. When the control unit determines that a step is complete, it can automatically switch to the next step, or, if the completed step was the last step in the interaction process, terminate the interaction process. In this way, the control unit can guide the system through the interaction process fully automatically, step by step, and in the correct sequence. The robot system can then monitor the correct sequence of all work steps and the correct execution of each step within that sequence. This prevents any work steps from being missed.Even if a work step is performed incorrectly, it can be prevented that the process continues to the next step. This increases the quality of the work results of the interaction process.

[0029] Preferably, the target work position is either a current or a future target work position. Therefore, if the display unit shows the user the target work position, this either indicates that the user's current actual work position is incorrect or that the user should change the actual work position for the next task. However, it is also conceivable that the display unit shows the user both the current and the future target work position, perhaps in different colors. This has the advantage for the user that they can prepare in advance for the upcoming change to the target work position. At the same time, the user also still sees the current target work position.

[0030] In particular, the control unit is designed to activate the display unit to show the target working position when the actual working position deviates from the target working position. This indicates to the user that a correction is necessary.

[0031] Preferably, the sensor unit comprises a plurality of distance sensors for detecting the work area. The distance sensors of the sensor unit arranged around the arm are, in particular, arranged in a ring around the robot's arm. The distance sensors can be arranged along a circumference of the sensor unit, especially at equal intervals. By detecting the distance to a detection object (user) within the work area, the precise position can be determined in two directions: by the detected distance and by the distance sensor that detected the distance. The portions of the work area detected by adjacent distance sensors can partially overlap, ensuring that the entire work area is covered by the distance sensors. Furthermore, redundancy of the distance sensors can be achieved in the overlapping areas.

[0032] The control unit can be configured to activate the display unit to announce the robot's upcoming movement. This informs the user where the robot, and in particular the arm with the work unit, will be moving next. This helps prevent situations that might surprise the user. For example, the color of the light can indicate to the user that the light is signaling an upcoming movement of the robot.

[0033] Preferably, the display unit is a spatially resolved display and, in particular, comprises a plurality of individually controllable LEDs. According to one embodiment, the display unit can have several display elements, in particular arranged parallel to one another, which are spaced apart from each other for display at different positions along the arm's extension direction. This allows, for example, the target working position to be displayed at different heights. The display elements can be designed as ring displays that run parallel and / or concentrically around the robot's arm.

[0034] The display unit can be configured to show different colors, with each color corresponding to different message types, such as warnings or announcements. For example, the display unit can indicate that maintenance is required and the current task is not yet complete, that a problem exists, or that the user should prepare for the next task or robot movement. These message types can be clearly distinguished by different colors; that is, each message type is assigned a different color.

[0035] It is also conceivable that certain display formats could be distinguished by a flashing, i.e., interrupted, display.

[0036] The invention is explained below by way of example and schematically with reference to an embodiment shown in the drawings. The drawings show: Fig. 1 a side view of an end section of a robot arm with a work unit, a sensor unit and a display unit and Fig. 2 a top view of a sensor unit according to the embodiment shown Fig. 1.

[0037] Fig. Figure 1 shows a section of a robot arm 11. A work unit 12 is located at the end of the arm 11, indicated here only schematically by a tapered end. In principle, anything that enables the robot to perform mechanical tasks can function as a work unit 12. Depending on the work unit 12, the robot can, for example, pick up and put down objects, weld, screw, rivet, glue, or paint. It is also conceivable that the work unit 12 is interchangeable, i.e., adaptable to changing tasks.

[0038] A housing 13, surrounding the arm 11 in a ring-like manner, is arranged on the arm 11. This housing contains a sensor unit 14, a display unit 15, and a control unit 16 that communicates with the sensor unit 14 and the display unit 15. Although in the present embodiment the sensor unit 14, the display unit 15, and the control unit 16 share the common housing 13, this is by no means necessary. The units 14, 15, and 16 can also be housed differently, for example, each in its own separate housing.

[0039] The sensor unit 14 consists of a multitude of distance sensors arranged along a circular path, as shown in Fig. 1 and Fig. Figure 2 shows that, in the present embodiment, the distance sensors are distributed in a periodic arrangement along the circular path. The distance sensors are also oriented towards the free end of the arm 11 so that they detect a working area surrounding the working unit 12. This design makes it possible to determine the position of an object detected by a distance sensor, for example, P1 in Fig. As shown in Figure 1, the distance can be determined in two directions: firstly, by assigning it to the distance sensor that detected the object, and secondly, by the detected distance.

[0040] The display unit 15 consists of a large number of individually controllable LEDs arranged along a circular path, see Fig.1. In the present embodiment, the LEDs are also distributed in a periodic arrangement along the circular path and oriented towards the free end of the arm 11. The LEDs of the display unit 15 can therefore illuminate the working area surrounding the working unit 12.

[0041] The control unit 16 monitors the correct execution of successive steps in an interaction process by the user. In such an interaction process, the user interacts with the robot's work unit 12. This can mean that the user and the robot, using arm 11, perform specific actions in a particular sequence or simultaneously to jointly achieve the goal of the interaction process.

[0042] The invention, according to the present embodiment, achieves this by having the sensor unit 14 detect the user's actual working position P1 using distance sensors during a specific step of the interaction process. The actual working position P1 can be, for example, the position of the user's hand or a tool used by the user. It may also be possible for the sensor unit 14 to detect changes in the actual working position P1 over time, allowing inferences to be made about the speed of movement, such as the speed of the hand or the tool.

[0043] The sensor unit 14 transmits the acquired data to the control unit 16 via the signal connection. The control unit 16 compares the acquired data concerning the actual working position P1 with a known target working position P2 for the currently executed work step of the interaction process and controls the display unit 15 to show the target working position P2 to the user via the LEDs. The LEDs then indicate the target working position P2 in the work area. However, it is also conceivable that the LEDs only indicate the target working position P2 in the work area if the actual working position P1 deviates from the target working position P2, possibly with an additional predefined tolerance known to the control unit 16.

[0044] The user receives immediate temporal and spatial cues regarding the interaction process from the light in the work area. The user does not need to take their eyes off the work area. This results in improved user guidance, as the system interacts directly with the user.

[0045] If a step of the interaction process is successfully completed, the procedure can be repeated for the next step, i.e. the control unit 16 compares newly recorded data concerning a current actual work position with the target work position(s) known for this step and instructs the user accordingly by displaying the target work position in the work area.

[0046] Switching from one work step to the next can be initiated by the user, for example, by pressing a designated button, operating a pedal, or giving a voice command. However, it is also possible for the control unit 16 to determine that a work step of the interaction process has been completed based on the detected current work position P1 or the detected change in the current work position P1 over time. Thus, the control unit can independently recognize when a work step has been successfully completed and then switch to the next work step without any additional input from the user.

[0047] The control unit 16 monitors whether the interaction procedure is correctly implemented by the user. Depending on the interaction procedure, this can include, for example, the correct sequence of the steps, the user's exact current working position, the user's movement speed (i.e., the change in the current working position over time), or the duration of a movement or a period of stillness, such as during an gluing process.

[0048] Alternatively or in addition to displaying the target work position P2, the control unit 16 can control the display unit 15 to announce a subsequent target work position by displaying it in the work area. This allows the user to prepare for the movement or activity required for the next work step while still performing the current one. For example, the current target work position P2 and the announced subsequent target work position can be displayed in different colors to prevent the user from confusing them.

[0049] The control unit 16 can also announce a subsequent movement of the robot or arm 11. This announcement can also be implemented using a different color to avoid user confusion. Announcing the robot's movement particularly reduces unwanted surprises for the user.

[0050] In general, individually controllable LEDs allow different states or information to be displayed by using different colors. For example, different colors can signal that the user should wait because the work step is not yet finished, that there is a problem and the user needs to intervene to solve it, or that the user should prepare for the next step. Reference symbol list 11 Arm 12 work units 13 cases 14 Sensor unit 15 Display unit 16 Control unit P1 Actual working position P2 Target working position

Claims

[1] Robot system, comprising a robot having an arm (11) with a work unit (12) arranged at its end, a sensor unit (14) arranged around the arm (11) for detecting an actual working position (P1) of a user in a work area surrounding an actual position of the work unit (12), a display unit (15) arranged around the arm (11) for displaying a target working position (P2) of the user in the work area and a control unit (16) in signal communication with the sensor unit (14) and the display unit (15) for controlling the display unit (15) based on the actual position of the work unit (12) and / or the detected actual working position (P1). [2] System according to claim 1, characterized by, that the control unit (16) is designed to monitor the correct execution of successive work steps of an interaction procedure in which the user interacts with the work unit (12) of the robot and to guide the user through the interaction procedure by controlling the display unit (15). [3] System according to claim 2, characterized by , that the control unit (16) is designed to compare the respective target work position (P2) of the work steps with the actual work position (P1) of the user in order to monitor the correct execution of the work steps. [4] System according to claim 2 or 3, characterized by , that the control unit (16) is designed to determine, based on a recorded actual work position (P1) and / or based on a recorded change in the actual work position (P1) over time, that a work step of the interaction procedure has been completed. [5] System according to any of the preceding claims, characterized by that the target work position (P2) is a current or a future target work position (P2). [6] System according to any of the preceding claims, characterized by , that the control unit (16) is designed to control the display unit (15) to display the target working position (P2) when the actual working position (P1) differs from the target working position (P2). [7] System according to any of the preceding claims, characterized by , that the sensor unit (14) has a plurality of distance sensors for detecting the working area. [8] System according to claim 7, characterized by , that the distance sensors are arranged along a circumference of the sensor unit (14) at equal distances from each other. [9] System according to claim 7 or 8, characterized by , that the sub-areas of the work area detected by neighboring distance sensors partially overlap. [10] System according to any of the preceding claims, characterized by , that the control unit (16) is designed to control the display unit (15) to announce a subsequent direction of movement of the robot. [11] System according to any of the preceding claims, characterized by , that the display unit (15) is a spatially resolved display. [12] System according to any of the preceding claims, characterized by , that the display unit (15) has several display elements which are spaced apart from each other for display at different positions in an extension direction of the arm (11). [13] System according to any of the preceding claims, characterized by a housing (13) that accommodates both the sensor unit (14) and the display unit (15). [14] System according to any of the preceding claims, characterized by, that the display unit (15) is designed to display in different colors, with different display forms such as warnings or announcements being assigned to the colors.

Images