System comprising an isolator and a manipulator and a method for controlling a manipulator

Abstract

System comprising an isolator (2) with a housing (20), a manipulator (3) arranged in the housing (20) and a control device (4) for controlling the manipulator (3), characterized in that the system (1) comprises an image acquisition device (5), wherein the image acquisition device (5) is arranged and configured to capture a control gesture of a person (7) located outside the housing (20) in a control zone (70), and wherein the control device (4) is configured to associate the captured control gesture with a control instruction for the manipulator (3) and to control the manipulator (3) according to this control instruction.

Description

[0001] The invention relates to a system comprising an insulator with a housing and a manipulator arranged in the housing. The invention further relates to a method for controlling a manipulator arranged in the housing of an insulator.

[0002] In the context of this application, an isolator is defined as a workspace sealed off from its environment, preferably containing a defined atmosphere. Isolators are used, for example, in the pharmaceutical industry or in pharmaceutical process engineering for the manufacture, processing, and / or filling of sensitive products under sterile, germ-free, and contaminant-free conditions.

[0003] For access to the interior of a housing by a person outside the housing, so-called glove access methods are known. In addition, manipulators are increasingly being used. These manipulators can be used for both planned tasks, such as opening packaging, unpacking containers, transporting containers and other objects, and / or preparing filling needles in a bottling plant, as well as for unplanned tasks, such as troubleshooting.

[0004] Manipulators for other applications, as well as methods and devices for controlling them, are known, for example, from DE 10 2017 007 908 A1, DE 10 2012 110 190 B4, WO 2023 / 013 832 A1, US 10 768 708 B1, US 10 406 686 B2 and CN 1 12 917 470 A. In particular, WO 2023 / 013 832 A1 describes a control system for a surgical robot that uses hand tracking technology.

[0005] One object of the invention is to provide a system comprising an insulator with a housing, a manipulator arranged in the housing, and a control device, wherein the control device enables intuitive control of the manipulator. A further object of the invention is to provide an intuitive method for controlling a manipulator arranged in the housing of an insulator.

[0006] According to a first aspect, a system comprising an isolator with a housing, a manipulator arranged in the housing, a control device for controlling the manipulator, and an image capture device, wherein the image capture device is arranged and configured to capture a control gesture of a person located outside the housing in a control zone, and wherein the control device is configured to associate the captured control gesture with a control instruction for the manipulator and to control the manipulator according to this control instruction.

[0007] In other words, a gesture control system is created for the manipulator, enabling intuitive operation. The operator is located in a control zone outside the isolator. The manipulator inside the isolator can then be controlled for tasks that would otherwise require the operator to perform using gloves.

[0008] In some configurations, the control zone is located directly adjacent to the isolator. However, it is also conceivable to provide a control zone spatially separate from the isolator for teleoperation of the manipulator.

[0009] Gesture control allows the manipulator to be operated without the need for special input devices that send signals to the manipulator. This is particularly advantageous for use in a cleanroom, as input devices for this purpose must be specially cleaned and / or embedded in protective casings. Depending on the configuration, however, an additional input device may be required. Depending on the configuration, the gesture control may be activated by an activation gesture, by input at a user interface, or by a contactless release signal.

[0010] The terms "a", "a", "one", etc. are used in connection with the registration as indefinite articles and not as counter words. In particular, the system may, in its various configurations, have multiple manipulators and / or multiple image capture devices. Likewise, the image capture device and the control device may be configured to record a multitude of control gestures and associate them with different control instructions.

[0011] In the context of this application, the term "control" is generally used to refer to applying a control variable to a system. In one embodiment, the actual movement of the manipulator can be detected and fed back to the control device. In other embodiments, no feedback occurs for gesture control.

[0012] Depending on its design, the isolator contains additional equipment to fully or partially automate a specific process, such as filling a pharmaceutical product.

[0013] In one embodiment, the control device comprises distributed units for associating a detected control gesture with a control instruction and for controlling the manipulator.

[0014] In some configurations, a control gesture is defined as the orientation and / or movement of a body part, particularly a hand, foot, and / or head, of a person located within the control zone. Alternatively or additionally, in other configurations, the control gesture is defined as the orientation and / or movement of a passive control element by the person located within the control zone. A passive control element is defined as an element that does not itself function as an input device by sending signals to the control device. For example, the passive control element is made of stainless steel and is therefore easily cleanable for use in a cleanroom. For easy detection, the passive control element is, in some configurations, designed wholly or partially in a user-defined color, with a user-defined symbol, and / or with another feature perceptible by the image capture device.

[0015] In one embodiment, the person's movements are directly converted into movements of the manipulator. In some embodiments, the manipulator has an additional sensor system to ensure collision-free movement.

[0016] In other configurations, the control device is designed to simulate the manipulator's movement using a model before it is operated, in accordance with the control instruction associated with the detected control gesture. This allows for the detection and avoidance of potential collisions. The model is also referred to as a digital twin. In some configurations, the simulation and subsequent control of the manipulator occur in real time in response to a control gesture. This means that the manipulator's reaction time to the control gesture is kept so short that the person in the control zone subjectively perceives the manipulator's activation as instantaneous. For this purpose, the reaction time in some configurations is less than 10 ms.

[0017] In another configuration, a monitor is provided as an alternative or additional feature, which can be operated via the control device to visualize the simulated movement. Following the simulation, the operator can then decide whether the manipulator should be controlled accordingly.

[0018] In one embodiment, the control device is operable in a first operating mode, wherein, in the first operating mode, a movement of a body part, in particular a hand, a foot, and / or a head, of the person located in the control zone and / or a movement of a passive control element by the person located in the control zone can be detected as a control gesture, in particular a movement in space, wherein the control device is configured to associate the detected control gesture with a first control instruction for the manipulator in the first operating mode, wherein, according to the first control instruction, the detected control gesture specifies a movement of the manipulator, in particular a movement of an end effector arranged at one end of the manipulator along a trajectory in space. By means of the control gesture, both a movement along the three spatial directions and an orientation of the end effector in space can be specified.Depending on the design, the movement of the manipulator is limited to selected spatial directions and / or orientations or is possible in all six degrees of freedom.

[0019] The terms "first," "second," etc., are used solely for differentiation and do not indicate a sequence. The use of the term "first operating mode" does not imply the existence of a further operating mode. Rather, one embodiment provides that the control device can only be operated in the first operating mode, whereby the manipulator is controlled in such a way that it follows the movement of a person within the control zone.

[0020] According to a further embodiment, the control device can be operated in a second operating mode in addition to or as an alternative to the first operating mode, wherein in the second operating mode a movement of a body part, in particular a hand, a foot and / or a head, of the person located in the control zone and / or a movement of a passive control element by the person located in the control zone can be detected as a control gesture, in particular a movement in space, wherein the control device is configured to associate the detected control gesture with a second control instruction for the manipulator in the second operating mode, wherein according to the second control instruction the detected control gesture specifies a speed of a moving manipulator, in particular a speed of an end effector arranged at one end of the manipulator along a trajectory in space.In the second operating mode, it is possible, for example, to reduce the manipulator's speed during a planned task to allow for visual monitoring. By tracking the person's movement in space, only individual speed components can be modified.

[0021] According to a further embodiment, the control device can be operated in a third operating mode in addition to or as an alternative to the first and / or the second operating mode, wherein in the third operating mode a predetermined orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person located in the control zone and / or an orientation and / or movement of a passive control element by the person located in the control zone can be detected as a control gesture, in particular a predetermined movement in space, wherein the control device is configured to associate the detected control gesture with a third control instruction for the manipulator in the third operating mode, wherein, according to the third control instruction, the detected control gesture specifies a predetermined activity of the manipulator.In other words, in the third operating mode, control gestures are assigned to pre-known activities of the manipulator, such as setting up certain components like filling needles or the like, performing a quality control check, or correcting a recurring error.

[0022] If the control unit has different operating modes, certain configurations allow switching between these modes. For example, it is conceivable to navigate the manipulator to a desired position in the first operating mode and, in the third operating mode, to use it to open or close an end effector, such as a gripper, located at a free end of the manipulator.

[0023] In one embodiment, the image acquisition device is located inside the isolator housing. In other embodiments, the image acquisition device is located on an outer surface of the isolator housing. For teleoperation, in other embodiments, the image acquisition device is located spatially separate from the housing.

[0024] According to a second aspect, a method is created for operating a manipulator arranged in a housing of an insulator by means of a control device, wherein a control gesture of a person located outside the housing in a control zone is detected by means of an image detection device, and wherein the detected control gesture is associated with a control instruction for the manipulator by means of the control device and the manipulator is controlled according to this control instruction.

[0025] The method allows intuitive control of the manipulator by a person located outside the manipulator, without requiring special knowledge or skills. The person's authorization to control the manipulator can be verified beforehand using appropriate methods.

[0026] In one embodiment, it is provided that, before the manipulator is controlled according to the control instruction associated with the detected control gesture, a movement of the manipulator is simulated using a model. The simulation allows the control instruction to be verified and thus prevents the manipulator from being controlled with an instruction that could lead to a collision.

[0027] According to one embodiment, in a first operating mode of the control device, an orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person located in the control zone and / or an orientation and / or movement of a passive control element by the person located in the control zone is detected as a control gesture, in particular a movement in space, wherein in the first operating mode the control device associates the detected control gesture with a first control instruction for the manipulator, wherein according to the first control instruction the detected control gesture specifies a movement of the manipulator.

[0028] Alternatively or additionally, it is provided that in a second operating mode of the control device, an orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person located in the control zone and / or an orientation and / or movement of a passive control element by the person located in the control zone is detected as a control gesture, in particular a movement in space, wherein in the second operating mode the control device associates the detected control gesture with a second control instruction for the manipulator, wherein according to the second control instruction the detected control gesture specifies a speed of a moving manipulator.

[0029] Furthermore, in one embodiment, alternatively or additionally to the first and / or the second operating mode, it is provided that in a third operating mode the control device detects as a control gesture a predetermined orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person located in the control zone and / or an orientation and / or movement of a passive control element by the person located in the control zone, in particular a predetermined movement in space, wherein in the third operating mode the control device associates the detected control gesture with a third control instruction for the manipulator, wherein according to the third control instruction the detected control gesture specifies a predetermined activity of the manipulator.

[0030] According to a third aspect, a computer program product is created which includes instructions which, when the computer program product is executed by a control device for controlling a manipulator arranged in the housing of an isolator, cause the control device to analyze data from an image acquisition device by means of which a control gesture of a person located outside the housing, in a control zone, was detected, to associate the control gesture detected by means of the image acquisition device with a control instruction for the manipulator, and to control the manipulator according to this control instruction.

[0031] The computer program also allows existing control devices to be retrofitted for gesture control.

[0032] Further advantages and aspects of the invention will become apparent from the claims and from the description of an embodiment of the invention, which is explained below with reference to the figure. This figure shows: Fig. 1 schematically a system comprising an isolator with a housing, a manipulator arranged in the housing, a control device for controlling the manipulator and an image acquisition device.

[0033] Fig.Figure 1 schematically shows a system 1 comprising a merely indicated insulator 2 with a housing 20, a manipulator 3 arranged in the housing 20 with an end effector 30 designed as a gripper, a control device 4 for controlling the manipulator 3, and an image acquisition device 5, for example, a camera. The depicted system 1 further includes a monitor 6, which is arranged on an outer surface of the housing 20. All elements and components are shown schematically and not to scale.

[0034] In the illustrated embodiment, the image acquisition device 5 is arranged in the housing 20 of the insulator 2 in a protected environment. The image acquisition device 5 is arranged such that it can detect a control gesture of a person 7 located outside the housing 20 in a control zone 70.

[0035] The image acquisition device 5 is wirelessly or wired connected to the control device 4 for data exchange. The data from the image acquisition device 5 can be analyzed by the control device 4, which is configured to associate the captured control gesture with a control instruction for the manipulator 4. In one embodiment, the image acquisition device 5 sends raw data to the control device 4, with image analysis being performed exclusively by the control device 4. In other embodiments, the data is pre-processed at the image acquisition device.

[0036] The control device 4 is further connected wirelessly or wired to the manipulator for data exchange and is configured to control the manipulator 4 according to the control instruction associated with the detected control gesture.

[0037] The control device 4 can be operated in various operating modes in one embodiment.

[0038] In particular, the control device 4 is operable in a first operating mode in one embodiment, wherein the control gesture is an orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person 7 located in the control zone 70 and / or an orientation and / or movement of a (not shown) passive control element by the person 7 located in the control zone, and the detected control gesture is associated with a first control instruction for the manipulator 3, wherein, according to the first control instruction, the detected control gesture specifies a movement of the manipulator 3. The manipulator 3 follows a movement of the body part, in particular the hand, foot and / or head, of the person 7 located in the control zone 70 and / or a movement of the passive control element by the person 7 located in the control zone.

[0039] Alternatively or additionally, the control device 4 can be operated in a second operating mode in one embodiment, wherein the control gesture is also an orientation and / or movement of a body part, in particular the hand, foot and / or head, of the person 7 located in the control zone 70 and / or an orientation and / or movement of a (not shown) passive control element by the person 7 located in the control zone, but the control device 4 associates the detected control gesture with a second control instruction for the manipulator 4, wherein, according to the second control instruction, the detected control gesture specifies a speed of a moving manipulator 3.

[0040] Furthermore, in one embodiment, the control device 4 can be operated in a third operating mode alternatively or additionally to the first and / or the second operating mode, wherein a predetermined orientation and / or movement of a body part, in particular the hand, foot and / or head, of the person 7 located in the control zone 70 and / or a predetermined orientation and / or movement of a (not shown) passive control element by the person 7 located in the control zone is detected as a control gesture, wherein the control device 4 associates the detected control gesture with a third control instruction for the manipulator 3, and wherein, according to the third control instruction, the detected control gesture specifies a predetermined activity of the manipulator.

[0041] In each of the three operating modes, intuitive control of the manipulator 3 by a person located outside the housing 20 is possible.

Claims

[1] System comprising an insulator (2) with a housing (20), a manipulator (3) arranged in the housing (20) and a control device (4) for controlling the manipulator (3), characterized by , that the system (1) comprises an image capture device (5), wherein the image capture device (5) is arranged and configured to capture a control gesture of a person (7) located outside the housing (20) in a control zone (70), and wherein the control device (4) is configured to associate the captured control gesture with a control instruction for the manipulator (3) and to control the manipulator (3) in accordance with this control instruction. [2] System according to claim 1, characterized by , that the control device (4) is set up to simulate a movement of the manipulator using a model before controlling the manipulator (3) in accordance with the control instruction associated with the detected control gesture. [3] System according to claim 1 or 2, characterized by that the control device (4) is operable in a first operating mode, wherein in the first operating mode an orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person (7) located in the control zone (70) and / or an orientation and / or movement of a passive control element by the person located in the control zone can be detected, in particular a movement in space, wherein the control device (4) is configured to associate the detected control gesture with a first control instruction for the manipulator (3) in the first operating mode, wherein according to the first control instruction the detected control gesture specifies a movement of the manipulator (3), in particular a movement of an end effector (30) arranged at one end of the manipulator (3) along a trajectory in space. [4] System according to claim 1, 2 or 3, characterized bythat the control device (4) is operable in a second operating mode, wherein in the second operating mode an orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person (7) located in the control zone (70) and / or an orientation and / or movement of a passive control element by the person located in the control zone can be detected, in particular a movement in space, wherein the control device (4) is configured to associate the detected control gesture with a second control instruction for the manipulator (3) in the second operating mode, wherein according to the second control instruction the detected control gesture specifies a speed of a moving manipulator (3), in particular a speed of an end effector (30) arranged at one end of the manipulator (3) along a trajectory in space. [5] System according to any one of claims 1 to 4, characterized bythat the control device (4) is operable in a third operating mode, wherein in the third operating mode a predetermined orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person (7) located in the control zone (70) and / or an orientation and / or movement of a passive control element by the person located in the control zone can be detected as a control gesture, in particular a predetermined movement in space, wherein the control device (4) is configured to associate the detected control gesture with a third control instruction for the manipulator (3) in the third operating mode, wherein, according to the third control instruction, the detected control gesture specifies a predetermined activity of the manipulator (3). [6] Method for controlling a manipulator (3) arranged in a housing (20) of an insulator by means of a control device (4), characterized by, that by means of an image capture device (5) a control gesture of a person (7) located outside the housing, in a control zone (70) is captured, and by means of the control device (4) the captured control gesture is associated with a control instruction for the manipulator (3) and the manipulator (3) is controlled according to this control instruction. [7] Method according to claim 6, characterized by , that before the manipulator (3) is controlled according to the control instruction associated with the detected control gesture, a movement of the manipulator is simulated using a model by means of the control device (4). [8] Method according to claim 6 or 7, characterized by, that in a first operating mode of the control device (4) an orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person (7) located in the control zone (70) and / or an orientation and / or movement of a passive control element by the person located in the control zone is detected as a control gesture, in particular a movement in space, wherein in the first operating mode the control device (4) associates the detected control gesture with a first control instruction for the manipulator (3), wherein according to the first control instruction the detected control gesture specifies a movement of the manipulator (3), in particular a movement of an end effector (30) arranged at one end of the manipulator (3) along a trajectory in space. [9] Method according to claim 6 or 7, characterized by, that in a second operating mode of the control device (4) an orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person (7) located in the control zone (70) and / or an orientation and / or movement of a passive control element by the person located in the control zone is detected as a control gesture, in particular a movement in space, wherein in the second operating mode the control device (4) associates the detected control gesture with a second control instruction for the manipulator (3), wherein according to the second control instruction the detected control gesture specifies a speed of a moving manipulator (3), in particular a speed of an end effector (30) arranged at one end of the manipulator (3) along a trajectory in space. [10] Method according to claim 6, 7 or 8, characterized by, that in a third operating mode of the control device (4) a predetermined orientation and / or movement of a body part, in particular a hand, a foot and / or a head, of the person (7) located in the control zone (70) and / or an orientation and / or movement of a passive control element by the person located in the control zone is detected as a control gesture, in particular a predetermined movement in space, wherein in the third operating mode the control device (4) associates the detected control gesture with a third control instruction for the manipulator (3), wherein according to the third control instruction the detected control gesture specifies a predetermined activity of the manipulator (3). [11] Computer program product comprising instructions which, when executed by a control device (4) for controlling a manipulator arranged in a housing (20) of an isolator, cause the control device (4) to analyze data from an image acquisition device by means of which a control gesture of a person (7) located outside the housing, in a control zone (70) has been detected, to associate the control gesture detected by means of the image acquisition device (5) with a control instruction for the manipulator (3), and to control the manipulator (3) according to this control instruction.

Images