Virtual pipetting

Abstract

Virtual pipetting is disclosed. A method for generating a control program for a laboratory automation device includes: receiving configuration data of the laboratory automation device, the configuration data encoding the position of a component in the laboratory automation device; generating a three-dimensional model of the component of the laboratory automation device according to the configuration data, the three-dimensional model also including a virtual pipette; displaying the three-dimensional model with a virtual reality headset; receiving movement data of a motion sensing controller controlled by a user wearing the virtual reality headset, the movement data indicating a three-dimensional movement of the motion sensing controller in space; determining the movement of the virtual pipette in the three-dimensional model according to the movement data and updating the three-dimensional model according to the movement; generating a control program for the laboratory automation device according to the movement data, the control program moving a pipette arm of the laboratory automation device with a pipette relative to the component according to the movement of the virtual pipette in the three-dimensional model.

Description

Technical Field

[0001] The invention relates to a method, a computer program and a computer-readable medium for generating a control program for a laboratory automation device and to a control system. Background Art

[0002] Laboratory automation equipment is used to automate the tasks of laboratory assistants, such as testing patients for specific diseases. Usually, samples of blood, urine, feces, etc. of patients are taken and analyzed with the help of biochemical procedures. Such procedures consist in various operations like adding substances, incubation, separation, etc., and measurement processes that quantitatively or qualitatively measure the amount or presence of substances that indicate specific diseases.

[0003] Typically, programming of laboratory automation equipment is a complex task requiring special skills. The program for controlling the laboratory automation equipment may have to be entered into a control PC. Although there may be graphical tools for generating such programs, more complex programs may have to be implemented in specific scripting languages, which require special knowledge in programming and in the operation of the laboratory automation equipment. Therefore, the laboratory assistants themselves often perform the programs with a small number of samples.

[0004] In order to simplify the programming of laboratory automation equipment with a robot arm, it is known to manually move the robot arm in a teaching mode, wherein the robot arm can be freely moved to any position and a teach point can be set by simply pressing a button. The position of the robot arm at the teach point is then saved and these positions are used to generate the control program.

[0005] EP 2 269 077 B1 relates to a graphical programming method, in which parts of the layout of a laboratory automation device are shown in a graphical interface. Pipettes in the graphical interface can be moved for defining a control program for the laboratory automation device. Summary of the invention

[0006] It is an object of the present invention to simplify the programming and / or control of laboratory automation equipment.

[0007] This object is achieved by the subject-matter of the independent claims. Further exemplary embodiments are evident from the dependent claims and the following description.

[0008] A first aspect of the present invention relates to a method for generating a control program for a laboratory automation device. The method can be performed with a system including a laboratory automation device, a controller thereof, a virtual reality headset, and a motion sensing controller. In general, the system can generate a three-dimensional virtual model of at least a portion of the laboratory automation device and can display the model in the virtual reality headset. Using the motion sensing controller, a virtual pipette that is also part of the three-dimensional model can be manipulated. A person or user wearing the virtual reality headset can use the motion sensing controller to move the virtual pipette and / or aspirate and expel virtual liquid within the model. The system can then record the manipulation of the virtual pipette and can generate a control program for the laboratory automation device that actually repeats the program executed in the three-dimensional model.

[0009] In this way, the programming of laboratory automation equipment can be simplified. The user can perform a task once with a virtual pipette in virtual reality. The generated control program can be executed several times and can actually perform the task multiple times with the laboratory automation equipment.

[0010] According to an embodiment of the invention, the method comprises: receiving configuration data of a laboratory automation device, the configuration data encoding a position of a component in the laboratory automation device.

[0011] Generally speaking, laboratory automation equipment can be any equipment adapted to automatically perform the tasks of a laboratory assistant. At least such laboratory automation equipment includes a pipetting arm adapted to move a pipette between different positions and to suck and discharge liquids at these positions. Liquids can be sucked from a cavity provided by the component and discharged in a cavity provided by the component. Such components may include at least one of a reservoir, a sample tube, a microtiter plate and / or a reagent container, etc.

[0012] The configuration data may encode the position of the component. For example, for each component, the configuration data may include the type of component and the position and / or orientation of the component provided in two-dimensional or three-dimensional coordinates.

[0013] The configuration data may be provided with a data file including information of a fixed layout of the laboratory automation device and / or with data generated by the laboratory automation device having sensors capable of determining which components are actually arranged in the laboratory automation device. For example, a laboratory assistant may arrange the components in the laboratory automation device and then may start the method. The laboratory automation device may then generate a configuration file encoding at least a portion of the positions of the arranged components, for example using a barcode reader adapted to read a barcode on the component.

[0014] According to an embodiment of the present invention, the method further comprises: generating a three-dimensional virtual model of a component of the laboratory automation device according to the configuration data, the three-dimensional model further comprising a virtual pipette. According to the configuration data, a virtual component in the three-dimensional model is generated. The virtual component can be modeled as an object of an object-oriented programming language and / or can include a three-dimensional layout of the component. For example, the three-dimensional layout of the virtual component can be determined according to the type of component in the configuration data and the standard geometric layout of the component, which can be stored in a device for performing the method, for example. The standard layout can then be moved by the position and / or orientation of the component in the configuration data.

[0015] It should be noted that the three-dimensional model does not include a model of a pipetting arm but includes a virtual pipette, which can replace the pipetting arm. The virtual pipette can be manipulated by a user who is shown with a three-dimensional model. Typically, the pipetting arm can be moved above the workbench of a laboratory automation device, and several components can be arranged on the workbench. For example, a box with a sample tube, one or more microtiter plates with a reservoir, and one or more containers with a reagent can be provided on the workbench. In addition, carriers for microtiter plates, shakers, incubators, trays with disposable tips, etc. can be a part of the laboratory automation device and / or be arranged on the workbench. All or some of these components can be encoded in the configuration data and / or provided in the three-dimensional model. The three-dimensional model can represent a part of the working area of ​​the laboratory automation device or a complete terrain and / or layout.

[0016] In general, the three-dimensional model may include a virtual component having a cavity in which a tip of a virtual pipette may be moved. The three-dimensional model may include a virtual component adapted to receive a liquid.

[0017] According to an embodiment of the invention, the method further comprises: displaying the three-dimensional model using a virtual reality headset. The virtual reality headset may be adapted to be worn by a user and / or to display a stereoscopic view of the three-dimensional model to the user. The virtual reality headset may generate two images for a person's eyes. The stereoscopic view of the three-dimensional model may depend on the position and / or orientation of the user's head. The virtual reality headset may include a sensor for determining the position and / or orientation of the user's head. It must be noted that augmented reality may additionally be considered virtual reality.

[0018] According to an embodiment of the present invention, the method also includes: receiving movement data of a motion sensing controller controlled by a user wearing a virtual reality headset, the movement data indicating three-dimensional movement of the motion sensing controller in space; determining the movement of a virtual pipette in a three-dimensional model based on the movement data; and updating the three-dimensional model according to the movement of the virtual pipette.

[0019] The motion sensing controller may have a sensor for determining the position and / or orientation of the motion sensing controller in space. For example, such a sensor may include a motion sensor, such as an accelerometer. The user may hold the motion sensor with his or her hand and may move it in the view of the three-dimensional model. The virtual pipette in the three-dimensional model is then moved according to the movement of the sensed motion sensing controller. For example, the motion sensing controller may only include a handle and the pipette tip of the virtual pipette is disposed at the lower end of the handle in the three-dimensional model.

[0020] According to an embodiment of the present invention, the method further comprises: generating a control program for the laboratory automation device according to the movement data, wherein the control program is adapted to move the pipette arm of the laboratory automation device with a pipette relative to the component according to the movement of the virtual pipette in the three-dimensional model. It is possible to generate and / or execute the control program during the time when the virtual pipette is moved by the user. In other words, the control program can be generated and / or executed (almost) simultaneously with the movement of the virtual pipette. It is also possible that the control program is generated and / or executed when the user has completed the movement of the virtual pipette. It is also possible that the control program is generated and stored when the user has completed the movement of the virtual pipette, and the control program is executed at a later time.

[0021] For example, specific tracking or teaching points on the trajectory of a virtual pipette can be saved. When the tip of the virtual pipette enters the cavity of the virtual component, such points can be defined and / or recorded. From these points, the movement of the pipetting arm can be derived, this movement also visiting all points. It must be noted that the trajectory of the pipetting arm can be different from the trajectory of the virtual pipette.

[0022] The movement of the pipetting arm can be encoded into a control program, which, when executed, performs the movement. The control program can be any data or data file, which, when processed by the controller of the laboratory automation device, causes the corresponding movement of the pipetting arm. The control program can also be a script program that can be modified by the user.

[0023] According to an embodiment of the present invention, the method further comprises: receiving activation data from a motion sensing controller, the activation data indicating a finger movement of a user on the motion sensing controller. In addition, a thumb may be considered as a finger. The motion sensing controller may comprise a button, the button may be adapted to sense whether it is pressed. The activation data may encode whether the button is pressed.

[0024] According to an embodiment of the present invention, the method further comprises: determining the suction and / or discharge of the virtual liquid in the three-dimensional model according to the position of the virtual pipette in the three-dimensional model, at which position the activation data indicates the activation of the motion sensing controller. The motion sensing controller can work like a mechanical and / or manual pipette. When the button is released, this can result in a virtual suction of the virtual pipette. When the button is pressed, this can result in a virtual discharge of the virtual pipette.

[0025] Which type of virtual liquid is contained in each cavity of the virtual component and / or which cavities are empty can be encoded in the three-dimensional model. These types of liquids can also be encoded in the configuration data. The type of liquid in the configuration data can be determined based on a barcode on the corresponding component, which barcode may have been read by a reader of a laboratory automation device. When the virtual pipette is sucked in, it can be assumed that the virtual pipette contains the type of liquid in the cavity into which its tip enters. When the virtual pipette is discharged, it can be assumed that the cavity into which the tip of the virtual pipette enters is filled with the type of liquid in the virtual pipette.

[0026] It must be noted that the type of liquid can refer to the chemical properties and / or content of the liquid, such as whether the liquid is a buffer solution, a sample, a reagent, etc., and / or refer to the physical properties of the liquid, such as viscosity, surface tension, density, etc.

[0027] According to an embodiment of the present invention, the control program is adapted to control the pipette of the pipette arm to inhale and discharge liquid according to the virtual pipette in the three-dimensional model. It can be recorded when the virtual pipette is inhaled and / or discharged. The control program generated then can also include a command for inhaling and / or discharging the pipette of the pipette arm when moving the pipette or its suction tip in the corresponding cavity.

[0028] According to an embodiment of the present invention, the suction point and the discharge point for liquid are determined according to movement data and activation data, and wherein the movement of pipetting arm is determined according to the suction point and the discharge point. The point of discharging or discharging liquid can be regarded as the most important tracking point and / or teaching point (teachpoint). It is possible that the control program is generated only from these points. The movement of pipetting arm between two continuous points (such as suction point and successive discharge points) can be determined independently of the movement of virtual pipette between these points. Therefore, the movement of pipetting arm can follow a path different from the virtual pipette. The movement of pipetting arm can be more optimized relative to path length and / or relative to the limitation of pipetting arm in view of the avoidance of collision in possible movement and / or laboratory automation equipment.

[0029] According to an embodiment of the present invention, the movement of the liquid actually sucked and discharged by the virtual pipette is performed in the three-dimensional model and / or displayed in the virtual reality headset. As already mentioned, the type of the virtual liquid contained in the cavity of the virtual assembly can be stored and / or tracked. In addition, the presence of the liquid can be indicated visually in the three-dimensional mode. Different types of liquids can be displayed with different colors. For example, the cavity of the virtual assembly for sample tubes, reservoirs, reagent containers, etc. can be shown as being full of colored and / or transparent substances.

[0030] It is also possible that different types of pipettes and / or pipette tips are displayed with differently colored virtual pipettes. The type of pipette tip can be defined by the maximum nominal volume of liquid that can potentially be aspirated and / or by the geometry of the pipette tip.

[0031] According to an embodiment of the present invention, the assembly includes a disposable tip and the installation and movement of a virtual disposable tip is performed in a three-dimensional model and displayed in a virtual reality headset. The three-dimensional model may contain a virtual assembly for the disposable tip. When a virtual pipette is moved on such a virtual disposable tip, it may be modeled so that the virtual disposable tip is installed on the virtual pipette. In addition, the handling of the virtual disposable tip may be modeled. The movement of the virtual disposable tip may be displayed to the user.

[0032] The control program determined based on the movement can then encode the installation and / or removal of the disposable tip at specific tracking points and / or teaching points.

[0033] According to an embodiment of the present invention, for each component, the configuration data encodes the type of component and the position of the component in the laboratory automation device. A three-dimensional model can be generated from modeling data encoding the geometric layout for each component. As already mentioned, the configuration data may include the type, position and / or orientation of the component and the three-dimensional layout of the virtual component may be determined based on the geometric layout, which may be associated with the type of the component. The geometric layout for the component may be modeled in advance and / or may be stored in a device that performs the method. In other words, the three-dimensional model may be assembled based on predefined geometric layout data of the component indicated in the configuration data.

[0034] According to an embodiment of the invention, the type of liquid contained in the component is specified in the configuration data of the laboratory automation device. These types can be determined by the sensors of the laboratory automation device and encoded in the configuration file. As already mentioned, the barcode on the component can encode the liquid contained. Different types of liquids can then be visualized differently in the three-dimensional model, for example with different colors.

[0035] It has to be noted that other computer-readable codes, such as QR codes and / or RFID tags, may additionally be provided on the component and read with the sensors of the laboratory automation device.

[0036] According to an embodiment of the present invention, the method further comprises: manually arranging at least some of the components in the laboratory automation device; and determining at least some of the configuration data using sensors of the laboratory automation device.

[0037] The laboratory automation device may include a fixed layout, ie components that are not movable and / or removable by a user, such components may include workstations, carriers, shakers, etc. Such a fixed layout may be provided in configuration data stored in a computing device executing the method.

[0038] Furthermore, the laboratory automation device may include a variable layout, i.e., components that can be moved, removed, and included in the laboratory automation device by a user. The position and / or orientation of these components may be determined using one or more sensors of the laboratory automation device. Configuration data thus determined may be sent from the laboratory automation device to a computing device that performs the method.

[0039] According to an embodiment of the present invention, movement data, activation data and / or installation / removal of a virtual disposable pipette tip are recorded for one virtual sample. A user performing a task in a three-dimensional model, i.e., in virtual reality, can perform this task for only one sample, but the three-dimensional model may include multiple samples.

[0040] A control program can then be generated so that it repeats the movement of the pipette, the aspiration and release of the pipette, and the installation and removal of disposable pipette tips for multiple real samples in the laboratory automation device. The control program can repeat these steps for each sample in the laboratory automation device (whose presence is determined by the sensor). It is also possible that the control program can be modified by the user after it is generated. For example, the user can include control structures in the control program that repeat the steps performed in virtual reality.

[0041] It must be noted that tracking points or teach points used to generate part of the control program can be defined relative to the cavity type keyed by the virtual pipette. Such points can not only encode coordinate positions. For example, tracking points or teach points can encode "pipette in sample tube", "pipette in empty reservoir", etc. In this way, not only the movement of the virtual pipette can be repeated with the laboratory automation device, but the steps of the task performed by the user in virtual reality for one virtual sample can be applied to several real samples in the laboratory automation device. In this case, it is not the exact movement of the virtual pipette but the movement scheme defined with the virtual pipette that can be repeated.

[0042] For example, the movement of the virtual pipette can be repeated for all samples of a microtiter plate. In this case, the movement can be determined by adding an offset to the original movement, the offset depending on the distance between the reservoirs in the microtiter plate.

[0043] It is also possible that the movements of the virtual single channel pipette are mapped to the movements of a real multi-channel pipette.

[0044] According to an embodiment of the present invention, a motion sensing controller is adapted to be held in the hand of a user wearing a virtual reality headset. The motion sensing controller may include a motion sensor, such as an acceleration sensor for generating movement data and / or may include a button for generating activation data. It is possible that the motion sensing controller is designed to be like the handle portion of a real pipette. The pipette portion may be modeled in a three-dimensional model and / or may only exist in virtual reality.

[0045] According to an embodiment of the invention, the virtual reality headset comprises two displays for generating a stereoscopic view of the three-dimensional model. It is possible that the virtual reality headset is an augmented reality headset. It must be noted that the stereoscopic view of the three-dimensional model can be displayed so that it is visible outside the real laboratory automation device. The user does not have to be inside the laboratory automation device to perform the task.

[0046] According to an embodiment of the present invention, the virtual pipette is a multichannel pipette including a plurality of pipette tips. As mentioned above, the handle portion of the pipette may be provided by a motion sensing controller. The pipette portion with a plurality of pipette tips may only exist in the three-dimensional model.

[0047] Another aspect of the invention relates to a computer program for generating a control program for a laboratory automation device, which control program, when executed by a processor, is adapted to perform the steps of the method as described above and in the following. The computer program can be executed in a computing device (such as a PC) communicatively interconnected with the laboratory automation device, the virtual reality headset and the motion sensing controller.

[0048] Another aspect of the invention relates to a computer readable medium in which such a computer program is stored. The computer readable medium may be a floppy disk, a hard disk, a USB (Universal Serial Bus) storage device, a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory) or a FLASH memory. The computer readable medium may also be a data communication network, such as the Internet, which allows downloading of program code. In general, the computer readable medium may be a non-transitory or a transient medium.

[0049] Another aspect of the invention relates to a control system for a laboratory automation device, the system comprising a laboratory automation device, a virtual reality headset, a motion sensing controller, and a computing device. The computing device is communicatively interconnected with the laboratory automation device, the virtual reality headset, and the motion sensing controller, for example via a USB, Ethernet, and / or wireless connection. In addition, the computing device is adapted to perform the method as described above and hereinafter.

[0050] The computing device can also be adapted to control laboratory automation devices and / or can also execute control programs generated with the aid of virtual reality.

[0051] It has to be understood that features of the method as described above and in the following may be features of the control system, the computer program and the computer-readable medium as described above and in the following, and vice versa.

[0052] Furthermore, the method may be a method for controlling a laboratory automation device, which may then also execute the generated control program.

[0053] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In the following, embodiments of the present invention are described in more detail with reference to the accompanying drawings.

[0055] Figure 1 A control system according to an embodiment of the present invention is schematically shown.

[0056] Figure 2 A three-dimensional model used in a method for generating a control program according to an embodiment of the present invention is schematically shown.

[0057] Figure 3 A virtual pipette used in the method for generating a control program according to an embodiment of the present invention is schematically shown.

[0058] Figure 4 A flow chart of a method for generating a control program and controlling a laboratory automation device according to an embodiment of the present invention is shown.

[0059] The reference symbols used in the drawings and their meanings are listed in summary form in the list of reference symbols. In principle, identical parts are provided with the same reference symbols in the figures. DETAILED DESCRIPTION

[0060] Figure 1A control system 10 is shown, which includes a laboratory automation device 12, a virtual reality headset 14, and a motion sensing controller 16. The control system 10 further includes a computing device 18, such as a PC, communicatively interconnected with the laboratory automation device 12, the virtual reality headset 14, and the motion sensing controller 16. and / or Perform communication.

[0061] The laboratory automation device 12 includes a workbench 20 to which several movable components 22 can be mounted. For example, the components 22 include a container 22a with a disposable pipette tip 24, a container 22b with a sample tube 26, a microtiter plate 22c with a reservoir 28, and a reagent container 22d. The reservoir 28 can contain a liquid 29.

[0062] The laboratory automation device 12 also includes a pipetting arm 30 having a pipette 32 and a sensor 34 adapted to determine the position and / or orientation of the component 22. The sensor 34 may also be and / or include a reader adapted to read a barcode or more generally a computer-readable code on the component 22. The sensor 34 may also be and / or include a camera, a laser scanner and / or any device adapted to determine the position and / or orientation of the component 22.

[0063] The virtual reality headset 14 includes two displays 36 adapted to provide two slightly different images to a user wearing the virtual reality headset 14. The two images may generate a stereoscopic view of a scene generated by the computing device 18, so that the user has a spatial impression of the scene. Furthermore, the virtual reality headset 14 may include a motion sensor 38, such as an acceleration sensor, adapted to determine the position and / or orientation of the user's head. In this way, the user's view position and / or view direction may be determined.

[0064] The motion sensing controller 16 is adapted to be held in the user's hand with a handle portion 40. As a virtual reality headset 14, the motion sensing controller 16 includes a motion sensor, such as an accelerometer 42 adapted to determine the position and / or orientation of the motion sensing controller 16. The motion sensing controller 16 is used in the system to move a virtual pipette displayed in the virtual reality headset 14.

[0065] Furthermore, the motion sensor controller comprises a button 44 which can be pressed by the user. With the button 44, aspiration and dispensing with the virtual pipette can be triggered.

[0066] Figure 1 Also shown are data that may be exchanged between components during operation of system 10 .

[0067] Initially, configuration data 46 from the laboratory automation device 12 may be sent to the computing device 18. The configuration data 46 may encode the position and / or orientation of at least some of the components 22 of the laboratory automation device 12. This data may be acquired using the sensors 34, for example.

[0068] Based on the configuration data 46, the computing device generates a three-dimensional or virtual model of at least a portion of the laboratory automation device 12. A virtual pipette is also displayed in this model.

[0069] The computing device 18 receives movement data 48 from the virtual reality headset 14 , which may encode the actual position and / or orientation of the virtual reality headset 14 .

[0070] The computing device 18 receives movement data and optional activation data 50 from the motion sensing controller 16 , which encodes the actual position and / or orientation of the motion sensing controller 16 and the button press state.

[0071] Based on the data 50 , the computing device 18 determines the position and / or orientation of the virtual pipette in the virtual model and generates display data 52 for the virtual reality headset 14 with the aid of the data 48 .

[0072] In this way, as will also be described in more detail below, the computing device 18 is able to record the movements and optional activations of the virtual pipette during the task performed by the user in the virtual model. When the user has completed the task, the computing device 18 generates a control program 54 for the laboratory automation device 12, which performs substantially the same task as the laboratory automation device 12.

[0073] Finally, the computing device 18 may execute a control program which then generates control commands 56 which are sent to the laboratory automation device 12 and / or control the pipetting arm 30 to repeat the task performed by the user with the virtual pipette.

[0074] Figure 2 Schematically shown is a three-dimensional model 58 which can be generated by the computing device 18 and can be displayed in the virtual reality headset 14. The three-dimensional model 58 can consist of virtual components 22' which have been generated with the aid of the configuration data 46. The three-dimensional model can be regarded as a virtual model of at least a part of the laboratory automation device.

[0075] For example, virtual components 22' may include virtual container 22a' with virtual disposable pipette tip 24', virtual container 22b' with virtual sample tube 26', virtual microtiter plate 22c' with virtual reservoir 28', and virtual reagent container 22d'.

[0076] The virtual components 22' can furthermore include components which are based on fixed real components of the laboratory automation device 12. For example, a virtual workbench 20' can be present.

[0077] It must be understood that the three-dimensional model 58 may be composed of objects of a programming language and / or the virtual component 22' may be entirely based on these objects. These objects may be transformed into display or image data 52 using a graphics rendering engine, which may then be displayed on the display 36.

[0078] In addition, the three-dimensional model 58 includes a virtual pipette 60. Figure 2 As shown in , the virtual pipette 60 can be positioned in the model 58 so that when the virtual reality headset 14 is adapted for generating augmented reality, the user sees the virtual pipette 60 as an extension of the motion sensing controller 16.

[0079] It is also possible that the three-dimensional model includes liquids 62, 64 as virtual components 22'. As an example, a virtual liquid 62 is shown in a virtual reservoir 28' and another virtual liquid 64 is shown in a virtual pipette. When these virtual liquids 62, 64 are representing different types of liquids (such as samples and reagents), they can be colored differently.

[0080] Additionally the virtual pipette 60 may be colored differently to indicate different types of pipette tips.

[0081] Figure 2 Additionally shown are a path 66 of the virtual pipette 60 in the three-dimensional model 58 and tracking points 68 a , 68 b , 68 c , 68 d on the path 66 , which can be recorded, for example, when a user presses the button 44 .

[0082] Figure 3 Another virtual pipette 60' is shown, which is modeled as a multichannel pipette 60' comprising a plurality of pipette tips 70. The method described above and below is not limited to a pipette with one tip. It can also be performed with a virtual pipette with a plurality of tips (e.g., 8, 12, 96, 384 tips).

[0083] Figure 4A flow chart of a method for generating a control program 54 for a laboratory automation device 12 and optionally for controlling the laboratory automation device 12 by executing the control program 54 is shown.

[0084] Initially, a user (e.g., a laboratory assistant) may manually arrange at least some of the components 22 in the laboratory automation device 12. For example, a user may manually arrange at least some of the components 22 in the laboratory automation device 12 at the workbench 20 (see Figure 1 ) are arranged on a container 22a having a disposable pipette tip 24, a container 22b having a sample tube 22b, a microtiter plate 22c and a reagent container 22d.

[0085] In step S10 , the method can then be started by starting a corresponding computer program in the computing device 18 .

[0086] The computing device 18 may then request a first portion of the configuration data 46 from the laboratory automation device 12, which may then determine the first portion of the configuration data 46 using one or more sensors 34. For example, a laser scanner and / or camera may determine the type, location, and / or orientation of the component 22. Additionally, a barcode scanner may scan a barcode on the component 22 and / or an RFID scanner may be used to determine its type and / or content.

[0087] For each component, the configuration data 46 may encode the type of component 22, the location of the component 22, the orientation of the component 22, and / or the content of the component 22. It must be noted that disposable tips 24, sample tubes 26, and reservoirs 28 may also be considered components 22 that are subcomponents of components 22a to 22c. For example, the configuration data 46 may be provided in a tree structure that illustrates the arrangement of components 22 within other components 22.

[0088] It is also possible that the configuration data 46 specify the type of liquid contained in the component 22. This type may be the content or may be derived from the content of the respective component 22.

[0089] A first part of the configuration data 46 relating to components 22 that can be arranged by a user can be regarded as configuration data for a variable layout of the laboratory automation device. A second part of the configuration data 46' relating to fixed components of the laboratory automation device 12 that cannot be removed or moved by the user (i.e., a fixed layout) can be stored directly in the computing device 18. However, this part can also be sent from the laboratory automation device 12 to the computing device 18. For example, the workbench 20 can be encoded as a fixed component in the configuration data 46'.

[0090] In step S12 , the configuration data 46 , 46 ′ of the laboratory automation device 12 are received in the computing device 18 , which then generates a three-dimensional model 58 of the component 22 of the laboratory automation device 12 based on the configuration data 46 , 46 ′.

[0091] In the computing device 18, modeling data 72 may be stored, which, for each type of component 22, encodes a geometrical layout for the corresponding component 22. The modeling data 72 may encode coordinates, surfaces, line models, etc. of the corresponding geometrical layout.

[0092] Using the modeling data 72 , the computing device 18 may generate the three-dimensional model 58 by moving and orienting the corresponding modeling data 72 using the positions and orientations encoded in the configuration data 46 , 46 ′.

[0093] The three-dimensional model 58 additionally comprises a virtual pipette 60. Also for this virtual pipette 60, modeling data may be stored in the computing device 18.

[0094] The user may now put on the virtual reality headset 14 and may pick up the motion sensing controller 16 .

[0095] In step S14, the computing device 18 receives the movement data 48 from the virtual reality headset 14 and determines the position and orientation of the user's head. From this, the view direction and field of view can be determined. Using the field of view and view direction, the computing device can render a corresponding scene for each eye of the user and / or can generate display data 52. The display data 52 can be generated using a rendering engine. When the corresponding view of the three-dimensional model 58 is then displayed, the display data 52 is then sent to the virtual reality headset 14, for example, Figure 2 as shown in .

[0096] The user now sees in virtual reality at least a portion of the laboratory automation device 12. In particular, he or she sees virtual components 20', 22', 24', 26', etc. that may appear similar to the corresponding real components 20, 22, 24, 26, etc. Modeling data 72 may be provided so that the virtual components may appear the same or similar to their real counterparts.

[0097] As already mentioned, different types of virtual liquids 62, 64 can be visualized differently in the three-dimensional model, for example using different colors. This can also be the case when the corresponding real liquids look quite similar. This can help the user to better distinguish the virtual liquids 62, 64 from each other. It is also possible that when the virtual liquid 64 is added to the other liquids 62 already present, a new type of liquid corresponding to the mixture is associated with the corresponding component 22', and this new type of liquid can be visualized with another different color.

[0098] In step S14, the computing device 18 also receives movement and activation data 50 from the motion sensing controller 16. The movement data 50 indicates the three-dimensional movement of the motion sensing controller 16 in space and the computing device 18 can determine the position and / or orientation of the virtual pipette 60 in the three-dimensional model 58. A view of the three-dimensional model 58 is displayed to the user together with the virtual pipette 60, and the user sees the virtual pipette 60 moved by him through the three-dimensional model 58.

[0099] In general, the movement of the virtual pipette 60 in the three-dimensional model 58 is determined based on the movement data 50 , and the three-dimensional model 58 is updated according to the movement of the virtual pipette 60 .

[0100] The computing device 18 may record the path 66 of one or more tips of the virtual pipette. This path 66 may later be used to generate the control program 54. It is also possible that the path 66 is displayed in the three-dimensional model 58. This may help the user to verify whether he has performed the task correctly.

[0101] The activation data 50 from the motion sensing controller 16 may indicate a user's finger presses and / or finger movements on the button 44 of the motion sensing controller 16. Whenever the user presses the button 44, it may be assumed that the virtual pipette 60 aspirates the liquid 62, 64 when its tip is within the virtual liquid 62, 64. Accordingly, it may be assumed that when the button is released, the liquid 62, 64 in the pipette 60 is discharged into the virtual component 22' where the tip of the pipette 60 is located. However, the activation data 50 may be evaluated in another manner, for example, a short button press may result in aspiration or discharge, regardless of whether the virtual pipette 60 is filled or not.

[0102] Generally speaking, aspiration and / or discharge of the virtual liquid 62 , 64 in the three-dimensional model 58 is determined based on the position of the virtual pipette 60 in the three-dimensional model 58 at which the activation data 50 indicates activation of the motion sensing controller 16 .

[0103] It must be noted that the three-dimensional model 58 can also show virtual liquids 62, 64 in the virtual pipette 60. In this way, when the virtual pipette 60 filled with liquids 62, 64 is moved, the movement of the liquid actually sucked in and discharged with the virtual pipette 60 can be performed in the three-dimensional model 58 and can be displayed in the virtual reality headset 14.

[0104] Furthermore, it is possible that the virtual assembly 22 ′ includes a virtual disposable tip 24 ′. In this case, the installation and movement of the disposable tip 24 can be performed in the three-dimensional model 58 and displayed in the virtual reality headset 14 .

[0105] For example, when the mounting portion of the virtual pipette 60 is located at the position of the virtual disposable tip 24', it can be determined that the virtual disposable tip 24' is mounted to the virtual pipette 60. The corresponding activation data 50 (such as double-clicking the button 44) can indicate the removal of the virtual disposable tip 24'.

[0106] The computing device 18 can also record tracking points 68a, 68b, 68c, 68d on the path 66. Whenever a specific event occurs (such as the installation or removal of the disposable tip 24' (tracking point 68a, see Figure 2 ), the suction of liquid 62, 64 (tracking points 68b, 68d), the discharge of liquid 62, 64 (tracking point 68c)), such tracking points can be recorded. Tracking point 68a can be a mounting point, tracking points 68b, 68d can be suction points and tracking point 68c can be a discharge point.

[0107] Generally speaking, an event occurs when the content and / or configuration of the virtual pipette 60 changes. It is also possible that an event occurs when the content of the virtual component 22' changes.

[0108] Tracking points 68a, 68b, 68c, 68d can encode the location of an event and / or the type of event. In the case of an event, the location can be encoded as a three-dimensional coordinate and / or as a component 22'. The type of event can be encoded with the type of liquid 62, 64 sucked or released by the virtual pipette.

[0109] The tracking points 68a, 68b, 68c, 68d may be used to generate the control program 54. It is also possible that the tracking points are displayed in the three-dimensional model 58. This may additionally help the user to verify whether he has performed the task correctly.

[0110] When the user has completed his or her task in virtual reality, he can take off the virtual reality headset 14 and can command the computing device 18 (or a corresponding computer program running in the computing device 18) to generate the control program 54.

[0111] In step S16 , a control program 54 for the laboratory automation device 12 is then generated from the movement and activation data 50 and in particular from the information derived therefrom, such as the path 66 and / or the tracking points 68 a , 68 b , 68 c , 68 d .

[0112] In general, the control program 54 is generated so that it is adapted to move the pipette arm 30 with the pipette 32 of the laboratory automation device 12 relative to the component 22 according to the movement of the virtual pipette 60 in the three-dimensional model 58, so as to control the pipette 32 of the pipette arm 30 so as to aspirate and discharge liquid according to the virtual pipette 60 in the three-dimensional model 58 and / or so as to install and remove the disposable pipette tip 24 according to the virtual pipette 60 in the three-dimensional model 58.

[0113] In one example, the control program 54 can be generated so that only the aspiration and discharge of the virtual pipette 60, the optional installation and removal of the disposable tip 24 and the movement to the corresponding position are repeated by the pipetting arm 30 with the pipette 32. The control program does not have to know the contents and / or liquids in the corresponding components.

[0114] In another example, the user performs his or her task for one virtual sample 26', i.e. records the movement and activation data 50 for one virtual sample and generates a control program 54 so that it repeats the movement of the pipette 32 arm and / or the aspiration and discharge of the pipette 32 for a plurality of real samples 26 in the laboratory automation device 12. This can be achieved, for example, by simply moving the position where the aspiration and discharge of the pipette 32 and optionally the installation and removal of the disposable tip take place to the next adjacent position at the corresponding component 22.

[0115] In another example, the movement of the pipetting arm 30 and the pipette 32, the aspiration and discharge of the pipette 32, and / or the installation and removal of the disposable tip 24 are determined according to the tracking points 68a, 68b, 68c, 68d. According to the events associated with the tracking points, corresponding commands for the control program 54 can be derived. For example, the tracking point 68a can be converted into the command "install movable tip" and the tracking point 68a can be converted into "longing for sample", etc.

[0116] In this case, the control program 54 can also be generated so that it repeats the task for several samples. For example, it is possible that the configuration data 46 may contain information for a plurality of real liquids 29 and the task is performed for one of the virtual liquids 62. Then the control program 54 can be generated to perform the task for all samples 26.

[0117] In step S18, the control program 54 may optionally be modified into a control program 54'. For example, the generated control program 54 may be a script, which may be further modified by a user. For example, a repeating control structure may be inserted into the control program 54 by a user. It is also possible that additional steps (e.g., incubation of a sample) are included in the control program 54.

[0118] In step S20, the control program 54 or 54' is executed by the computing device 18, for example, when the user instructs the computer program in the computing device to execute the control program 54, 54'. This can be done several times. For example, when the control program 54, 54' has been completed, the user can arrange new components 22 in the laboratory automation device 12 according to the same layout and can start the control program 54, 54' again.

[0119] When the control programs 54, 54' are executed, control commands 56 are generated and the laboratory automation device 12 performs the task that has been designed by the user in virtual reality. In particular, the same task performed in virtual reality for one sample 26' can be performed multiple times for multiple samples 26 using the laboratory automation device 12.

[0120] Although the present invention has been illustrated and described in detail in the drawings and the above description, such illustration and description are to be considered illustrative or exemplary rather than restrictive; the present invention is not limited to the disclosed embodiments. From a study of the drawings, the disclosure and the appended claims, a technician experienced in the art and practicing the claimed invention can understand and implement other variations of the disclosed embodiments. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or controller or other unit may perform the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any figure marks in the claims should not be interpreted as limiting the scope.

[0121] List of reference numbers

[0122] 10 Control System

[0123] 12 Laboratory Automation Equipment

[0124] 14. Virtual Reality Headsets

[0125] 16 Motion Sensing Controller

[0126] 18 Computing Devices

[0127] 20 Workbench

[0128] 22 Components

[0129] 22a Container

[0130] 22b Container

[0131] 22c Microtiter Plate

[0132] 22d Reagent container

[0133] 24 Disposable Pipette Tips

[0134] 26 Sample tubes

[0135] 28 Storage

[0136] 29 Liquid

[0137] 30 Pipetting arm

[0138] 32 Pipette

[0139] 34 Sensors

[0140] 36 Display

[0141] 38 Motion Sensor

[0142] 40 Handle

[0143] 42 Motion Sensor

[0144] 44 Buttons

[0145] 46 Configuration Data

[0146] 46' Configuration data

[0147] 48 Mobile Data

[0148] 50 Moving and Activating Data

[0149] 52 Display data

[0150] 54 Control Program

[0151] 54' Modified Control Procedure

[0152] 56 Control Commands

[0153] 58 3D Models

[0154] 20' Virtual Workbench

[0155] 22' Virtual Components

[0156] 22a' Virtual Container

[0157] 22b' Virtual Container

[0158] 22c' Virtual microtiter plate

[0159] 22d' Virtual reagent container

[0160] 24' Virtual Disposable Pipette Tips

[0161] 26' Virtual Sample Tube

[0162] 28' Virtual Storage

[0163] 60 Virtual Pipette

[0164] 62 Virtual Liquid

[0165] 64 Virtual Liquid

[0166] 66 Path

[0167] 68a Tracking Point

[0168] 68b Tracking Point

[0169] 68c Tracking Point

[0170] 68d Tracking Point

[0171] 60' Multichannel Virtual Pipette

[0172] 70 Pipette Tips

[0173] 72 Modeling Data

Claims

1. A method for generating a control program (54) for a laboratory automation device (12), the method comprising: receiving configuration data (46) of the laboratory automation device (12), the configuration data (46) encoding a position of a component (22) in the laboratory automation device (12); Generating a three-dimensional model (58) of a component (22) of the laboratory automation device (12) based on the configuration data (46), the three-dimensional model (58) also including a virtual pipette (60); displaying the three-dimensional model (58) using a virtual reality headset (14); receiving movement data of a motion sensing controller (16) controlled by a user wearing the virtual reality headset (14), the movement data indicating three-dimensional movement of the motion sensing controller (16) in space; Determining the movement of the virtual pipette (60) in the three-dimensional model (58) based on the movement data and updating the three-dimensional model (58) based on the movement of the virtual pipette (60); receiving activation data from the motion sensing controller (16), the activation data indicating movement of a finger of the user on the motion sensing controller (16); determining aspiration and / or discharge of liquid (62, 64) in the three-dimensional model (58) based on a position of the virtual pipette (60) in the three-dimensional model (58) at which the activation data indicates activation of the motion sensing controller (16); A path (66) of one or more tips of the virtual pipette (60) and a plurality of tracking points on the path (66) recorded in the three-dimensional model (58), the plurality of tracking points encoding the location of an event and / or the type of event; A control program (54) for the laboratory automation device (12) is generated based on the movement data and the multiple tracking points, wherein the control program (54) is adapted to move a pipette arm (30) with a pipette (32) of the laboratory automation device (12) relative to the component (22) according to the movement of the virtual pipette (60) in the three-dimensional model (58), and wherein the control program (54) is adapted to control the pipette (32) of the pipette arm (30) so as to aspirate and discharge liquid according to the virtual pipette (60) in the three-dimensional model (58).

2. The method according to claim 1, in, The tracking points include an intake point (68a, 68d) and a discharge point (68c) for the liquid (62, 64), the intake point (68a, 68d) and the discharge point (68c) being determined based on the movement data and the activation data; The movement of the pipetting arm (30) is determined according to the suction point (68a, 68d) and the discharge point (68c).

3. The method according to claim 1 or 2, in, The movement of the liquid (62, 64) being virtually aspirated and discharged with the virtual pipette (60) is performed in the three-dimensional model (58) and displayed in the virtual reality headset (14).

4. The method according to claim 1 or 2, in, The assembly (22) includes a disposable pipette tip (24), and installation and movement of the disposable pipette tip is performed in the three-dimensional model (58) and displayed in the virtual reality headset (14).

5. The method according to claim 1 or 2, in, For each component (22), the configuration data (46) encodes the type of component (22) and the location of the component (22) in the laboratory automation device (12); The three-dimensional model (58) is generated based on modeling data (72) encoding the geometric layout of each component (22).

6. The method according to claim 1 or 2, in, The assembly (22) includes at least one of: a reservoir (28), a microtiter plate (22c), a reagent container (22d), and a sample tube (26).

7. The method according to claim 1 or 2, in, The type of liquid (62, 64) contained in the component (22) is specified in the configuration data (46) of the laboratory automation device (12); Therein, different types of liquids (62, 64) are visualized differently in the three-dimensional model (58).

8. The method according to claim 1 or 2, further comprising: manually arranging at least some of the components (22) in the laboratory automation device (12); At least some of the configuration data (46) are determined using sensors (34) of the laboratory automation device (12).

9. The method according to claim 1 or 2, in, recording movement data and / or activation data for a virtual sample; The control program (54) is generated so that it repeats the movement of the pipetting arm (30) and / or the suction and discharge of the pipette (32) for a plurality of real samples in the laboratory automation device (12).

10. The method according to claim 1 or 2, in, The motion sensing controller (16) is adapted to be held in the hand of a user wearing the virtual reality headset (14); wherein the motion sensing controller (16) includes a motion sensor (42) for generating the movement data; The motion sensing controller (16) includes a button (44) for generating activation data.

11. The method according to claim 1 or 2, in, The virtual pipette (60) is a multi-channel pipette comprising a plurality of pipette tips (70).

12. A computer-readable medium having stored therein a computer program for generating a control program (54) for a laboratory automation device (12), said computer program being adapted to perform the steps of the method according to one of claims 1 to 11 when executed by a processor.

13. A control system (10) for a laboratory automation device (12), the system comprising: The laboratory automation equipment (12); Virtual reality headsets (14); Motion sensing controller (16); A computing device (18) communicatively interconnected with the laboratory automation device (12), the virtual reality headset (14) and the motion sensing controller (16) and adapted to perform the method according to one of claims 1 to 11.

Images