Video-Based Pipette Tracking for Lab Automation Programming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The programming of laboratory automation devices is complex and requires specialized skills, limiting laboratory assistants to perform procedures with only a small number of samples due to the need for scripting languages and manual movement of robot arms for teach modes.
Innovation Solution
A method using a camera to record and evaluate video data of a laboratory assistant's movements, allowing for the generation of a control program for laboratory automation devices by detecting pipette tips and container positions, enabling the automation of pipetting actions and labware movements, and adapting the robot pipette's movements based on the assistant's actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If programming is done using scripting languages and manual teach mode, then control precision is improved, but device complexity and difficulty of operation increase
Solution Approach 1:
The system records video data of the laboratory assistant's manual pipetting actions and creates a digital copy of these movements. The control program is generated by copying the observed manual actions into automated robot arm movements, eliminating the need for complex scripting while preserving operational precision.
Solution Approach 2:
The patent replaces manual mechanical teaching operations with automated video recording and image processing. Instead of manually moving the robot arm through teach mode, the system uses a camera to record and analyze the assistant's manual pipetting actions, then automatically generates control commands to replicate these actions.
2Manufacturing precision
If specialized programming skills are required, then control precision is improved, but ease of operation deteriorates
Solution Approach 1:
The system performs self-programming by automatically generating control commands from video recordings of manual operations. The laboratory assistant does not need to write programming code; instead, the system observes the assistant's actions and autonomously creates the control program, making the process accessible to non-programmers while maintaining precision.
Solution Approach 2:
The patent introduces video recording and image processing as an intermediary between the laboratory assistant's manual actions and the robot arm's automated execution. This intermediary layer translates human movements into machine commands without requiring the user to learn programming languages, bridging the gap between manual operation and automated control.
3Ease of operation
If manual teach mode is used to program robot arm positions, then ease of operation is improved, but productivity decreases
Solution Approach 1:
The system enables continuous operation by recording video data during normal laboratory work and generating control programs without interrupting the workflow. The laboratory assistant can perform manual pipetting while the system simultaneously records and processes the data, then generates automated control commands that can be executed immediately, eliminating downtime associated with traditional programming methods.
Data Source
Figure 1
Figure 2~3
AI summary
A method for generating a control program (52) for a laboratory automation device (12) comprises: receiving video data (48) displaying a work area (36) of a laboratory assistant, the work area (36) containing a hand-held pipette (16) and containers (22') for receiving a liquid; detecting openings (56) of the containers (22') in the video data (48) and determining positions of the openings (56); detecting a pipette tip (40) of the hand-held pipette (16) in the video data (48) and determining a movement (58) of the tip (40); and generating the control program (52) for the laboratory automation device (12) from the movement of the pipette tip (40) with respect to the positions of the openings (56), wherein the control program (52) is adapted for moving a pipetting arm (30) with a robot pipette (32) of the laboratory automation device (12) with respect to containers (22) of the laboratory automation device (12) accordingly to the movement of the hand-held pipette (16) in the work area (36).