Medicament preparation device with imaging enhanced preparation process component placement

Abstract

A drug preparation device (PPD) with imaging enhanced preparation process component placement, the drug preparation device comprising: a camera configured to capture an image of a PPIL from a pharmaceutical formulation interconnect site (PPIL) camera site, the PPIL being associated with pre-calibration data; a delivery unit (TU) comprising a gripper for gripping the medication dispensing component, the TU being configured to move at least the gripper in one or more of x, y and / or z directions in response to a control signal; and processing circuitry operably connected to the TU and the camera, the processing circuitry configured to: receive, at a current location in the PPIL, at least one real-time digital image captured by the camera from a PPIL camera site, the received digital image depicting a gripper of the TU and / or a pharmaceutical formulation component gripped by the gripper; determining a direction and distance of movement of the TU based at least on the received real-time digital image and pre-calibration data; and controlling the TU via a control signal to move the gripper in the determined direction by the determined distance.

Description

Technical Field

[0001] The subject matter of the present disclosure relates to robotic preparation equipment for drugs and the movement of components within such equipment, and more particularly to enhanced imaging placement of such components. Background Art

[0002] In the conventional field, the implementation problems of drug preparation automation have been recognized, and various techniques have been developed to provide solutions. Some solutions include automatic or semi-automatic drug preparation equipment and systems for preparing drugs designated for administration to patients. These equipment and systems include fluid transfer stations for transferring fluids between drug formulation components, and robotic arms for gripping and moving drug formulation components between stations. Summary of the Invention

[0003] The subject matter of the present disclosure generally relates to robotic drug preparation equipment and / or systems. The robotic drug preparation equipment and its fluid transfer stations are configured to perform operations related to the transfer of drugs between different drug formulation components (or preparation process components), which include containers, fluid transfer assemblies, connectors, conduits, pumps, syringes, vials, intravenous bags, adapters, needles, ampoules, etc. The robotic drug preparation equipment (or robotic device) according to the subject matter of the present disclosure includes a robotic station, a robotic arm, a motor, a control unit (controller), mechanisms, transfer units, and manipulators to move drug formulation components relative to each other and control the fluid transfer therebetween. The robotic drug preparation equipment can be operated to perform any activity related to the preparation of drugs (such as drugs designated for administration to patients), including, for example, compounding, diluting, reconstituting, transferring, filling, aspirating, stirring, and / or other processes associated with drug preparation.

[0004] The robotic drug preparation equipment is configured to receive and optionally manipulate various types of containers, such as drug vials, intravenous (IV) bags, syringes, tubes, elastomeric pumps, and / or other containers suitable for containing and / or transferring fluids and / or powders. In some examples, the robotic drug preparation equipment is configured to: receive at least one drug vial; dilute or reconstitute the drug in the vial as needed; optionally, stir the vial; and then obtain a limited quantity of ready-to-use drug by aspirating from the vial. In some cases, the drug is then prepared for administration to the patient, for example, by transferring the drug to a syringe, an elastomeric pump, an IV bag, or any other suitable container.

[0005] The pharmaceutical preparation device can be deployed for preparing any type of medicine, including hazardous and non-hazardous medicines prepared in a closed system. In a closed fluid transfer device or system deployed for preparing hazardous drugs or medicines, measures are taken to prevent hazardous leakage of fluids and / or fumes from containers and / or further prevent contaminants from infiltrating into the medicines. To ensure sterility, alignment of containers, and provide a secure connection during fluid transfer, connectors or adapters can be used with containers and / or are typically used at the fluid transfer interfaces of the device.

[0006] The pharmaceutical preparation device according to the subject matter of the present disclosure utilizes imaging to enhance the placement of preparation process components to ensure that the preparation process components or pharmaceutical formulation components are aligned with each other before and during the transfer of fluid (medicine) between them. It should be understood herein that for the purposes of this specification, any two containers between which fluid transfer is to occur are referred to as preparation process components or pharmaceutical formulation components, and their placement and / or alignment relative to the device and / or each other are enhanced.

[0007] According to a first aspect of the subject matter of the present disclosure, for example, there is provided a pharmaceutical preparation device having imaging-enhanced preparation process component placement, the pharmaceutical preparation device including: a camera configured to capture an image of a Pharmaceutical Preparation Interconnection Location (PPIL) from a PPIL camera location, the PPIL being associated with pre-calibrated data; a transfer unit (TU) including a gripper for gripping a pharmaceutical formulation component, the TU being configured to move at least the gripper in one or more of the x, y, and / or z directions in response to a control signal; and a processing circuit operably connected to the TU and the camera, the processing circuit being configured to: at a current position in the PPIL, receive at least one real-time digital image captured by the camera from the PPIL camera location, the received digital image depicting the gripper of the TU and / or the pharmaceutical formulation component gripped by the gripper; determine the direction and distance of movement of the TU based at least on the received real-time digital image and the pre-calibrated data; and control the TU via the control signal to move the gripper the determined distance in the determined direction.

[0008] For the purposes of this specification, it should be understood herein that a "Pharmaceutical Preparation Interconnection Location (PPIL)" includes a location within the camera's field of view where the gripper of the TU and / or the pharmaceutical formulation component gripped by the gripper is positioned for establishing an interconnection between a pharmaceutical formulation component and another pharmaceutical formulation component, between their respective connectors, or between a pharmaceutical formulation component and its connector. The camera's field of view is the field of view of the camera when the camera is positioned at a specific location, herein referred to as the PPIL camera location in the device.

[0009] For the purposes of this specification, it should also be understood herein that a "delivery unit" includes an apparatus for gripping, holding, and moving any drug formulation components within the device, the apparatus including a manipulator in the form of a robotic arm, platform, robotic station, etc., the manipulator having a holder for holding the components and moving the components relative to each other and performing fluid transfer. The manipulator can be a fluid transfer assembly manipulator, such as a syringe manipulator, or a container manipulator, such as a vial manipulator or an IV bag manipulator.

[0010] For the purposes of this specification, it should also be understood herein that a "holder" includes an apparatus for gripping, holding, or otherwise engaging a drug formulation component. The holder can be a gripping arm of a fluid transfer assembly manipulator, a vial holder, an IV bag holder, and any other suitable apparatus for holding any drug formulation component. The holder can hold the drug formulation component from any of its parts, including the body or a connector / adapter attached to the holder.

[0011] For the purposes of this specification, it should also be understood herein that a "drug formulation component" includes any container that is a component of a fluid transfer device, the fluid transfer device having or not having an adapter or connector for establishing fluid communication between the container and other containers. For example, the container can form a container assembly that has the container and a container connector (or adapter) for establishing fluid communication between the container and other containers. For example, the container can be a vial together with a vial adapter, or an intravenous bag together with a spike adapter, or a syringe together with a syringe adapter. For example, the container can be one or more of the following: a syringe, an IV bag, an elastomeric pump, a vial, a bottle, an ampoule, or any vessel or container generally suitable for containing a fluid or liquid. The container can be accessible via a container septum, which can be a container cap, a septum of a container port, or can be part of a connector.

[0012] For the purposes of this specification, it should also be understood herein that "pre-calibration data" includes one or more images and / or processing data obtained during image processing of one or more images of the PPIL, the one or more images and / or processing data being used by a processing circuit, together with information regarding the desired position (also referred to herein as the calibration position) of the holder and / or the drug formulation component held by the holder, to determine the required movement of the holder to bring the holder and / or the drug formulation component held by the holder from the current (real-time) position in the PPIL to the desired position. The pre-calibration data is collected before the actual real-time use of the device and is stored in a memory. During the real-time use of the device, the processing circuit uses the pre-calibration data to determine whether the holder and / or the drug formulation component is in the desired position, and if not, generates a control signal for causing the transfer unit to move the holder from its current position to the desired position.

[0013] For the purposes of this specification, it should be understood herein that the desired position represents a position in the PPIL where the gripper and / or the drug formulation component is / are intended to be positioned for interconnection with another container and / or where drug transfer is to be performed, or where the gripper is intended to be positioned for gripping the drug formulation component. Further, the current position represents a position in the PPIL where the gripper and / or the drug formulation component is / are currently (i.e., in real time or during the use of the device for performing operations related to moving and positioning the gripper) positioned when the camera captures a real-time image from the PPIL location.

[0014] In some cases, the pre-calibration data may include two-dimensional (2D) or three-dimensional (3D) pre-calibration images captured by the camera from the PPIL camera location, the 2D or 3D pre-calibration images depicting the gripper in a calibration position (also referred to herein as the desired position) and / or the drug formulation component gripped by the gripper. Once a real-time image of the PPIL during device use is obtained, the processing circuit determines, based on the pre-calibration image and the real-time image, whether the gripper and / or the drug formulation component is in the desired position, and if not, generates a control signal for causing the transfer unit to move the gripper from its current position to the desired position. The control signal may be generated based on a comparison of the pre-calibration image with the real-time image to determine a pixel offset between the current position and the calibration position. Further, based on the pixel offset, the direction and distance of movement of the TU are determined. The direction and distance of movement of the TU represent the direction and distance by which the TU needs to move the gripper to position the gripper and / or the drug formulation component in its desired position.

[0015] In some cases, the pre-calibration data may include a calibration transformation matrix, which is a derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the drug formulation component gripped by the gripper, the calibration transformation matrix being adapted to convert the x, y, and / or z coordinates of the image to corresponding coordinates in the reference frame of the delivery unit.

[0016] It should be understood herein that the delivery unit is at a particular location and is controlled by a controller to move the delivery unit (or its gripper) various distances in the x, y, and z directions. These distances are the coordinates of the reference frame of the delivery unit.

[0017] In some examples, the processing circuitry is configured to perform a calibration process to derive a calibration transformation matrix. The calibration process includes receiving at least three images of the PPIL captured by the camera from the PPIL camera location, each image depicting the gripper and / or the drug formulation component held by the gripper. In some examples, the images may be 2D images, and the calibration process includes receiving at least three images. In some examples, the images may be 3D images, and the calibration process includes receiving at least four images. Each image is associated with an x-coordinate, a y-coordinate, and a z-coordinate in the delivery unit reference frame that correspond to a given point on the gripper or the drug formulation component held by the gripper. These x, y, and z coordinates can be obtained as follows:

[0018] - Place the gripper and / or the drug formulation component held by the gripper in a specific location in the PPIL;

[0019] - The processing circuitry receives an image of the gripper and / or the drug formulation component held by the gripper and the PPIL; and

[0020] - The processing circuitry also receives information from the TU indicating that the gripper and / or the drug formulation component held by the gripper is now located at x1, y1, and z1.

[0021] The processing circuitry performs the calibration process to obtain data indicative of a calibration transformation matrix that is adapted to convert the x, y, and z coordinates of the image to corresponding coordinates in the delivery unit reference frame.

[0022] In some examples, the pre-calibration data may further include the desired position of the gripper and / or the drug formulation component in the PPIL in the delivery unit reference frame (TUFoR), and thus, the processing circuitry may have desired coordinates (in the TUFoR), i.e., the coordinates of the position where the gripper and / or the drug formulation component needs / desires to be positioned. In some examples, the desired coordinates (in the TUFoR), i.e., the coordinates of the position where the gripper and / or the drug formulation component needs / desires to be positioned, may be received by the processing circuitry from an external source.

[0023] During real-time use of the device, a real-time digital image depicting the PPIL and the gripper or a drug formulation component held by the gripper (in its current position) is captured by a camera from the PPIL location and received by a processing circuit. The processing circuit applies a calibration transformation matrix to the received real-time digital image to determine the direction and distance of TU movement, which represents the direction and distance that the TU needs to move the gripper to position the gripper and / or the drug formulation component at its desired position. Applying the calibration transformation matrix by the processing circuit to the received real-time digital image includes: identifying the x, y, and z coordinates of a given point on the gripper or the drug formulation component held by the gripper in the received image; and using the calibration transformation matrix to transform the identified x, y, and z coordinates in the received image into x, y, and z coordinates in the transport unit reference frame. The processing circuit determines the direction / distance of TU movement by comparing the transformed x, y, z coordinates with the desired coordinates (in the TU reference frame).

[0024] In some examples, the processing circuit may be further configured to use the identification information to determine the direction / distance of TU movement to position the gripper and / or the drug formulation component at the desired position. The identification information may be related to the identification of one or more of the following: the gripper, the drug formulation component grasped by the gripper, another drug formulation component to be interconnected with the drug formulation component, or the drug formulation component to be grasped by the gripper. It should be understood herein that the identification information may be determined by the processing circuit, for example, through image processing, or may be received from an external source.

[0025] In some examples, the processing circuit may be further configured to control the gripper of the TU to perform at least one of the following:

[0026] Grasp a drug formulation component;

[0027] Release the drug formulation component that the gripper is grasping; and

[0028] Establish a fluid connection between a second drug formulation component and the drug formulation component that the gripper is grasping.

[0029] According to a second aspect of the subject matter of the present disclosure, for example, there is provided a method for enhancing the placement of preparation process components in a pharmaceutical preparation device (PPD) based on a processing circuit, the method comprising: utilizing a transport unit (TU) configured to move a gripper of at least the transport unit in one or more of the x, y, and / or z directions in response to a control signal; utilizing pre-calibration data; at a current position, receiving at least one real-time digital image of a pharmaceutical preparation interconnection location (PPIL) captured by a camera located at a PPIL camera location, the received digital image depicting the gripper of the TU and / or a pharmaceutical formulation component grasped by the gripper; determining at least based on the received real-time digital image and the pre-calibration data a direction and a distance of movement of the TU; and controlling the TU to move the gripper the determined distance in the determined direction.

[0030] According to a third aspect of the subject matter of the present disclosure, for example, there is provided a computer program product comprising a computer-readable non-transitory storage medium containing program instructions that, when read by a processor, cause a processing circuit to execute a method for enhancing the placement of preparation process components in a pharmaceutical preparation device (PPD), the method comprising: utilizing a transport unit (TU) configured to move a gripper of at least the transport unit in one or more of the x, y, and / or z directions in response to a control signal; utilizing pre-calibration data; at a current position, receiving at least one real-time digital image of a pharmaceutical preparation interconnection location (PPIL) captured by a camera located at a PPIL camera location, the received digital image depicting the gripper of the TU and / or a pharmaceutical formulation component grasped by the gripper; determining at least based on the received real-time digital image and the pre-calibration data a direction and a distance of movement of the TU; and controlling the TU to move the gripper the determined distance in the determined direction.

[0031] In some examples, in either the second or third aspect, the pre-calibration data may include two-dimensional (2D) or three-dimensional (3D) pre-calibration images captured by the camera from the PPIL camera location, the two-dimensional (2D) or three-dimensional (3D) pre-calibration images depicting the gripper of the TU and / or a pharmaceutical formulation component grasped by the gripper of the TU in a calibration position.

[0032] In some examples, in either the second or third aspect, the method may further include determining that there is a pixel offset between the current position and the calibration position before determining the direction and distance of movement of the TU. In some examples, in either the second or third aspect, the method may include determining the direction and distance of movement of the TU at least based on the pixel offset.

[0033] In some examples, in either the second or third aspect, the pre-calibration data may include a calibration transformation matrix, which is the derivative of a calibration process that utilizes at least three images from the PPIL camera location, where the at least three images depict the gripper of the TU and / or the drug formulation components grasped by the gripper. The calibration transformation matrix is adapted to convert the x, y, and / or z coordinates of the images into corresponding coordinates in the reference frame of the delivery unit. The method may include determining the direction and distance of TU movement at least by applying the calibration transformation matrix to the received real-time digital image. In some examples, in either the second or third aspect, the method may include performing a calibration process.

[0034] It should be understood herein that the descriptions of the components and features of the drug preparation device in the first aspect provided above similarly apply to the corresponding components and features included in the method of the second aspect and the computer program product of the third aspect.

[0035] Embodiment

[0036] A more specific description is provided in the detailed description, while the following are non-limiting examples of different embodiments of the subject matter of the present disclosure.

[0037] 1. A drug preparation device (PPD) with imaging-enhanced placement of components for the preparation process, the drug preparation device comprising:

[0038] A camera configured to capture an image of the PPIL from a PPIL camera location, the PPIL being associated with pre-calibration data;

[0039] A delivery unit (TU) including a gripper for grasping a drug formulation component, the TU being configured to move at least the gripper in one or more of the x, y, and / or z directions in response to a control signal; and a processing circuit operably connected to the TU and the camera, the processing circuit being configured to:

[0040] At a current position in the PPIL, receive at least one real-time digital image captured by the camera from the PPIL camera location, the received digital image depicting the gripper of the TU and / or the drug formulation component grasped by the gripper;

[0041] Determine the direction and distance of TU movement based at least on the received real-time digital image and the pre-calibration data; and

[0042] Control the TU via the control signal to move the gripper the determined distance in the determined direction.

[0043] 2. The PPD according to embodiment 1, wherein the received digital image is two-dimensional (2D), and wherein the determined direction includes the x-direction and / or the z-direction.

[0044] 3. The PPD according to embodiment 1, wherein the received digital image is three-dimensional (3D), and wherein the determined direction includes the x-direction, the y-direction, and / or the z-direction.

[0045] 4. The PPD according to any one of embodiments 1 to 3, wherein the pre-calibration data includes a two-dimensional (2D) or three-dimensional (3D) pre-calibration image captured by the camera from the PPIL camera location, the two-dimensional (2D) or three-dimensional (3D) pre-calibration image depicting the gripper of the TU in a calibration position and / or a drug formulation component grasped by the gripper of the TU.

[0046] 5. The PPD according to embodiment 4, wherein the processing circuit is further configured to: determine that there is a pixel offset between the current position and the calibration position before determining the direction and the distance of the TU movement.

[0047] 6. The PPD according to embodiment 5, wherein the processing circuit is configured to: determine the direction and the distance of the TU movement based at least on the pixel offset.

[0048] 7. The PPD according to any one of embodiments 1 to 3, wherein the pre-calibration data includes a calibration transformation matrix, the calibration transformation matrix being a derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the drug formulation component grasped by the gripper, the calibration transformation matrix being adapted to convert the x, y, and / or z coordinates of an image to corresponding coordinates in the reference frame of the delivery unit.

[0049] 8. The PPD according to embodiment 7, wherein the processing circuit is configured to: determine the direction and the distance of the TU movement at least by applying the calibration transformation matrix to the received real-time digital image.

[0050] 9. The PPD according to embodiment 7 or 8, wherein the received digital image is two-dimensional (2D), and the pre-calibration data includes a calibration transformation matrix, the calibration transformation matrix being a derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the drug formulation component grasped by the gripper.

[0051] 10. The PPD according to embodiment 7 or 8, wherein the received digital image is three-dimensional (3D), and the pre-calibration data includes a calibration transformation matrix, which is the derivative of a calibration process that utilizes at least four images from the PPIL camera location, and the at least four images depict the gripper of the TU and / or the drug formulation component grasped by the gripper.

[0052] 11. The PPD according to any one of embodiments 7 to 10, wherein the processing circuit is configured to perform the calibration process.

[0053] 12. The PPD according to any one of embodiments 1 to 11, wherein the processing circuit is further configured to: control the gripper of the TU to perform at least one of the following:

[0054] Grasp a drug formulation component;

[0055] Release the drug formulation component being grasped by the gripper; and

[0056] Establish fluid communication between a second drug formulation component and the drug formulation component being grasped by the gripper.

[0057] 13. A method for enhancing the placement of components in an imaging-based drug preparation process in a drug preparation device (PPD) based on a processing circuit, the method comprising:

[0058] Utilize a transport unit (TU) configured to move at least the gripper of the transport unit in one or more of the x, y, and / or z directions in response to a control signal;

[0059] Utilize pre-calibration data;

[0060] At a current position, receive at least one real-time digital image of the PPIL captured by a camera positioned at a pharmaceutical preparation interconnection location (PPIL) camera location, the received digital image depicting the gripper of the TU and / or the drug formulation component grasped by the gripper;

[0061] Determine at least based on the received real-time digital image and the pre-calibration data the direction and distance of movement of the TU; and

[0062] Control the TU to move the gripper the determined distance in the determined direction.

[0063] 14. The method according to embodiment 13, wherein the pre-calibration data includes two-dimensional (2D) or three-dimensional (3D) pre-calibration images captured by the camera from the PPIL camera location, the two-dimensional (2D) or three-dimensional (3D) pre-calibration images depicting the gripper of the TU and / or the drug preparation component grasped by the gripper of the TU in a calibration position.

[0064] 15. The method according to embodiment 14, the method further comprising: determining that there is a pixel offset between the current position and the calibration position before determining the direction and the distance of the TU movement.

[0065] 16. The method according to embodiment 15, the method comprising: determining the direction and the distance of the TU movement based at least on the pixel offset.

[0066] 17. The method according to embodiment 13, wherein the pre-calibration data includes a calibration transformation matrix, the calibration transformation matrix being a derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the drug preparation component grasped by the gripper, the calibration transformation matrix being adapted to convert the x, y, and / or z coordinates of an image to corresponding coordinates in a reference frame of the delivery unit.

[0067] 18. The method according to embodiment 17, the method comprising: determining the direction and the distance of the TU movement by applying the calibration transformation matrix to the received real-time digital image at least.

[0068] 19. The method according to embodiment 18, the method further comprising: performing the calibration process.

[0069] 20. A computer program product, the computer program product comprising a computer-readable non-transitory storage medium containing program instructions that, when read by a processor, cause the processing circuit to execute a method for placing an imaging enhancement preparation process component in a drug preparation device (PPD), the method comprising:

[0070] utilizing a delivery unit (TU) configured to move at least a gripper of the delivery unit in one or more of the x, y, and / or z directions in response to a control signal;

[0071] utilizing pre-calibration data;

[0072] At the current position, receive at least one real-time digital image of the PPIL captured by a camera located at the PPIL camera location, the received digital image depicting the gripper of the TU and / or a drug formulation component grasped by the gripper;

[0073] Determine the direction and distance of TU movement based at least on the received real-time digital image and the pre-calibration data; and

[0074] Control the TU to move the gripper the determined distance in the determined direction.

[0075] 21. The computer program product according to embodiment 20, wherein the pre-calibration data includes a two-dimensional (2D) or three-dimensional (3D) pre-calibration image captured by the camera from the PPIL camera location, the two-dimensional (2D) or three-dimensional (3D) pre-calibration image depicting the gripper of the TU in a calibration position and / or a drug formulation component grasped by the gripper of the TU.

[0076] 22. The computer program product according to embodiment 21, wherein the method further includes: determining that there is a pixel offset between the current position and the calibration position before determining the direction and the distance of TU movement.

[0077] 23. The computer program product according to embodiment 22, wherein the method further includes: determining the direction and the distance of TU movement based at least on the pixel offset.

[0078] 24. The computer program product according to embodiment 20, wherein the pre-calibration data includes a calibration transformation matrix, the calibration transformation matrix being a derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the drug formulation component grasped by the gripper, the calibration transformation matrix being adapted to convert the x, y, and / or z coordinates of an image to corresponding coordinates in the reference frame of the delivery unit.

[0079] 25. The computer program product according to embodiment 24, wherein the method includes: determining the direction and the distance of TU movement by at least applying the calibration transformation matrix to the received real-time digital image.

[0080] 26. The computer program product according to embodiment 24, wherein the method further includes; performing the calibration process. Description of the Drawings

[0081] For a better understanding of the subject matter disclosed herein and to illustrate how the subject matter may be implemented in practice, embodiments will now be described by way of non-limiting example only with reference to the accompanying drawings, in which:

[0082] Figure 1A A perspective view of a portion of a pharmaceutical preparation device (PPD) exemplifying an example of the subject matter according to the present disclosure is shown;

[0083] Figure 1B Illustrates Figure 1A A side view of a portion of a pharmaceutical preparation device (PPD);

[0084] Figure 1C Schematically illustrates a pharmaceutical formulation component and another container exemplifying an example of the subject matter according to the present disclosure;

[0085] Figure 2 Schematically illustrates a portion of a pharmaceutical preparation device, showing preparation process interconnection locations and a camera positioned at a PPIL camera location;

[0086] Figures 3A to 3C Schematically illustrates an example positioning sequence of a pharmaceutical formulation component exemplifying an example of the subject matter according to the present disclosure;

[0087] Figure 3D And Figure 3E Illustrates a portion of a pharmaceutical preparation device exemplifying an example of the subject matter according to the present disclosure, depicting an example positioning sequence of a pharmaceutical formulation component;

[0088] Figure 4 Illustrates a block diagram of an example pharmaceutical preparation device utilizing imaging enhanced preparation process component placement exemplifying an example of the subject matter according to the present disclosure;

[0089] Figure 5 Is a flowchart of a specific example method of imaging enhanced preparation process component placement exemplifying an example of the subject matter according to the present disclosure;

[0090] Figure 6A Is an example 2D pre-calibration image exemplifying an example of the subject matter according to the present disclosure;

[0091] Figure 6B And Figure 6C Is an example of a 2D real-time capture image exemplifying an example of the subject matter according to the present disclosure; and

[0092] Figure 7 Is a flowchart of a generalized example method of imaging enhanced preparation process component placement exemplifying an example of the subject matter according to the present disclosure. Detailed Description

[0093] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, those skilled in the art will understand that the subject matter of the present disclosure may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the subject matter of the present disclosure.

[0094] Unless otherwise specifically stated, as will be apparent from the following discussion, it should be understood that throughout the specification discussion, the use of terms such as "processing", "computing", "comparing", "determining", "calculating", "receiving", "providing", "obtaining", "detecting", etc. refer to actions and / or processes in which a computer manipulates data and / or transforms data into other data, where the data is represented as physical (such as electronic) quantities and / or the data represents physical objects. The term "computer" should be broadly interpreted to cover any kind of hardware-based electronic device having data processing capabilities, as a non-limiting example, the electronic device includes the processors, mitigation units, and inspection units disclosed in the present application.

[0095] The terms "non-transitory memory" and "non-transitory storage medium" used herein should be broadly interpreted to cover any volatile or non-volatile computer memory suitable for the subject matter of the present disclosure.

[0096] The operations taught herein may be performed by a computer specifically constructed for the desired purpose or by a general-purpose computer configured by a computer program stored in a non-transitory computer-readable storage medium specifically for the desired purpose.

[0097] The embodiments of the subject matter of the present disclosure are not described with reference to any specific programming language. It should be understood that various programming languages may be used to implement the teachings of the subject matter of the present disclosure as described herein.

[0098] It should be understood herein that although the following examples have been described with reference to a drug formulation component as a syringe and a syringe adapter / connector, the following description relates to any type of drug formulation component generally described hereinabove.

[0099] It should be understood herein that although the following examples have been described with reference to a transfer unit TU as a syringe manipulator and a gripper as its gripping arm, the following description relates to any type of transfer unit and gripper generally described hereinabove.

[0100] Now note Figure 1A , which illustrates a pharmaceutical preparation device (PPD) according to some embodiments of the subject matter of the present disclosure.

[0101] In the illustrated example, the PPD includes a vial holder 140 that is adapted to hold a fluid vial 142 containing fluid to be aspirated into a syringe 130. The PPD also includes a syringe carousel 100 that includes the syringe 130, which is used by the PPD, e.g., by a syringe delivery unit 120 that delivers and manipulates the syringe 130.

[0102] Figure 1B An example delivery unit 120, as a syringe manipulator, of some embodiments of the subject matter of the present disclosure is illustrated, which has an engagement arm 160, a clamping arm 170, and a plunger arm 180. The clamping arm 170 forms a gripper that is configured to grip the syringe 130 at the syringe connector in the illustrated example. It should be understood that the gripper can grip the syringe at any other part of the syringe (e.g., the barrel) for the purpose of holding and moving the syringe.

[0103] Figure 1C An example syringe assembly 130 with associated fluid vial components of some embodiments of the subject matter of the present disclosure is schematically illustrated. The syringe assembly includes a plunger flange 115C, a plunger 125C, a syringe piston 135C, a syringe barrel 145C, and a syringe connector 165C.

[0104] In some examples, the PPD positions the syringe assembly that includes the syringe connector 165C to achieve its purpose of establishing a fluidic interconnection with a fluid container adapter 175C as part of a drug preparation process. The fluid container adapter 175C enables fluid transfer between the syringe 130 and a fluid container 185C.

[0105] Now note Figure 2 , which illustrates a generalized top view of an example path of a delivery unit in a drug preparation device (PPD) that utilizes imaging enhancement for component placement in a preparation process, of some embodiments of the subject matter of the present disclosure.

[0106] Figure 2 The example PPD delivery unit path depicted in includes a number of preparation process interconnection locations (PPILs) 220A, 220B, 220C. The term "interconnection location" refers at least to a physical location in the PPD where components for preparing a drug dose can be placed for performing pharmaceutical preparation operations. In Figure 2In an example, a delivery unit (e.g., a syringe delivery unit, a transporter, a robotic arm, etc. as described above) can move a component (e.g., a syringe) along line 205 between preparation process interconnection location 220A, preparation process interconnection location 220B, and preparation process interconnection location 220C. At each of these locations, the PPD can perform operations including drug transfer.

[0107] A controller of the delivery unit (e.g., as described below with reference to Figure 4 the like) can include processing circuitry that includes a processor and a memory, and the controller can be programmed to move the delivery unit such that the component is correctly positioned (at a desired location) at one of the preparation process interconnection locations 220A, 220B, 220C. In some examples, the delivery unit controller maintains a fixed offset indicating movement along delivery unit movement line 205 associated with the PPIL, and then moves the delivery unit each required distance using delivery unit servo control or a similar mechanism.

[0108] Note that in some examples, the delivery unit needs to deliver / move a gripper (as shown below with reference to Figure 4 the like) to a specific preparation process interconnection location such that the gripper can grasp the component. It should also be noted that in some examples, the delivery unit needs to deliver the component grasped by the gripper to a specific preparation process interconnection location such that an operation can be performed using the delivered component.

[0109] As a non-limiting example: The delivery unit controller can control the delivery unit (e.g., a syringe delivery unit) to move the gripper to a first preparation process interconnection location adjacent to a syringe component in a syringe component queue. Then, the delivery unit controller can control the delivery unit to, for example, extend a gripper arm associated with the gripper and control the gripper to grasp the syringe component. After grasping the syringe component, the delivery unit controller can control the delivery unit to move the gripper to a second preparation process interconnection location that is directly below the location where a vial has been placed in a vial holder. The delivery unit controller can control the delivery unit to move the gripper upward to establish an interconnection between the syringe component and a vial adapter attached to the vial. It should be understood herein that although the container has been described as a vial, it can be any of the containers described above.

[0110] There can be various reasons for errors when placing the gripper and / or the components grasped by the gripper. For example, the mechanical control (e.g., servo) of the conveying gripper may be inaccurate, or in some cases, its movement accuracy may degrade over time. Additionally, the drug formulation components (e.g., syringe assemblies) may deviate from the precise expected dimensions (or the components themselves may be imperfectly assembled). Furthermore, the formulation components grasped by the gripper may be positioned in an imperfect manner. For example: the syringe grasped by the gripper of the syringe delivery unit may be grasped higher or lower than expected, or at an angle that deviates from being perfectly in the desired orientation.

[0111] In some embodiments of the subject matter of the present disclosure, cameras 210A, 210B, 210C can be placed within the PPD to monitor the corresponding preparation process interconnect sites 220A, 220B, 220C. The respective positions of the cameras monitoring the PPIL are referred to as the PPIL camera sites.

[0112] Note that cameras 210A, 210B, 210C can actually be other types of imaging devices (e.g., radar or capacitive sensors). In some examples, one or more of cameras 210A, 210B, 210C capture two-dimensional (2D) digital images. In some examples, one or more of cameras 210A, 210B, 210C capture three-dimensional (3D) digital images.

[0113] The 2D digital image can include, for example, a pixel grid in a certain x-dimension and y-dimension, where each pixel is composed of, for example, a grayscale intensity or a triple of red, green, and blue intensities. The 3D digital image can additionally include, for example, a distance value associated with each pixel.

[0114] In some embodiments of the subject matter of the present disclosure, the controller can control one of the cameras to capture an image of the corresponding preparation process interconnect site. As described below, the controller can utilize the image and apply methods to identify potential misalignment or displacement of the gripper or the formulation components grasped by the gripper, and control the conveying arm to perform a corrective action.

[0115] Now note Figures 3A to 3C , which illustrates an example sequence of positioning a syringe assembly according to some embodiments of the subject matter of the present disclosure. Note that Figures 3A to 3C not drawn to scale.

[0116] In Figure 3AIn this case, the syringe connector of the syringe assembly is located at the initial syringe connector location 350. The initial syringe connector location 350 can be a location in three-dimensional space and is associated with x, y, and z coordinates. The syringe connector of the syringe assembly can be grasped, for example, by a syringe gripper, which in turn can be moved by a delivery unit that is controlled by a delivery unit controller (e.g., as shown below with reference to Figure 4 ).

[0117] The syringe connector pre-calibration interconnection location 370 (also referred to herein as the calibration position or desired position) is a location within the PPD at which the syringe connector needs to be placed to perform the drug preparation step. The syringe connector pre-calibration interconnection location 370 is located at the same position as or within the corresponding pharmaceutical preparation interconnection location (PPIL).

[0118] At this stage, the delivery unit controller can initiate the movement of the gripper in the x direction relative to the camera 310 (shown by the vertical line 355) and / or in the z direction (i.e., the up and down movement relative to the camera 310, which is not visible in the Figure 3A top view shown) to place the syringe connector at the syringe connector pre-calibration interconnection location 370. As a non-limiting example, the delivery unit controller can move the gripper 4 centimeters (cm) in the x direction.

[0119] The camera 310 (and the lens 320) can be pointed at the syringe connector pre-calibration interconnection location 370 and can capture images of the PPIL and the syringe connector pre-calibration interconnection location 370.

[0120] Note that in some embodiments, the gripper may need to move in the y direction relative to the camera 310 (possibly in addition to the movement in the x and / or z directions). The movement in the y direction relative to the camera 310 can be described as the forward and backward movement relative to the camera. In some such embodiments, a second camera is placed (e.g., placed at a 90-degree position relative to the first camera to monitor this movement). In some other embodiments, the camera 310 can be a 3D camera with 3D imaging capabilities as described above.

[0121] Figure 3B An example is shown where the delivery unit controller has moved the syringe connector to position the syringe connector at the location required for the next drug preparation step (i.e., 4 cm in the x direction). In this example, due to, for example, inaccurate placement by the servo mechanism of the delivery unit, or due to the gripper gripping the syringe with a certain displacement, the syringe assembly is placed at the syringe connector current location 360 (also referred to herein as the current position (i.e., having a certain x-direction displacement from the target / desired position)).

[0122] The camera 310 can capture an image of the syringe connector pre - calibration location 370 or the corresponding PPIL, and the image can include the current location 360 of the syringe connector. The camera 310 can capture the image in response to, for example, an instruction from a controller that is controlling the drug preparation process, or in response to an instruction from a local controller that captures images periodically.

[0123] A process controller (for example) can receive the captured image and perform the image processing and analysis techniques as detailed above in the Summary of the Invention section with reference to determining the distance and direction in the x and / or z directions for improving (e.g., optimizing) the placement of the syringe connector at the desired location. In some embodiments, the controller can also (or specifically) determine the distance and direction in the y direction (relative to the camera). Then, the process controller (for example) can control, for example, the delivery unit controller to move the gripper in the x, y, and / or z directions based on the determined distance and direction.

[0124] Figure 3C An example result of the second movement of the delivery unit controller is illustrated: the current location 360 of the syringe connector is aligned with the syringe connector pre - calibration interconnection location 370 in the x - direction (as visible) and in the z - direction (not visible). In this way, even in the presence of, for example, imprecise movement of the delivery unit, measurement deviations of syringe assembly components, changes in the position and orientation of the syringe in the gripper, etc., steps in the drug preparation process (e.g., establishing an interconnection between the syringe connector and the vial adapter) can be successfully performed.

[0125] Figure 3D A side view of a PPD according to some embodiments of the subject matter of the present disclosure is illustrated, where the syringe assembly is incorrectly placed.

[0126] In Figure 3D , the syringe connector 390D is to the left of the opening of the fluid vial 395D. This is similar to the arrangement shown above Figure 3B before the image - based placement of the syringe.

[0127] Figure 3E A side view of a PPD according to some embodiments of the subject matter of the present disclosure is illustrated, where the syringe assembly is correctly placed.

[0128] In Figure 3E , the syringe connector 390D is below the opening of the fluid vial 395D. This is similar to the arrangement shown above Figure 3C after the image - based placement of the syringe.

[0129] Now note Figure 4, which is a block diagram of an exemplary PPD that utilizes imaging enhancement to prepare process component placement according to some embodiments of the subject matter of the present disclosure.

[0130] The system controller 410 can be a controller that controls PPD components to perform all or part of a drug preparation process. The system controller 410 can include processing circuitry 420, which in turn can include a processor 430A and a memory 440A. The system controller 410 can be operably connected to a camera 495 and a delivery unit controller 405.

[0131] The processor 430A can be a suitable hardware-based electronic device having data processing capabilities, such as, for example, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), one or more cores of a multi-core processor, etc. The processor 430A can also be composed of, for example, multiple processors, multiple ASICs, virtual processors, combinations thereof, etc.

[0132] The memory 440A can be, for example, a suitable type of volatile and / or non-volatile storage device, and can include, for example, a single physical memory component or multiple physical memory components. The memory 440A can also include virtual memory. The memory 440A can be configured to store various data used in the computation, for example.

[0133] The processing circuitry 420 can be configured to execute a number of functional modules according to computer-readable instructions implemented on a non-transitory computer-readable storage medium. Such functional modules are hereinafter referred to as being included in the processing circuitry. These modules can include, for example, a camera control unit 450, a motion control unit 460, and an image processing unit 470.

[0134] The camera control unit 450 can control the image capture performed by the camera 495 and can receive the captured image data from the camera 495.

[0135] The motion control unit 460 can control the movement of the PPD components and the data received regarding such movement, for example, by communicating with the delivery unit controller 405.

[0136] The image processing unit 470 can perform image processing on the captured images and can also perform analysis to determine the direction and associated distance in which, for example, the delivery unit needs to be moved.

[0137] The delivery unit controller 405 can include a processor 430b and a memory 440b. The delivery unit controller 405 can be operably connected to the delivery unit 415 and exchange commands and / or status information.

[0138] The delivery unit 415 can be an assembly including a gripper arm 480, a gripper 490, and / or other movable components, and can actuate the movement of these components (e.g., in response to a command from the delivery unit controller 405).

[0139] In some embodiments, the delivery unit 415 is the syringe delivery unit as described above with reference to Figures 1B to 1C the syringe delivery unit. In some other embodiments, the delivery unit 415 is a different mechanism for grasping, moving, and releasing components (e.g., a Selective Compliance Assembly Robot Arm (SCARA), etc.).

[0140] Note that in some embodiments, the delivery unit controller 405 can be incorporated into the system controller 410 such that, for example, the system controller 410 can directly control and / or receive status information from physical components such as the gripper arm 480 and the gripper 490.

[0141] Note that the teachings of the subject matter of this disclosure are not bound by the entities described with reference to Figure 4 Equivalent and / or modified functionality can be combined or divided in another way and can be implemented in any suitable combination of software with firmware and / or hardware and executed on a suitable device. The system controller and the delivery unit controller can be independent network entities or can be integrated, in whole or in part, with other entities. Those skilled in the art will be clear on how to adopt the control system in other embodiments.

[0142] Now note Figure 5 which illustrates a flowchart of an example method of imaging-enhanced preparation process component placement in a pharmaceutical preparation device (PPD) according to some embodiments of the subject matter of this disclosure.

[0143] The processing circuit 420 (e.g., the motion control unit 460) can perform 510 initial control of the delivery unit 415 (e.g., via the delivery control unit 405) to initiate the movement of the syringe connector 165C from, for example, an initial syringe connector location 350 to a syringe connector pre-calibration interconnection location 370. In Figure 5 the example method illustrated, the syringe connector pre-calibration interconnection location 370 can be located directly below the fluid container adapter 175C.

[0144] Next, the processing circuit 420 (e.g., the camera control unit 450) can then control 520 the camera 495 to capture an image of the syringe connector 165C in its current position in real time (during the use of the PPD). In some examples, the captured image is a 2D image. In some other examples, the captured image is a 3D image, which can include distance information associated with each pixel, as described above.

[0145] The processing circuit 420 (e.g., the image processing unit 470) can then utilize the image captured at 530 in combination with, for example, pre-calibration data to determine the distance and direction for moving the delivery unit 415 such that the syringe connector 165C is directly below the fluid container adapter 175C, i.e., ready to establish an interconnection therewith.

[0146] In some cases, the pre-calibration data can include two-dimensional (2D) or three-dimensional (3D) pre-calibration images captured by the camera from the PPIL camera location, where the 2D or 3D pre-calibration images depict the gripper and / or syringe in a calibration position (also referred to herein as the desired position). Once the image of the PPIL captured in step 520 is obtained, the processing circuit determines whether the gripper and / or syringe is in the desired position based on the pre-calibration image and the captured (real-time) image (e.g., by analyzing the pixel offset between them), and if not, determines the distance and direction for moving the delivery unit 415 such that the syringe connector 165C is directly below the fluid container adapter 175C, i.e., in the desired position.

[0147] In some cases, the pre-calibration data can include a calibration transformation matrix, which is a derivative of a calibration process that utilizes at least three images from the PPIL camera location depicting the gripper and / or syringe, and the calibration transformation matrix is adapted to convert the x, y, and / or z coordinates of the image into corresponding coordinates in the delivery unit reference frame. It should be understood herein that all the descriptions provided above in the general description regarding the calibration transformation matrix apply to step 530. The calibration transformation matrix can be derived, for example, as described in "The 9Points Calibration Using SCARA Robot" by Joochim et al. (https: / / ieeexplore.ieee.org / abstract / document / 8999901).

[0148] In some examples, the pre-calibration data can also include the desired position of the gripper and / or syringe in the PPIL in the delivery unit reference frame (TUFoR), and thus, the processing circuit can have the desired coordinates (in the TUFoR), i.e., the coordinates of the position where the gripper and / or syringe needs / desires to be positioned.

[0149] When the captured image is a 2D image, the processing circuit 420 (e.g., the image processing unit 470) can determine the direction of movement in the x and / or z directions, as described above with respect to Figures 3A to 3D described.

[0150] When the captured image is a 3D image, the processing circuit 420 (e.g., the image processing unit 470) can determine the direction including movement in the x, y, and / or z directions.

[0151] In some other embodiments, the processing circuit 420 (e.g., the image processing unit 470) determines the distance and direction of movement of the delivery unit 415 from the captured 2D or 3D image and the pre-calibration image in different suitable ways.

[0152] Note that when initializing or first using the PPD, the pre-calibration image can be acquired during the "calibration phase". For example, the syringe can be positioned manually or mechanically at the syringe connector pre-calibration interconnection location 370, and then the pre-calibration image can be captured.

[0153] Similarly, in some examples, the PPD can facilitate capturing two 2D images or three 3D images such as PPILs, where the syringe connector 165C is placed with a known deviation from the correct position. The system controller 410 can then determine the pixel-to-distance value based on the image and the known deviation from the correct / desired position.

[0154] The processing circuit 420 (e.g., the motion control unit 460) can then perform 540 control of the syringe delivery unit 415 to move it according to the determined distance and direction. For example, such that the syringe connector 165C is directly below the fluid container adapter 175C, i.e., in the desired position.

[0155] The processing circuit 420 (e.g., the motion control unit 460) can control 550 the delivery unit 415 to raise the syringe connector 165C in order to establish the interconnection between the syringe connector 165C and the fluid container adapter 175C.

[0156] Note that the teachings of the subject matter of the present disclosure are not limited by Figure 5 and Figure 7 the flowcharts shown. The illustrated operations may not occur in the illustrated order. It should also be noted that although the flowcharts are described with reference to the Figures 1A to 1C and Figure 4 elements of the system, this is by no means restrictive, and the operations can be performed by elements other than those described herein.

[0157] Figure 6A is an example 2D "pre-calibration image" according to some embodiments of the subject matter of the present disclosure.

[0158] Figure 6A Depicts a syringe assembly attached to the syringe connector 165C. The syringe connector 165C is positioned at the optimal location and orientation, i.e., the desired position.

[0159] Figures 6B to 6C is an example of capturing a 2D image according to some embodiments of the subject matter of the present disclosure. Figure 6B is an image of the syringe connector 165C having a deviation in the x-direction. Similarly, Figure 6C is an example image of the syringe connector 165C having a deviation in the z-direction.

[0160] Note that in some examples, the pre-calibrated image will depict the gripper 490 in the position it needs to be in before gripper engagement. In such examples, the captured digital image may include the gripper 490, and the system controller's calculations of orientation and distance can utilize pixel offsets of the gripper, etc.

[0161] Now note that Figure 7 , which illustrates a flowchart of a generalized example method of imaging enhanced preparation process component placement in a pharmaceutical preparation device (PPD) according to some embodiments of the subject matter of the present disclosure.

[0162] Figure 7 The described method is applicable both to correctly positioning the gripper (e.g., to grasp an object) and to correctly positioning the grasped object, such as a pharmaceutical preparation component.

[0163] The processing circuit 420 (e.g., the motion control unit 460) may perform 710 initial control of the delivery unit 415 (e.g., via the delivery control unit 405) to move the delivery unit 415 from an initial location to position the gripper 490 or the component grasped by the gripper 490 at a preparation process interconnection location (PPIL).

[0164] Next, the processing circuit 420 (e.g., the camera control unit 450) may control 720 the camera to capture in real time (during the use of the PPD) an image of the gripper 490 (or the component grasped by the gripper 490) in its current position. In some examples, the captured image is a 2D image. In some other examples, the captured image is a 3D image, which may include distance information associated with each pixel, as described above.

[0165] The processing circuit 420 (e.g., the image processing unit 470) may then utilize 730 the captured image in combination with, for example, pre-calibrated data to determine the distance and direction for moving the delivery unit 415 such that the gripper 490 (or the component grasped by the gripper 490) is positioned at the PPIL and, for example, ready for pharmaceutical preparation. It should be understood herein that all the descriptions provided above regarding the pre-calibrated data (in the general description or with reference to step 530) also apply to step 730.

[0166] The processing circuit 420 (e.g., the motion control unit 460) may perform a second control 740 of the conveying unit 415 to move the gripper 490 (or the component grasped by the gripper 490) by a determined distance and direction, so as to achieve the correct placement of the gripper 490 (or the component grasped by the gripper 490) at the PPIL, for example, at the desired position.

[0167] The processing circuit 420 (e.g., the motion control unit 460) may perform an action 750 associated with the PPIL, for example, control the conveying unit 415 to:

[0168] - Grasp a component in the gripper 490,

[0169] - Release the component from the grasp of the gripper 490,

[0170] - Establish a fluid connection between another component and the component held in the gripper 490.

[0171] It should be understood that the present invention is not limited in its application to the details set forth in the specification contained herein or shown in the drawings. The present invention is capable of other embodiments and of being practiced and carried out in various ways. Accordingly, it should be understood that the language and terminology used herein are for the purpose of description and should not be regarded as limiting. Thus, those skilled in the art will appreciate that the concepts upon which this disclosure is based can readily be utilized as a basis for designing other structures, methods, and systems for carrying out several purposes of the presently disclosed subject matter.

[0172] It should also be understood that the system according to the present invention can be implemented at least in part on a suitably programmed computer. Similarly, the present invention contemplates a computer program capable of being read by a computer for performing the method of the present invention. The present invention also contemplates a non-transitory computer-readable memory that tangibly includes an instruction program executable by a computer for performing the method of the present invention.

[0173] Those skilled in the art will readily understand that various modifications and changes can be applied to the embodiments of the present invention as described above without departing from the scope of the present invention defined by the appended claims.

[0174] It should be understood herein that, for the purpose of making this specification concise and clear, the examples described in this specification (referring to the drawings and others) are described only with reference to several components of the drug preparation equipment among all the drug preparation equipment covered by the scope of the present disclosure. Various examples similar to those described herein with different components of the drug preparation equipment and different robotic stations should be considered to be within the scope of this specification.

Claims

1. A pharmaceutical preparation device (PPD) with component placement in the imaging enhancement preparation process, the pharmaceutical preparation device (PPD) comprising: A camera configured to capture an image of a pharmaceutical preparation interconnection location (PPIL) from a PPIL camera location, the PPIL being associated with pre-calibration data; A transport unit (TU) including a gripper for gripping a pharmaceutical formulation component, the TU being configured to move at least the gripper in one or more of the x, y, and / or z directions in response to a control signal; And A processing circuit operably connected to the TU and the camera, the processing circuit being configured to: At a current position in the PPIL, receive at least one real-time digital image captured by the camera from the PPIL camera location, the received digital image depicting the gripper of the TU and / or the pharmaceutical formulation component gripped by the gripper; Determine at least based on the received real-time digital image and the pre-calibration data the direction and distance of TU movement; And Control the TU via the control signal to move the gripper the determined distance in the determined direction.

2. The PPD according to claim 1, wherein The received digital image is two-dimensional (2D), and wherein the determined direction includes the x direction and / or the z direction.

3. The PPD according to claim 1, wherein The received digital image is three-dimensional (3D), and wherein the determined direction includes the x direction, the y direction, and / or the z direction.

4. The PPD according to any one of claims 1 to 3, wherein, The pre-calibration data includes a two-dimensional (2D) or three-dimensional (3D) pre-calibration image captured by the camera from the PPIL camera location, the two-dimensional (2D) or three-dimensional (3D) pre-calibration image depicting the gripper of the TU in a calibration position and / or the pharmaceutical formulation component gripped by the gripper of the TU.

5. The PPD according to claim 4, wherein, The processing circuit is further configured to: before determining the direction and distance of TU movement, determine that there is a pixel offset between the current position and the calibration position.

6. The PPD according to claim 5, wherein The processing circuit is configured to: determine at least based on the pixel offset the direction and distance of TU movement.

7. The PPD according to any one of claims 1 to 3, wherein The pre-calibration data includes a calibration transformation matrix, which is the derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the pharmaceutical formulation component gripped by the gripper, the calibration transformation matrix being adapted to convert the x, y, and / or z coordinates of an image into corresponding coordinates in the transport unit reference frame.

8. The PPD according to claim 7, wherein The processing circuit is configured to: determine at least by applying the calibration transformation matrix to the received real-time digital image the direction and distance of TU movement.

9. The PPD according to claim 7 or 8, wherein The received digital image is two-dimensional (2D), and the pre-calibration data includes a calibration transformation matrix, which is the derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the pharmaceutical formulation component gripped by the gripper.

10. The PPD according to claim 7 or 8, wherein, The received digital image is three-dimensional (3D), and the pre-calibration data includes a calibration transformation matrix that is a derivative of a calibration process that utilizes at least four images from the PPIL camera location, the at least four images depicting the gripper of the TU and / or the drug formulation component grasped by the gripper.

11. The PPD according to any one of claims 7 to 10, wherein, The processing circuit is configured to: perform the calibration process.

12. The PPD according to any one of claims 1 to 11, wherein, The processing circuit is further configured to control the gripper of the TU to perform at least one of the following: Grasp a drug formulation component; Release the drug formulation component being grasped by the gripper; and Establish fluid communication between a second drug formulation component and the drug formulation component being grasped by the gripper.

13. A method for imaging-enhanced preparation process component placement in a pharmaceutical preparation device (PPD) based on a processing circuit, the method comprising: Utilizing a transport unit (TU) configured to move at least a gripper of the transport unit in one or more of the x, y, and / or z directions in response to a control signal; Utilizing pre-calibration data; At a current position, receiving at least one real-time digital image of the PPIL captured by a camera located at a pharmaceutical preparation interconnection location (PPIL) camera location, the received digital image depicting the gripper of the TU and / or a drug formulation component grasped by the gripper; Determining a direction and distance of TU movement based at least on the received real-time digital image and the pre-calibration data; And Controlling the TU to move the gripper the determined distance in the determined direction.

14. The method according to claim 13, wherein, The pre-calibration data includes two-dimensional (2D) or three-dimensional (3D) pre-calibration images captured by the camera from the PPIL camera location, the two-dimensional (2D) or three-dimensional (3D) pre-calibration images depicting the gripper of the TU and / or a drug formulation component grasped by the gripper of the TU in a calibration position.

15. The method according to claim 14, the method further comprising: Before determining the direction and distance of TU movement, determining that there is a pixel offset between the current position and the calibration position.

16. The method according to claim 15, the method comprising: Determining the direction and distance of TU movement based at least on the pixel offset.

17. The method according to claim 13, wherein The pre-calibration data includes a calibration transformation matrix that is a derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the drug formulation component grasped by the gripper, the calibration transformation matrix being adapted to convert x, y, and / or z coordinates of an image to corresponding coordinates in a transport unit reference frame.

18. The method according to claim 17, the method comprising: Determining the direction and distance of TU movement by at least applying the calibration transformation matrix to the received real-time digital image.

19. The method according to claim 18, wherein the method further comprises: Performing the calibration process.

20. A computer program product comprising a computer-readable non-transitory storage medium containing program instructions that, when read by a processor, cause the processing circuit to execute a method for placing an imaging enhancement preparation process component in a drug preparation device (PPD), the method comprising: Utilizing a transport unit (TU) configured to move at least a gripper of the transport unit in one or more of the x, y, and / or z directions in response to a control signal; Utilizing pre-calibration data; At a current position, receiving at least one real-time digital image of a PPIL captured by a camera located at a PPIL camera location, the received digital image depicting the gripper of the TU and / or a drug preparation component grasped by the gripper; Determining the direction and distance of TU movement based at least on the received real-time digital image and the pre-calibration data; And Controlling the TU to move the gripper the determined distance in the determined direction.

21. The computer program product according to claim 20, wherein, The pre-calibration data includes two-dimensional (2D) or three-dimensional (3D) pre-calibration images captured by the camera from the PPIL camera location, the two-dimensional (2D) or three-dimensional (3D) pre-calibration images depicting the gripper of the TU and / or a drug preparation component grasped by the gripper of the TU in a calibration position.

22. The computer program product according to claim 21, wherein, The method further includes: determining that there is a pixel offset between the current position and the calibration position before determining the direction and distance of TU movement.

23. The computer program product according to claim 22, wherein, The method further includes: determining the direction and distance of TU movement based at least on the pixel offset.

24. The computer program product according to claim 20, wherein, The pre-calibration data includes a calibration transformation matrix that is a derivative of a calibration process that utilizes at least three images from the PPIL camera location, the at least three images depicting the gripper of the TU and / or the drug preparation component grasped by the gripper, the calibration transformation matrix being adapted to convert x, y, and / or z coordinates of an image to corresponding coordinates in a transport unit reference frame.

25. The computer program product according to claim 24, wherein, The method includes: determining the direction and distance of TU movement by applying the calibration transformation matrix to the received real-time digital image at least.

26. The computer program product according to claim 24, wherein, The method further includes: performing the calibration process.

Images