Slide imaging equipment
The method enables autonomous teaching of target locations for a slide imaging facility feeding device by using an operating system to detect collisions and determine positions, addressing the complexity of manual teaching and allowing for efficient system adjustments.
Patent Information
- Application Number
- JP2024563367
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-05-04
- Filing Date
- 2023-05-03
- Publication Date
- 2025-05-13
AI Technical Summary
The existing methods for programming robotic feeding devices in slide imaging facilities require manual teaching by humans, which is complex, time-consuming, and difficult to adapt to changes in the system.
A method for autonomous teaching of target locations for a slide imaging facility feeding device, where the feeding device is driven into pre-located positions until collisions with the imaging device are detected, and the target locations are determined by evaluating these collisions using an operating system.
This method allows for quick and simplified adjustment of target positions, reducing the complexity and time required for programming the feeding device, and enabling autonomous operation without human intervention.
Smart Images

Figure 2025514965000001_ABST
Abstract
Description
[Technical Field]
[0001] The present disclosure relates to a method for autonomously teaching at least one target position of a feeder of a slide imaging equipment, and to a slide imaging equipment, wherein the slide imaging equipment can preferably be used in digital pathology, although further applications are also feasible. [Background technology]
[0002] Slide imaging facilities include imaging devices configured to generate images of slide-mounted samples. Images generated by modern imaging devices are typically digital images, and therefore such images are sometimes referred to as "digital slides." Typically, the slide-mounted samples are biological specimens, such as tissue samples. Typically, the slides are glass slides. Slide imaging facilities are typically used in digital pathology, which can be understood as an image-based information environment that enables management of information generated from digital slides.
[0003] When the imaging device can generate an image that may cover most or the entire surface of the slide, for example, by using a scanning process, the corresponding slide imaging equipment may be referred to as a "whole slide imaging" equipment. Slide imaging equipment may generate an image of the sample mounted on the slide using a 2D (two-dimensional) camera or a line scan detector. Examples of slide imaging equipment are described in, for example, EP 00534247, EP 0245089, U.S. Pat. No. 6,118,582, U.S. Pat. No. 6,522,774, U.S. Pat. No. 6,640,014, U.S. Pat. No. 6,711,283, U.S. Pat. No. 7,682,573, WO 2013 / 017855, U.S. Pat. No. 8,712,116, and U.S. Pat. No. 9,116,035.
[0004] Generally, imaging devices have the capacity to process 1 to 1,000 slides simultaneously. A distinction can be made between low-throughput imaging devices, which typically involve the simultaneous processing of fewer than 10 slides, and high-throughput imaging devices, which typically involve the simultaneous processing of more than 100 slides; thus, imaging devices configured to process 10 to 100 slides simultaneously can be referred to as medium-throughput imaging devices. To load one or more slides into the imaging device, individual slides are typically inserted manually or automatically into a slide storage location, typically selected from a slide tray or slide rack, and then introduced into the imaging device to generate the desired image of the sample mounted on the slide.
[0005] WO 2021 / 191411 describes a slide imaging installation and a method for imaging a plurality of slides, the slide imaging installation comprising: at least one first imaging device and at least one second imaging device, each configured to generate an image of a sample mounted on a slide; a storage device capable of holding a plurality of slides and configured to store the slides; and a supply device configured to selectively supply slides from the storage device to the at least one first imaging device or the at least one second imaging device, the at least one first imaging device and the at least one second imaging device comprising at least one visual indicator configured to indicate an operating state of the at least one first imaging device and the at least one second imaging device, the slide imaging installation further comprising at least one vision sensor configured to detect an operating state of the at least one first imaging device and the at least one second imaging device using the at least one visual indicator.
[0006] WO 2021 / 191410 describes a slide imaging installation and a method for imaging a plurality of slides, the slide imaging installation comprising: at least one imaging device configured to generate an image of a sample mounted on the slide and having at least one operating button; a storage device capable of loading a plurality of slides and configured to store the slides; and a supply device configured to supply the slides from the storage device to the imaging device, the supply device being configured to press the operating button. The slide imaging installation and the method for imaging a plurality of slides enable improved processing of slides processed in the imaging device, the slides being provided to be introduced into a slide receiving section of the imaging device.
[0007] U.S. Patent Application Publication No. 2002 / 068992 describes a robot calibration system for calibrating a workpiece handling robot relative to a station. U.S. Patent Application Publication No. 2009 / 302795 describes a robot teaching tool for automatically teaching pick and place positions for a robot. U.S. Patent Application Publication No. 6,323,616 describes a wafer handling facility having an input and output robotic system directed by a programmed controller. U.S. Patent Application Publication No. 2019 / 301980 describes a system and method for transporting and processing sectioned biological specimens, particularly for using multiple imaging and processing aspects to evaluate characteristics of the sectioned tissue specimens.
[0008] The automated feeding of slides in a slide imaging facility, for example from an archive to an imaging device, can be performed by a feeding device, which may comprise a robot or a robotic arm.
[0009] To program a robot, a position is usually taught by a human. If the position of the system changes, the human must teach the robot the adapted position again. However, this procedure is complicated and difficult. [Prior art documents] [Patent documents]
[0010] [Patent Document 1] European Patent No. 00534247 [Patent Document 2] European Patent Application Publication No. 0245089 [Patent Document 3] U.S. Patent No. 6,118,582 [Patent Document 4] U.S. Patent No. 6,522,774 [Patent Document 5] U.S. Patent No. 6,640,014 [Patent Document 6] U.S. Patent No. 6,711,283 [Patent Document 7] U.S. Patent No. 7,682,573 [Patent Document 8] International Publication No. 2013 / 017855 [Patent Document 9] U.S. Patent No. 8,712,116 [Patent Document 10] U.S. Patent No. 9,116,035 [Patent Document 11] International Publication No. 2021 / 191411 [Patent Document 12] International Publication No. 2021 / 191410 [Patent Document 13] US Patent Application Publication No. 2002 / 068992 [Patent Document 14] US Patent Application Publication No. 2009 / 302795 [Patent Document 15] U.S. Patent No. 6,323,616 [Patent Document 16] US Patent Application Publication No. 2019 / 301980 Summary of the Invention [Problem to be solved by the invention]
[0011] It is therefore desirable to provide a slide imaging facility and method for autonomous teaching of at least one target position of a feeder device of the slide imaging facility that enables autonomous teaching of a target position of the feeder device. [Means for solving the problem]
[0012] This problem is solved by a method for autonomously teaching at least one target position of a supply device of a slide imaging installation and a slide imaging installation having the features of the independent claims. Advantageous embodiments, which may be realized alone or in any combination, are listed in the dependent claims as well as in the entire specification.
[0013] When used below, the terms "having," "comprising," or "including," or any grammatical variants thereof, are used in a non-exclusive manner. These terms can therefore refer both to a situation in which, besides the features introduced by these terms, no further features are present in the entity described in this context, and to a situation in which one or more further features are present. For example, the expressions "A has B," "A comprises B," and "A includes B" can refer both to a situation in which no other elements are present in A besides B (i.e., a situation in which A consists exclusively of B), and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D, or even further elements.
[0014] Furthermore, it should be noted that the terms "at least one" or "one or more" or similar expressions, indicating that a feature or element may be present more than once, are typically used only once when introducing each feature or element. In the following, in most cases, when referring to each feature or element, the expressions "at least one" or "one or more" will not be repeated, despite the fact that each feature or element may be present more than once.
[0015] Furthermore, when used hereinafter, the terms "preferably," "more preferably," "particularly," "even more particularly," "particularly," "more particularly," or similar terms are used in conjunction with optional features without limiting the possibilities for substitution. Features introduced by these terms are therefore optional features and are not intended to limit the scope of the claims in any way. The present invention may be implemented by using alternative features, as would be understood by one skilled in the art. Similarly, features introduced by "in an embodiment of the invention" or similar phrases are intended to be optional features without any limitations regarding alternative embodiments of the invention, without any limitations regarding the scope of the invention, and without any limitations regarding the possibility of combining a feature introduced in this manner with other optional or non-optional features of the invention.
[0016] In a first aspect of the present invention, a method for autonomously teaching at least one target position of a feeding device of a slide imaging equipment is disclosed. The slide imaging equipment includes at least one imaging device configured to generate an image of a sample mounted on a slide. The target position is a position on the imaging device. The slide imaging equipment includes at least one operating system configured to control operation of the feeding device.
[0017] As used herein, the term "sample" is a broad term and should be given its common and ordinary meaning to those skilled in the art, and should not be limited to a specific or special meaning. The term may specifically refer to, but is not limited to, a biological specimen such as a tissue or smear. However, other types of samples may also be feasible.
[0018] As used herein, the term "slide" is a broad term and should be given its common and ordinary meaning to those skilled in the art, and should not be limited to a specific or special meaning. This term may specifically, but not exclusively, refer to a substrate designated for a sample to be placed on the surface of the slide. The substrate may be mechanically stable. The substrate may include any material that provides sufficient mechanical stability. The substrate may be configured to support the sample without any change during processing into a slide. The substrate may exhibit a surface configured to be compatible with biological materials. As an example, the slide is a glass slide. Glass is known to provide sufficient mechanical stability on the one hand and to have high compatibility with biological materials on the other hand. However, additional types of materials for the slide may also be feasible.
[0019] The slide may have a shape that may enable imaging of the sample mounted on the slide. The slide may be a plate having a 2D extent and a thickness. The 2D extent of the plate may have a rectangular or circular shape. The thickness of the plate may be small compared to the size of the extent, for example, 20%, 10%, or 5% or less of the linear extent scale of the 2D extent of the plate. Such a design may enable and / or assist in generating a desired image of the sample.
[0020] As used herein, the terms "imaging" or "generating an image" are broad terms and should be given their common and ordinary meaning to those skilled in the art, and should not be limited to any specific or special meaning. These terms can specifically, but are not limited to, referring to providing a 2D two-dimensional representation of at least one characteristic of a sample, also referred to by the term "image," which can typically be processed and displayed on a screen to be captured by an observer's eye without any further aid other than, for example, the observer's glasses. For this purpose, an imaging device may be used.
[0021] As used herein, the term "slide imaging equipment" is a broad term and should be given its common and ordinary meaning to one skilled in the art, and should not be limited to a specific or special meaning. The term may specifically, but is not limited to, refer to any device configured to image a sample mounted on a slide. Furthermore, as used herein, the terms "equipment" and "slide imaging equipment" are broad terms and should be given their common and ordinary meaning to one skilled in the art, and should not be limited to a specific or special meaning. These terms may specifically, but are not limited to, refer to an apparatus having multiple components, such as those disclosed in more detail below.
[0022] As used herein, the term "imaging device" is a broad term and should be given its common and ordinary meaning to one of ordinary skill in the art, and should not be limited to a specific or special meaning. The term may specifically refer to, but is not limited to, a device configured to generate a 2D representation of at least one visual characteristic of a sample. For example, the imaging device may be one or more of a 2D camera or a line-scan detector. However, additional types of imaging devices may be possible.
[0023] The slide imaging facility may include at least one first imaging device and at least one second imaging device, each configured to generate an image of a sample mounted on a slide. The slide imaging facility may include one, two, three, four, five, six, or more individual imaging devices. The feeder may be configured to selectively feed slides to the first imaging device or the second imaging device. The individual imaging devices may be individually addressed by the feeder.
[0024] The slide imaging equipment may have a modular configuration. The slide imaging equipment may include a frame, such as a frame with wheels, and at least one plate connected to the frame, and one or more imaging devices may be disposed on the at least one plate. Herein, at least one plate may be extendable from the frame independently of the other plates, thus improving maintenance personnel's access to each imaging device. However, in embodiments in which at least one imaging device is considered too delicate to move, a plate may not be necessary. Two or more imaging devices may be disposed adjacent to each other, particularly vertically, one above the other, or horizontally, adjacent to each other. Generally, the adjacent arrangement of at least two imaging devices may facilitate the robot arm of the supplying device to reach the second imaging device. Furthermore, the slide imaging equipment may include a table, and at least the storage device and the supplying device may be attached to the table. The slide imaging equipment may include a housing at least partially surrounding the supplying device, and the housing may include a safety door and a safety switch configured to detect the state of the safety door. Further, the slide imaging equipment may include an emergency stop switch, and may further include an emergency stop button, and the emergency stop switch may be operable using the emergency stop button. For further details regarding further preferred configurations of the slide imaging equipment, reference may be made to WO 2021 / 191411 and WO 2021 / 191410, the disclosures of which are incorporated herein by reference.
[0025] The slide imaging equipment may include at least one monitor configured to display at least one image, e.g., a plurality of images, in a format viewable by a user of the slide imaging equipment. For example, the monitor may be mounted on a pivotable holder to facilitate viewing of the at least one image by the user from various positions.
[0026] As used herein, the term "delivery device" is a broad term and should be given its common and ordinary meaning to one of ordinary skill in the art and should not be limited to a specific or special meaning. The term may specifically refer to, but is not limited to, a device configured to transfer slides, for example, from a storage device to an imaging device.
[0027] The feeding device may comprise at least one robotic arm. As used herein, the term "robotic arm" is a broad term and should be given its common and ordinary meaning to those skilled in the art, and should not be limited to a specific or special meaning. The term may specifically, but is not limited to, refer to a programmable mechanical unit having the form of at least one of a hand or an arm and configured to move in a manner similar to a hand or an arm using electric and / or pneumatic drives, for example, but not limited to, grasping and transferring at least one object, particularly a slide or slide holder, to a predetermined destination.
[0028] The feeding device may include a protrusion. As used herein, the term "protrusion" is a broad term and should be given its common and ordinary meaning to those skilled in the art, and should not be limited to a specific or special meaning. The term may specifically, but is not limited to, refer to an element of the feeding device that protrudes beyond other elements of the feeding device. For example, the protrusion is spear-shaped or finger-shaped. However, additional types of shapes may be possible. The protrusion may be located on a robot arm, for example, at the end of the robot arm. The protrusion may be located on the robot arm by attaching the protrusion to the robot arm or by forming the protrusion on the robot arm.
[0029] The robot arm may include a gripping device. As used herein, the term "gripping device" is a broad term and should be given its common and ordinary meaning to those skilled in the art and should not be limited to a specific or special meaning. This term may specifically, but not exclusively, refer to a mechanical element typically designed to grasp an object, transfer it to a desired location, and release it at the desired location. A particular gripping device may be configured to grasp a slide or slide holder and transfer it securely to a slide receiving portion of an imaging device to avoid instability or loss of the slide during transport, preferably releasing it into a slide tray as described elsewhere herein. Here, a protrusion may be disposed on the gripping device, such that the gripping device can insert a slide into the slide receiving portion of the imaging device and simultaneously press an operating button, such as an eject button, of the imaging device to eject a slide tray configured to hold the slide during scanning of the slide in the imaging device. The gripping device may include a first gripper and a second gripper. The first and second gripping portions may be movable relative to one another, for example, in a linear manner. The first and second gripping portions may have opposing surfaces. The surfaces may be substantially planar, thus supporting a firm grip. However, other types of protrusion arrangements are also contemplated.
[0030] The supply device may be configured to press the operation button using a protrusion. The protrusion may collide with the operation button in various events, potentially leading to wear on at least one of the protrusion and the operation button. The supply device may include a protective cover configured to cover the protrusion. As used herein, the term "protective cover" is a broad term and should be given its general and ordinary meaning to those skilled in the art and should not be limited to a specific or special meaning. Specifically, but not exclusively, the term may refer to a preferably flexible object that can be positioned to separate the component to be covered from surrounding objects. Here, the shape of the protective cover may preferably be formed to cover the spear-shaped or finger-shaped protrusion in a very tight manner. Consequently, the protective cover may be configured to protect at least one of the protrusion and / or the operation button from wear.
[0031] The slide imaging equipment may include at least one storage device configured to load and store a plurality of slides. The storage device may be capable of loading a slide holder. The slide holder may be configured to hold a plurality of slides. The supply device may be configured to transport slides from the imaging device to the storage device. The supply device may be configured to supply slides from the storage device to the imaging device. As used herein, the term "storage device" is a broad term and should be given its common and ordinary meaning to those skilled in the art and should not be limited to a specific or special meaning. Specifically, but not exclusively, the term may refer to a slide storage location designated to accept an individual slide holder or to simultaneously accept two or more slide holders, each configured to hold two or more slides. For example, the storage device may include at least two compartments, each configured to store a portion of the slides. As used herein, the compartments may include at least two adjacently arranged rows, thereby storing slides adjacent to each other. The storage device may be selected from a slide tray or a slide rack, although further types of storage devices may be feasible. However, further types of arrangements of slides and / or slide holders, respectively, within the storage device may also be feasible. Thus, it may be possible to load a plurality of slides into the storage device, for example manually, although automatic loading of the storage device may also be envisaged.
[0032] The slide imaging facility may be configured to subsequently introduce the slide into an imaging device to generate a desired image of the specimen mounted on the slide. In the slide imaging facility, a plurality of slides may be loaded into a storage device, where the slides are stored until they are delivered from the storage device to an imaging device. Delivery of the slides from the storage device to the at least one imaging device may be performed using a delivery device in an automated manner.
[0033] As used herein, the terms "in an automated manner" and "automatically" are broad terms and should be given their common and ordinary meaning to those skilled in the art, and should not be limited to any specific or special meaning. These terms may specifically, but are not limited to, refer to types of processing that are performed based on an algorithm and without direct interaction by a user of the slide imaging equipment. For example, moving a slide may be performed without manual intervention by a user.
[0034] The supply device may be further configured to transport the slides from the at least one imaging device to a storage device after the scanning process is completed, e.g., to the associated position in the storage device where the slides were loaded. In other words, the supply device may be configured to return the slides from the imaging device to the same position in the storage device where the slides were loaded before being transferred from the storage device to the imaging device. Thus, a user of the slide imaging facility can receive the slides again in the same order in which they were provided to the storage device, thus facilitating subsequent identification and possible further processing of the slides.
[0035] As described above, the imaging device includes an operating button. As used herein, the term "operation button" is a broad term and should be given its general and ordinary meaning to those skilled in the art, and should not be limited to a specific or special meaning. This term may specifically, but not exclusively, refer to a component of the device, where such a request element is typically located on a surface of the device accessible to a person or object, and may refer to a component that can be pressed or released to transmit information to the device and subsequently cause the device to perform an action. Here, the terms "press," "pressed," or "pressing" may specifically, but not exclusively, refer to a person or object bumping into a button, while the terms "release," "released," or "releasing" may specifically, but not exclusively, refer to the removal of pressure from the button, in either case causing the position of the button relative to the surface of the device to change, which may exceed a predetermined threshold.
[0036] The operating button may be or may include an eject button. As used herein, the term "eject button" is a broad term and should be given its general and ordinary meaning to one skilled in the art, and should not be limited to a specific or special meaning. The term may specifically, but is not limited to, refer to a particular type of button that may be designed to provide an object, specifically a component of a device, to a user or object that presses the button. With particular reference to the present disclosure, the eject button is configured to eject a slide tray when the eject button is pressed. As used herein, the term "slide tray" is a broad term and should be given its general and ordinary meaning to one skilled in the art, and should not be limited to a specific or special meaning. The term may specifically, but is not limited to, refer to an element configured to hold at least one slide or slide holder, for example, during scanning of the slide in an imaging device.
[0037] As outlined above, the slide imaging equipment includes an operating system that may be configured to control the operation of at least one component of the slide imaging equipment, which may be selected from, for example, at least one of a feeding device and at least one imaging device.
[0038] An operating system may include at least one processing unit. As used herein, the term "processing unit" is a broad term and should be given its common and ordinary meaning to those skilled in the art, and should not be limited to any specific or special meaning. The term may specifically, but not exclusively, refer to any logic circuitry configured to perform the basic operations of a computer or system, and / or may generally refer to a device configured to perform calculations or logical operations. A processing unit may be configured to process the basic instructions that run a computer or system. By way of example, a processing unit may include at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math coprocessor or coprocessor, multiple registers, specifically registers configured to provide operands to the ALU and store calculation results, and memory, such as L1 and L2 cache memories. A processing unit may be a multi-core processor. Specifically, a processing unit may be or include a central processing unit (CPU). Additionally or alternatively, the processing unit may be or comprise a microprocessor, and thus, in particular, elements of the processing unit may be included on one single integrated circuit (IC) chip. Additionally or alternatively, the processing unit may be or comprise one or more application specific integrated circuits (ASICs) and / or one or more field programmable gate arrays (FPGAs), etc.
[0039] The operating system may include at least one communication interface. As used herein, the term "communication interface" is a broad term and should be given its common and ordinary meaning to those skilled in the art and should not be limited to a specific or special meaning. The term may specifically, but not exclusively, refer to an item or element forming a boundary configured to transfer information. In particular, the communication interface may be configured to transfer information from a computing device, such as a computer, for example, to transmit or output information to another device. Additionally or alternatively, the communication interface may be configured to transfer information to a computing device, such as a computer, for example, to receive information. The communication interface may specifically provide a means for transferring or exchanging information. In particular, the communication interface may provide a data transfer connection, such as Bluetooth, NFC, inductive coupling, or the like. By way of example, the communication interface may be or include at least one port comprising one or more of a network or internet port, a USB port, and a disk drive. The communication interface may also be at least one web interface.
[0040] The operating system may include at least one human-machine interface. As used herein, the term "human-machine interface" is a broad term and should be given its common and ordinary meaning to those skilled in the art, and should not be limited to a specific or special meaning. The term may specifically refer to an interface configured for human-machine interaction, such as, but not limited to, a keyboard, a display, a screen, a touchscreen, gesture and / or voice recognition. For example, the human-machine interface may include a keyboard and a display, such as the monitor described above, or an additional display. For example, the display may be configured to present at least one piece of information regarding the operating status of the slide imaging equipment to the user.
[0041] The positions of elements of the slide imaging equipment may be variable, adaptable, and / or adjustable within the frame. For example, at least one imaging device plate and / or the imaging device plate may be height-adjustable. In this case, the position, e.g., translation and / or orientation, of one or more imaging devices may change, and the positions of the operating buttons and additional imaging device elements may change accordingly. Additionally or alternatively, the position of the slide receiving portion of the imaging device may be variable, adaptable, and / or adjustable. However, knowledge of one or more of these positions, also referred to as target positions, may be essential for the feeder device to properly feed and / or introduce and / or remove slides from the imaging device. As used herein, the term "target position" is a broad term and should be given its common and ordinary meaning to those skilled in the art and should not be limited to a specific or special meaning. This term may specifically, but not exclusively, refer to the position of the imaging device and / or elements of the imaging device. The target position may be a position used to operate the feeder device. The target position may be a position to which the feeding device is driven to perform a predetermined action, e.g., one or more of grasping at least one slide, pressing a button, releasing at least one slide, etc. The target position may be the position of at least one operating button of the imaging device and / or at least one slide receiving portion of the imaging device. The relative positions of the elements of the imaging device, e.g., the operating button and / or the slide receiving portion, with respect to the imaging device may be known. For example, the relative positions may be stored in a database of the operating system. Thus, once the position of the imaging device in space, i.e., translation and orientation, is known, the positions of the elements of the imaging device are also known.
[0042] Typically, for programming a supplying device, a target position may be taught to the supplying device by a human. However, in the case of a change in the system, the teaching procedure must be repeated, which is complicated, time-consuming, and difficult. Therefore, the present invention proposes autonomous teaching of at least one target position.
[0043] The method includes, by way of example, the following steps, which may be performed in the given order. However, it should be noted that different orders are possible. Furthermore, one or more of the method steps may be performed once or repeatedly. Furthermore, two or more method steps may be performed simultaneously or overlapping in time. The method may include additional method steps not listed.
[0044] The method comprises the following steps: i) providing at least six predetermined locations by using an operating system; ii) using the operating system to drive the feeding device to six predetermined positions until it impacts the imaging device and detects the impact with the imaging device; iii) determining a target position by evaluating the collision detected by using the operating system; iv) is included.
[0045] As used herein, the term "teach" is a broad term and should be given its general and ordinary meaning to those skilled in the art, and should not be limited to a specific or special meaning. This term may specifically refer to, but is not limited to, a procedure for programming a robot. As used herein, the term "autonomous teach" is a broad term and should be given its general and ordinary meaning to those skilled in the art, and should not be limited to a specific or special meaning. This term may specifically refer to, but is not limited to, the fact that teaching a robot is performed without requiring manual or human interaction with the robot during teaching. To this end, at least six predetermined positions to which the feeding device is driven until it collides with the imaging device may be provided to the operating system. The coordinates, e.g., points, reached in this manner may be stored by the operating system, for example, in at least one database of the operating system. A program sequence for driving the feeding device to the six predetermined positions may include the feeding device moving autonomously to all of the predetermined positions.
[0046] As used herein, the term "providing" is a broad term and should be given its common and ordinary meaning to those skilled in the art and should not be limited to a specific or special meaning. This term may specifically refer to, but is not limited to, retrieving and / or selecting predetermined positions. Providing the six predetermined positions may include user input of the predetermined positions via a human-machine interface and / or receiving the six predetermined positions from an external database, such as an operating system database and / or an additional computer or cloud. Additionally, additional parameters related to movement between the individual positions, such as speed, acceleration, and / or accuracy, may be input by the user and / or retrieved from the database. The predetermined positions may be, for example, evenly distributed in space at the expected locations of the imaging devices. For example, the predetermined positions may be distributed on at least one expected plane on at least one side of the imaging device where the target location is located. In the case of multiple imaging devices, more than six predetermined positions may be used, such as six predetermined positions per imaging device.
[0047] Driving the feeder to each of the six predetermined positions may include moving the robot arm from an initial position along a path. The path may be predefined, e.g., preprogrammed. The operating system may drive the feeder along the path until a collision with the imaging device is detected. In step ii), the feeder may be continuously driven along each of the paths from the initial position to each of the six predetermined positions until a collision with the imaging device is detected. The feeder continues to be driven along each of the paths even after each predetermined position until a collision with the imaging device is detected. The operating system may be configured to limit the drive of the feeder along the path, e.g., taking into account a time limit. If a collision is not detected within a predetermined time limit of drive along the path, drive may be stopped and / or continued in a different direction. For example, if a collision is not detected within a predetermined time limit, an indication, e.g., a message and / or a request for user action, may be issued by the operating system via a human-machine interface.
[0048] The driving of the feeding device in step ii) may be performed by point-to-point (P2P) driving and / or by using continuous path (CP). In the case of P2P, the feeding device may be driven from position n to position n+1. The path between the points may be pre-programmed or calculated online, i.e. during teaching, taking into account detected collision points. In the case of CP, the feeding device may consider predetermined paths between six predetermined positions.
[0049] In step ii), the feeding device may be driven at a speed such that a collision with the imaging device does not cause deformation to the feeding device and the imaging device.
[0050] As used herein, the term "impact" is a broad term and should be given its common and ordinary meaning to one of ordinary skill in the art and should not be limited to a specific or particular meaning. The term may specifically, but is not limited to, refer to an interaction, e.g., contact and / or collision, between an imaging device and a supply device.
[0051] The detection of a collision with the imaging device may be performed by using at least one collision sensor. The collision sensor may include one or more of at least one optical sensor and at least one tactile sensor. The tactile sensor may be a mechanical tactile sensor, an inductive tactile sensor, and / or a capacitive tactile sensor. The collision sensor may be an element of the feeding device, for example, an element of the protrusion. Additionally or alternatively, the sensor may be an external sensor, for example, an imaging sensor of the slide imaging equipment. The slide imaging equipment may include at least one wired and / or wireless connection between the sensor and the operating system for exchanging data, such as sensor data for evaluation by the operating system, and / or commands, for example, for controlling the sensor. The point in space where the collision is detected may be referred to as the collision location.
[0052] Step iii) may include determining coordinates, e.g., 3D coordinates, of each collision location i, where i ranges from 1 to n and n is the number of predetermined locations. Evaluating the detected collisions may include determining a target location by solving a system of linear equations taking into account the coordinates of the determined collision locations. The target location may be defined by a six-dimensional pose including translations in three perpendicular axes x, y, and z and three orientation values rot(x), rot(y), and rot(z), hereinafter denoted as rotx, roty, and rotz. For example, the locations of the six collision locations may each have six values (X1', X2', Y1', Y2', Z1', and Z2'), where X1', Y1', Z1', X2', Y2', and Z2' define the initial location and X1, Y1, Z1, X2, Y2, and Z2 define the detected collision locations. The system of equations may be defined as follows: x=(X1.x-X2.x) / 2+X2.x y=(Y1.y-Y2.y) / 2+Y2.y z=(Z1.z-Z2.z) / 2+Z2.z rotx=atan 2 ((Y2.y-Y1.y),(Y2.x-Y1.x)) roty=0 rotz=atan 2 ((Z2.z-Z1.z),(Z2.x-Z1.x))
[0053] Step iii) may further include at least one coordinate transformation to Euler orientations. For example, the operating system may use Euler orientations q1, q2, q3, q4 to control the feeding device. For example, the following transformations may be performed: x1_:=cos(rotz)*cos(roty); x2_:=sin(rotz)*cos(roty); x3_:=-sin(roty); y1_:=cos(rotz)*sin(roty)*sin(rotx)-sin(rotz)*cos(rotx); y2_:=sin(rotz)*sin(roty)*sin(rotx)+cos(rotz)*cos(rotx); y3_:=cos(roty)*sin(rotx); z1_:=cos(rotz)*sin(roty)*cos(rotx)+sin(rotz)*sin(rotx); z2_:=sin(rotz)*sin(roty)*cos(rotx)-cos(rotz)*sin(rotx); z3_:=cos(roty)*cos(rotx); x1:=-z1_; x2:=-z2_; x3:=-z3_; y1:=x1_; y2:=x2_; y3:=x3_; z1:=-y1_; z2:=-y2_; z3:=-y3_; IF y3-z2>=0 THEN sigQ2:=1; ELSE sigQ2:=-1; ENDIF IF z1-x3>=0 THEN sigQ3:=1; ELSE sigQ3:=-1; ENDIF IF x2-y1>=0 THEN sigQ4:=1; ELSE sigQ4:=-1; ENDIF q1:=sqrt(x1+y2+z3+1) / 2; q2:=sigQ2*sqrt(x1-y2-z3+1) / 2; q3:=sigQ3*sqrt(y2-x1-z3+1) / 2; q4:=sigQ4*sqrt(z3-x1-y2+1) / 2;
[0054] After performing step iii), the operating system may have knowledge of the target positions at its own degrees of freedom. This method may allow teaching to be performed autonomously, particularly fully autonomously. Therefore, adjustments and changes to the positions of the imaging devices can be performed more quickly and with reduced complexity. For example, in a slide imaging installation having two imaging devices in two different positions, the positions of one or both imaging devices relative to the supply device may change during commissioning and / or after maintenance work. The operating system may perform the method described above and autonomously calculate the target positions of the operating buttons and / or slide receiving parts and teach them to the supply device.
[0055] The method may be computer-implemented. As used herein, the term "computer-implemented" is a broad term and should be given its common and ordinary meaning to those skilled in the art, and should not be limited to any specific or special meaning. The term may specifically refer to a process that is fully or partially implemented using data processing means, such as, but not limited to, a data processing means having at least one processing unit. Thus, the term "computer" may generally refer to a device having at least one data processing means, such as at least one processing unit, or a combination or network of devices. A computer may further comprise one or more additional components, such as at least one of a data storage device, an electronic interface, or a human-machine interface.
[0056] In a further aspect of the present invention, a slide imaging system is disclosed. - at least one imaging device configured to generate an image of a sample mounted on a slide, the at least one imaging device having at least one operating button and at least one slide receiving portion configured to receive a slide for generating an image; - at least one feeder configured to feed slides into a slide receiving portion of the imaging device, the at least one feeder configured to press an operating button; at least one operating system configured to control the operation of the supply device; the operating system comprises: driving the delivery device to at least six predetermined positions until it collides with the imaging device, and detecting the collision with the imaging device; Determining the position of the operating button and / or slide receptacle by evaluating the collision detected by using the operating system It is configured as follows.
[0057] The slide imaging facility may be configured to carry out the method according to the invention as described above or as explained in more detail below, and therefore reference may be made to the above description or to the description given in more detail below for possible definitions, options or embodiments.
[0058] Further disclosed and proposed herein is a computer program comprising computer-executable instructions which, when executed on a computer or a computer network, performs the method according to the present invention in one or more of the embodiments contained herein. In particular, the computer program may be stored on a computer-readable data carrier and / or a computer-readable storage medium.
[0059] As used herein, the terms "computer-readable data carrier" and "computer-readable storage medium" may specifically refer to non-transitory data storage means such as a hardware storage medium having computer-executable instructions stored thereon. A computer-readable data carrier or storage medium may specifically be or include a storage medium such as a random access memory (RAM) and / or a read-only memory (ROM).
[0060] Thus, in particular, one, several or even all of the method steps i) to iii) set out above may be implemented using a computer or a computer network, preferably by using a computer program.
[0061] Further disclosed and proposed herein is a computer program product, which comprises program code means for carrying out the method according to the invention in one or more of the embodiments contained herein when the program is run on a computer or a computer network. In particular, the program code means may be stored on a computer-readable data carrier and / or a computer-readable storage medium.
[0062] Further disclosed and proposed herein is a data carrier having stored thereon a data structure that, after being loaded into a computer or computer network, such as a working memory or main memory of the computer or computer network, is capable of performing the methods according to one or more of the embodiments disclosed herein.
[0063] Further disclosed and proposed herein is a non-transitory computer-readable storage medium containing instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to the present invention. The non-transitory computer-readable medium may be configured to control an operating system.
[0064] Further disclosed and proposed herein is a computer program product having program code means stored on a machine-readable carrier for performing the method according to one or more of the embodiments disclosed herein when the program is executed on a computer or a computer network. As used herein, a computer program product refers to a program as a tradeable product. The product can generally exist in any format, such as a paper format, or can exist on a computer-readable data carrier and / or a computer-readable storage medium. In particular, the computer program product may be distributed over a data network.
[0065] Finally, a modulated data signal containing instructions readable by a computer system or computer network for carrying out a method according to one or more of the embodiments disclosed herein is disclosed and suggested herein.
[0066] With respect to computer-implemented aspects of the present invention, one or more or all of the method steps of the methods according to one or more of the embodiments disclosed herein may be performed using a computer or a computer network. Thus, in general, any method step involving providing and / or manipulating data may be performed using a computer or a computer network. In general, these method steps may include any method step, except for method steps that typically require manual intervention, such as providing a sample and / or certain aspects of performing the actual measurement.
[0067] Specifically, in this specification, a computer or computer network comprising at least one processor, the processor being configured to execute a method according to one of the embodiments described herein, a computer-loadable data structure configured to perform a method according to one of the embodiments described herein when it is executed on a computer, a computer program configured, when it is run on a computer, to carry out a method according to one of the embodiments described herein; a computer program comprising program means for carrying out the method according to one of the embodiments described herein when said computer program is run on a computer or a computer network, a computer program comprising program means according to the preceding embodiment, the program means being stored on a computer-readable storage medium; a storage medium storing a data structure, the data structure being configured to perform a method according to one of the embodiments described herein after being loaded into a main memory and / or a working memory of a computer or a computer network, and a computer program product comprising program code means storable or stored on a storage medium, the program code means performing the method according to one of the embodiments described herein when the program code means is executed on a computer or a computer network; is further disclosed.
[0068] In summary, without excluding further embodiments, the following embodiments can be envisaged:
[0069] Embodiment 1. A method for autonomous teaching of at least one target position of a feeder of a slide imaging facility, comprising: the slide imaging equipment includes at least one imaging device configured to generate an image of a sample mounted on a slide, the target location being a location on the imaging device, and the slide imaging equipment includes at least one operating system configured to control operation of the feeder; The method comprises the following steps: i) providing at least six predetermined locations by using an operating system; ii) using the operating system to drive the feeding device to six predetermined positions until it impacts the imaging device and detects the impact with the imaging device; iii) determining a target position by evaluating the collision detected by using the operating system; A method comprising:
[0070] Embodiment 2. A method according to the preceding embodiment, wherein the target location is the location of at least one operating button of the imaging device and / or at least one slide receiving portion of the imaging device.
[0071] Embodiment 3. A method according to any one of the preceding embodiments, wherein the step of providing the six predetermined locations includes user input of the predetermined locations via at least one human-machine interface, and / or receiving the six predetermined locations from a database.
[0072] Embodiment 4. A method according to any one of the preceding embodiments, wherein the predetermined locations are distributed on at least one prospective plane on at least one side of the imaging device in which the target location is located.
[0073] Embodiment 5. A method according to any one of the preceding embodiments, wherein in step ii), the feeding device is driven at a speed such that a collision with the imaging device does not cause deformation to the feeding device and the imaging device.
[0074] Embodiment 6. A method according to any one of the preceding embodiments, wherein in step ii), the feeding device is driven successively along a path from the initial position to each of six predetermined positions until a collision with the imaging device, and the feeding device is driven along each of the paths past each predetermined position until a collision with the imaging device is detected.
[0075] Embodiment 7. A method according to any one of the preceding embodiments, wherein the detection of a collision with the imaging device is performed by using at least one collision sensor, the collision sensor comprising one or more of at least one optical sensor, at least one tactile sensor.
[0076] Embodiment 8. The method according to any one of the preceding embodiments, wherein step iii) includes determining coordinates of each collision location, and wherein evaluating the detected collisions includes determining a target location by solving a system of linear equations that takes into account the coordinates of the determined collision locations.
[0077] Embodiment 9. The method according to any one of the preceding embodiments, wherein step iii) includes at least one coordinate transformation to an Euler orientation.
[0078] Embodiment 10. The method according to any one of the preceding embodiments, wherein the method is computer-implemented.
[0079] Embodiment 11. At least one imaging device configured to generate an image of a sample mounted on a slide, the imaging device having at least one operating button and at least one slide receiving portion configured to receive a slide for generating an image; at least one feeder configured to feed slides into a slide receiving portion of the imaging device, the at least one feeder configured to press an operating button; at least one operating system configured to control operation of the supply device; the operating system comprises: driving the delivery device to at least six predetermined positions until it collides with the imaging device, and detecting the collision with the imaging device; Determining the position of the operating button and / or slide receptacle by evaluating the collision detected by using the operating system A slide imaging facility configured as follows.
[0080] Embodiment 12. A slide imaging installation according to the preceding embodiments, configured to carry out a method according to any one of the preceding embodiments relating to a method.
[0081] Embodiment 13. A slide imaging apparatus according to any one of the preceding embodiments relating to a slide imaging apparatus, wherein the operation button is an ejection button or includes an ejection button, and the imaging device is configured to eject the slide tray when the ejection button is pressed.
[0082] Embodiment 14. A slide imaging facility according to any one of the preceding embodiments relating to a slide imaging facility, wherein the slide imaging facility comprises at least one first imaging device and at least one second imaging device, each imaging device configured to generate an image of a sample mounted on a slide, and the supply device configured to selectively supply the slide to the first imaging device or the second imaging device.
[0083] Embodiment 15. A slide imaging installation according to any one of the preceding embodiments relating to a slide imaging installation, wherein the feeding device comprises at least one robotic arm.
[0084] Embodiment 16. A slide imaging system according to the preceding embodiment, wherein the supply device comprises a protrusion configured to press an operating button, the protrusion being spear-shaped or finger-shaped, and the protrusion being disposed on the robot arm.
[0085] Embodiment 17. A slide imaging equipment according to the preceding embodiment, wherein the slide imaging equipment includes a protective cover, and the protective cover is configured to protect at least one of the protrusion and the operation button.
[0086] Embodiment 18. A slide imaging facility according to any one of the preceding three embodiments, wherein the robotic arm comprises a gripping device configured to grip at least one of a slide or a slide holder configured to hold two or more slides.
[0087] Embodiment 19. A slide imaging facility according to the preceding embodiment, wherein the gripping device comprises a first gripping portion and a second gripping portion, the first gripping portion and the second gripping portion being movable relative to each other to grip at least one of a slide or a slide holder configured to hold two or more slides.
[0088] Embodiment 20. A slide imaging facility according to any one of the preceding embodiments relating to a slide imaging facility, wherein the slide imaging facility comprises at least one storage device capable of loading a plurality of slides and configured to store the slides, and the supply device is configured to supply the slides from the storage device to the imaging device.
[0089] Embodiment 21. A slide imaging facility according to the preceding embodiment, wherein the storage device is capable of being loaded with a slide holder, the slide holder being configured to hold a plurality of slides.
[0090] Embodiment 22. A slide imaging installation according to any one of the preceding two embodiments, wherein the supply device is configured to transport slides from the imaging device to the storage device.
[0091] Embodiment 23. A computer program including instructions, which when the program is executed by a slide imaging facility according to any one of the preceding embodiments relating to a slide imaging facility, cause the slide imaging facility to perform a method according to any one of the preceding embodiments relating to a method.
[0092] Embodiment 24. A computer-readable storage medium containing instructions, which when executed by a slide imaging facility according to any one of the preceding embodiments relating to a slide imaging facility, cause the slide imaging facility to perform a method according to any one of the preceding embodiments relating to a method.
[0093] Embodiment 25. A non-transitory computer-readable medium containing instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any one of the preceding embodiments relating to a method.
[0094] Further optional features and embodiments are disclosed in more detail in the subsequent description of the embodiments, preferably in conjunction with the dependent claims. Each optional feature therein may be realized in an independent manner as well as in any possible combination, as will be understood by those skilled in the art. The scope of the present invention is not limited by the preferred embodiments. The embodiments are illustrated diagrammatically in the figures, where identical reference numerals in these figures refer to identical or functionally equivalent elements. [Brief explanation of the drawings]
[0095] [Figure 1] 1 is a schematic side view of one embodiment of a slide imaging facility. [Figure 2]1 illustrates schematically an embodiment of a method for autonomous teaching of at least one target position of a feeder device of a slide imaging installation. DETAILED DESCRIPTION OF THE INVENTION
[0096] FIG. 1 schematically illustrates, in side view, one embodiment of a slide imaging equipment 110 according to the present disclosure. As shown in FIG. 1, the slide imaging equipment 110 may include a frame 112. The frame 112 may include wheels 114, thus allowing the slide imaging equipment 110 to be moved to a desired location. As further shown in FIG. 1, the frame 112 further includes a table 116. At least a storage device 118 and a supply device 120 may be attached to the table 116. However, additional types of arrangements for the storage device 118 and the supply device 120 may be envisioned.
[0097] 1 , the first plate 122 and the second plate 124 are connected to the frame 112, with the first imager 126 disposed on the first plate 122 and the second imager 128 disposed on the second plate 124. Further embodiments (not shown here) of the slide imaging apparatus 110 including only a single imager or three, four, five, six, or even more individual imagers may also be possible, with the feeder 120 being able to individually address at least one imager 126, 128, as disclosed above or described in more detail below. As a result of the particular arrangement shown in FIG. 1 , the first imager 126 and the second imager 128 are disposed one above the other. However, other types of arrangements for the first imager 126 and the second imager 128 may also be envisioned, particularly as long as they are reachable by the feeder 120. Furthermore, the first plate 122 and the second plate 124 may be extendable from the frame 112 in an independent manner, in particular by using telescoping rails (not shown here), to facilitate access, in particular in maintenance cases. In this embodiment, precise positioning can be achieved by recesses in the plates 122, 124 designed to receive the respective bases of the imaging devices 126, 128, respectively. However, further types of configurations may also be feasible.
[0098] 1, the slide imaging equipment 110 may include a housing 130 that at least partially encloses the feeding device 120 and thus supports unimpeded operation of the feeding device 120. As shown schematically in FIG. 1, the housing 130 has a safety door 132 and a safety switch 134 that is configured to detect the state of the safety door 132, such as an open or closed state of the safety door 132. However, additional types of safety measures may be possible.
[0099] 1 includes an emergency stop switch 136, which can be operated here using an emergency stop button 138. However, additional types of emergency equipment may be possible. When the emergency stop button 138 is pressed, the emergency stop switch 136 can immediately stop operation of the slide imaging equipment 110 if deemed necessary by service personnel.
[0100] As shown schematically in FIG. 1 , the storage device 118 is loadable with and configured to store a plurality of slides 140. The plurality of slides 140 may be manually loaded into the storage device 118. To this end, the storage device 118 may be loadable with at least one of individual slides and slide holders 142, each slide holder 142 configured to hold two or more slides 140. As further shown in FIG. 1 , the storage device 118 is configured to store the slides 140 or slide holders 142 in rows 144 arranged adjacent to one another. However, additional configurations of the storage device 118 may be envisioned.
[0101] In general, the supply device 120 may be configured to process slides 140 or slide holders 142 along a predetermined routine route, for example, starting from the top row 146 of the storage device 118 and continuing to the bottom row 148. In the particular embodiment shown here, the storage device 118 may include an express lane 150, which may be configured to store at least one sample mounted on a designated slide 152, and each designated slide 152 located in the express lane 150 may be designated for privileged processing outside of the predetermined routine route normally used by the supply device 120.
[0102] 1, the feeder 120 may be configured to selectively feed the slides 140 from the storage device 118 to the first imager 126 or the second imager 128. The feeder 120 may be configured to automatically feed the slides 140 from the storage device 118 selectively to the first imager 126 or the second imager 128.
[0103] As shown schematically therein, the supply device 120 may include, inter alia, a robotic arm 154 that may be configured to introduce the slide 140 into a slide receiving portion 156, particularly a slit, included in each of the first and second image capture devices 126 and 128, the slide receiving portion 156 being configured to receive the slide 140 for imaging purposes. The robotic arm 154 may include a gripping device 158 configured to grip the slide 140 or the holder 142. The gripping device 158 may have a protrusion (not shown here), particularly a spear-shaped or finger-shaped protrusion, configured to press an operating button 160 of the first or second image capture device 126 or 128. However, further types of configurations of the supply device 120 may also be possible.
[0104] Additionally, the supply device 120 may be configured to return the slides 140 from the first imager 126 or the second imager 128 to the storage device 118 after scanning. The slides 140 can then be manually removed from the storage device 118 to provide space for additional slides 140. The supply device 120 may be configured to return the slides 140 from the first imager 126 or the second imager 128 to the storage device 118 to the associated position within the storage device 118 where the slides 140 were loaded prior to scanning. As a result, the user may receive the slides 140 again in the same order in which they were provided to the storage device 118.
[0105] 1 includes a first imager 126 and a second imager 128, each configured to generate an image 162 of a sample mounted on a slide 140. To this end, each of the imagers 126, 128 may preferably be selected from a 2D camera or a line-scan detector. However, additional types of imagers may be feasible.
[0106] In certain embodiments, at least one of the first imager 126 and the second imager 128 may include at least one indicator (e.g., in the form of one or more LEDs 164) configured to indicate an operational status of the first imager 126 or the second imager 128. In this particular embodiment, the slide imaging equipment 110 may further include at least one vision sensor 166 configured to detect an operational status of at least one of the first imager 126 and the second imager 128 using an indication provided by the indicator (e.g., one or more LEDs 164), as shown in FIG. 1. To this end, the vision sensor 166 may include an optical recording device 168, as shown schematically in FIG. 1. In further embodiments (not shown here), the slide imaging equipment 110 may include at least two individual vision sensors 166, each of which may include an individual optical recording device 168, and each of which may be assigned to detect the operating state of an individual imaging device 126, 128. However, further embodiments are still possible.
[0107] In this manner, the slide imaging equipment 110 may independently access the operational status of each of the first imaging device 126 or the second imaging device 128 without having to obtain this information directly from the first imaging device 126 and the second imaging device 128. However, direct communication may also be possible between one side, i.e., the first imaging device 126 or the second imaging device 128, and the other side, i.e., the operating system 170, to provide the operational status of the first imaging device 126 and the second imaging device 128.
[0108] As described above, the slide imaging equipment 110 may include an operating system 170 that may be configured to control the operation of one or more of the supply device 120, the first imaging device 126, and the second imaging device 128.
[0109] Here, the operating system 170 may preferably include a processing device such as a computer 172 and a human-machine interface such as an input device 174 configured to input instructions to the computer 170, which may include a keyboard 176 and at least one display device 178. Furthermore, the slide imaging equipment 110 may include at least one monitor 180, which may preferably be mounted on a pivotable holder (not shown here) so that a user can easily observe the image 162 of the sample mounted on the slide 140 after it has been scanned by the first imaging device 126 or the second imaging device 128 from various positions.
[0110] Thus, the robotic arm 154 included in the feeder 120 has a protrusion 182 disposed thereon. The protrusion 182 can be attached to or formed on the robotic arm 154 in any conceivable manner. The robotic arm can include a gripping device 158 configured to grip a slide holder 142, which, as shown, is configured to hold a plurality of slides 140, with the protrusion 182 disposed on the gripping device 158. However, further arrangements of the protrusion 182 on the feeder 120 are also feasible. The protrusion 182 included in the feeder 120, specifically the gripping device 158, is configured to press an operating button 160 on the imaging devices 126, 128. For this purpose, the protrusion 182 can be spear-shaped or finger-shaped. The operating button 160 can be or include an ejection button, and the imaging devices 126, 128 can be configured to eject a slide tray (not shown here) when the ejection button is pressed. Here, the slide tray may be configured to hold slides 140 or slide holders 142 during scanning of at least one slide 140 .
[0111] The gripping device 158 may include a first gripping portion and a second gripping portion, which are not visible in FIG. 1 . The first gripping portion and the second gripping portion may be movable relative to each other, for example, linearly relative to each other. Furthermore, the first gripping portion and the second gripping portion may have opposing surfaces, which may preferably be substantially planar, so as to be received by adjacent surfaces of the slide 140 or slide holder 142. In this manner, the gripping device 158 may transfer the slide 140 or slide holder 142 to the slide receiving portion 156 of the imaging device 126, 128 in a tight manner, potentially avoiding instability or loss of the slide 140 or slide holder 142 during transport.
[0112] The positions of elements of the slide imaging equipment 110 may be variable, adaptable, and / or adjustable within the frame 112. For example, the plate of at least one of the imagers 126, 128 and / or plates 122, 124 may be height-adjustable. In this case, the position, e.g., translation and / or orientation, of one or more imagers may change, which may result in a corresponding change in the position of the operating button 160 and additional elements of the imagers 126, 128. Additionally or alternatively, the position of the slide receiving portion 156 of the imagers 126, 128 may be variable, adaptable, and / or adjustable. However, knowledge of one or more of these positions, also referred to as target positions, may be essential for the feeder 120 to properly feed and / or introduce and / or remove slides from the imagers 126, 128. The target positions may be the positions of the imagers 126, 128 and / or elements of the imagers 126, 128. The target positions may also be positions used to operate the feeder. The target position may be a position to which the feeder 120 is actuated to perform a predetermined action, such as one or more of grasping at least one slide 140, pressing a button, releasing at least one slide 140, etc. The target position may be the position of at least one operating button 160 of the imaging devices 126, 128 and / or at least one slide receiving portion 156 of the imaging devices 126, 128. The relative positions of elements of the imaging devices 126, 128, such as the operating button 160 and / or the slide receiving portion 156, relative to the imaging devices 126, 128, may be known. For example, the relative positions may be stored in a database of the operating system 170. Thus, once the positions, i.e., translation and orientation, of the imaging devices 126, 128 in space are known, the positions of the elements of the imaging devices 126, 128 are also known.
[0113] Typically, for programming the feeding device 120, a target position may be taught to the feeding device 120 by a human. However, in case of a change in the system, the teaching procedure must be repeated, which is complicated, time-consuming, and difficult. Therefore, the present invention proposes autonomous teaching of at least one target position.
[0114] 2 shows an embodiment for autonomously teaching at least one target position of a supply device 120 of a slide imaging system 110. The method includes, by way of example, the following steps, which may be performed in a given order. However, it should be noted that a different order is also possible. Furthermore, one or more of the method steps may be performed once or repeatedly. Furthermore, two or more method steps may be performed simultaneously or overlapping in time. The method may also include further method steps not listed.
[0115] The method comprises the following steps: i) providing at least six predetermined locations by using the operating system 170 (indicated by reference numeral 186); ii) driving the feeder 120 to six predetermined positions until it collides with the imagers 126, 128 by using the operating system 170 (indicated by reference numeral 188) and detecting the collision with the imagers 126, 128; iii) determining a target position by evaluating the detected collisions by using the operating system 170 (denoted by reference numeral 190); Includes.
[0116] The teaching may be or may include a procedure for programming the feeding device 120. The teaching of the feeding device 120 is performed without manual or human interaction with the robot during teaching. To this end, at least six predetermined positions to which the feeding device 120 is driven until it collides with the imaging devices 126, 128 may be provided to the operating system 170. The coordinates, e.g., points, reached in this manner may be stored by the operating system 170, for example in at least one database of the operating system 170. The program sequence for driving the feeding device 120 to the six predetermined positions may include the feeding device 120 moving autonomously to all of the predetermined positions.
[0117] The providing step 186 may include retrieving and / or selecting the predetermined positions. The providing step 186 of the six predetermined positions may include user input of the predetermined positions via a human-machine interface and / or receiving the six predetermined positions from, for example, the operating system 170's database and / or an external database, such as a further computer or the cloud. Additionally, additional parameters related to movement between the individual positions, such as speed and / or acceleration and / or accuracy, may be input by the user and / or retrieved from the database. The predetermined positions may be, for example, evenly distributed in space at expected positions of the imaging devices 126, 128. For example, the predetermined positions may be distributed on at least one expected plane on at least one side of the imaging devices 126, 128 where the target positions are located.
[0118] Driving the feeder 120 to each of the six predetermined positions may involve moving the robot arm 154 from an initial position along a path. The path may be predefined, e.g., preprogrammed. The operating system 170 may drive the feeder 120 along the path until a collision with the imaging devices 126, 128 is detected. In step ii) 188, the feeder 120 may be continuously driven along the path from the initial position to each of the six predetermined positions until a collision with the imaging devices 126, 128 is detected. The feeder 120 continues to be driven along each path even after each predetermined position until a collision with the imaging devices 126, 128 is detected. The operating system 170 may be configured to limit the drive of the feeder 120 along the path, e.g., taking into account a time limit. If a collision is not detected within a predetermined time limit of drive along the path, drive may cease and / or continue in a different direction. For example, if a collision is not detected within a predetermined time limit, an indication, such as a message and / or a request for user action, may be issued by operating system 170 via the human-machine interface.
[0119] The driving of the feeding device 120 in step ii) 188 may be performed by point-to-point (P2P) driving and / or by using continuous path (CP). In the case of P2P, the feeding device 120 may be driven from position n to position n+1. The path between the points may be pre-programmed or calculated online, i.e. during teaching, taking into account detected collision points. In the case of CP, the feeding device 120 may consider a predetermined path between six predetermined positions.
[0120] In step ii) 188, the feeder 120 may be driven at a speed such that a collision with the imaging device does not cause deformation to the feeder or imaging device. The collision may be or may include an interaction between the imaging device 126, 128 and the feeder 120, such as contact and / or collision.
[0121] The detection of a collision with the imaging devices 126, 128 may be performed by using at least one collision sensor (not shown). The collision sensor may include one or more of at least one optical sensor and at least one tactile sensor. The tactile sensor may be a mechanical tactile sensor, an inductive tactile sensor, and / or a capacitive tactile sensor. The collision sensor may be an element of the feeding device, such as an element of the protrusion. Additionally or alternatively, the sensor may be an external sensor, such as an imaging sensor of the slide imaging equipment. The slide imaging equipment 110 may include at least one wired and / or wireless connection between the sensor and the operating system 170 for exchanging data, such as sensor data for evaluation by the operating system 170, and / or commands, such as for controlling the sensor. The point in space where the collision is detected may be referred to as the collision location.
[0122] Step iii) 190 may include determining coordinates, e.g., 3D coordinates, of each collision location i, where i ranges from 1 to n and n is the number of predetermined locations. Evaluating the detected collisions may include determining a target location by solving a system of linear equations taking into account the coordinates of the determined collision locations. The target location may be defined by a six-dimensional pose including translations in three perpendicular axes x, y, and z and three orientation values rot(x), rot(y), and rot(z), hereinafter denoted as rotx, roty, and rotz. For example, the locations of the six collision locations may each have six values (X1', X2', Y1', Y2', Z1', and Z2'), where X1', Y1', Z1', X2', Y2', and Z2' define the initial location and X1, Y1, Z1, X2, Y2, and Z2 define the detected collision locations. The system of equations may be defined as follows: x=(X1.x-X2.x) / 2+X2.x y=(Y1.y-Y2.y) / 2+Y2.y z=(Z1.z-Z2.z) / 2+Z2.z rotx=atan 2 ((Y2.y-Y1.y),(Y2.x-Y1.x)) roty=0 rotz=atan 2 ((Z2.z-Z1.z),(Z2.x-Z1.x))
[0123] Step iii) 190 may further include at least one coordinate transformation to Euler orientations. For example, operating system 170 may use Euler orientations q1, q2, q3, q4 to control feeding device 120. For example, the following transformations may be performed: x1_:=cos(rotz)*cos(roty); x2_:=sin(rotz)*cos(roty); x3_:=-sin(roty); y1_:=cos(rotz)*sin(roty)*sin(rotx)-sin(rotz)*cos(rotx); y2_:=sin(rotz)*sin(roty)*sin(rotx)+cos(rotz)*cos(rotx); y3_:=cos(roty)*sin(rotx); z1_:=cos(rotz)*sin(roty)*cos(rotx)+sin(rotz)*sin(rotx); z2_:=sin(rotz)*sin(roty)*cos(rotx)-cos(rotz)*sin(rotx); z3_:=cos(roty)*cos(rotx); x1:=-z1_; x2:=-z2_; x3:=-z3_; y1:=x1_; y2:=x2_; y3:=x3_; z1:=-y1_; z2:=-y2_; z3:=-y3_; IF y3-z2>=0 THEN sigQ2:=1; ELSE sigQ2:=-1; ENDIF IF z1-x3>=0 THEN sigQ3:=1; ELSE sigQ3:=-1; ENDIF IF x2-y1>=0 THEN sigQ4:=1; ELSE sigQ4:=-1; ENDIF q1:=sqrt(x1+y2+z3+1) / 2; q2:=sigQ2*sqrt(x1-y2-z3+1) / 2; q3:=sigQ3*sqrt(y2-x1-z3+1) / 2; q4:=sigQ4*sqrt(z3-x1-y2+1) / 2;
[0124] After performing step iii) 190, the operating system 170 may have knowledge of the target positions at its own discretion. This method may allow teaching to be performed autonomously, particularly fully autonomously. Therefore, adjustments and changes to the positions of the imaging devices may be performed more quickly and with reduced complexity. For example, in a slide imaging installation 110 having two imaging devices 126, 128 in two different positions, the positions of one or both imaging devices 126, 128 relative to the supply device 120 may change during commissioning and / or after maintenance work. The operating system 170 may execute the method described above and autonomously calculate the target positions of the operating button 160 and / or the slide receiving portion 156 and teach them to the supply device 120. [Explanation of symbols]
[0125] 110 Slide Imaging Equipment 112 frames 114 Wheels 116 tables 118 Storage device 120 Feeding device 122 First Plate 124 Second Plate 126 First imaging device 128 Second Imaging Device 130 Housing 132 Safety Door 134 Safety Switch 136 Emergency stop switch 138 Emergency Stop Button 140 slides 142 Slide Holder 144 columns 146 Top row 148 Bottom row 150 Express Lane 152 designated slides 154 Robot Arm 156 Slide receiving part 158 Gripping device 160 Operation Buttons 162 images 164 Light Emitting Diode (LED) 166 Vision Sensor 168 Optical Recording Device 170 Operating Systems 172 Computers 174 Input Devices 176 keyboards 178 displays 180 monitors 182 Protrusion 186 Step i) 188 Step ii) 190 Step iii)
Claims
1. A method for autonomous teaching of at least one target position of a feeder (120) of a slide imaging facility (110), comprising: the slide imaging facility (110) comprises at least one imaging device (126, 128) configured to generate an image of a sample mounted on a slide (140), the target location being a location on the imaging device (126, 128), the slide imaging facility (110) comprises at least one operating system (170) configured to control the operation of the feeder device (120); The method comprises the following steps: i) providing (186) at least six predefined locations by using said operating system (170); ii) driving (188) the supplying device (120) to the six predetermined positions by using the operating system (170) until it impacts the imaging device (126, 128) and detecting the impact with the imaging device (126, 128); iii) determining (190) the target position by evaluating the detected collisions by using the operating system (170); A method comprising:
2. 2. The method of claim 1, wherein the target location is a location of at least one control button (160) of the imaging device (126, 128) and / or at least one slide receiving portion (156) of the imaging device (126, 128).
3. 3. The method of claim 1 or 2, wherein in step ii) (188), the feeding device (120) is continuously driven along a path from an initial position to each of the six predetermined positions until a collision with the imaging device (126, 128), and the feeding device (120) is driven along each of the paths past each of the predetermined positions until a collision with the imaging device (126, 128) is detected.
4. The method according to any one of claims 1 to 3, wherein detection of a collision with the imaging device (126, 128) is performed by using at least one collision sensor, the collision sensor comprising one or more of at least one optical sensor, at least one tactile sensor.
5. 5. The method of claim 1, wherein step iii) (190) comprises determining coordinates of respective collision locations, and wherein evaluating the detected collisions comprises determining the target location by solving a system of linear equations that takes into account the determined collision location coordinates.
6. The method of any one of claims 1 to 5, wherein the method is computer-implemented.
7. at least one imaging device (126, 128) configured to generate an image of a sample mounted on a slide (140), the at least one imaging device (126, 128) having at least one operating button (160) and at least one slide receiving portion (156) configured to receive the slide (140) for generating the image; at least one feeder (120) configured to feed a slide (140) to the slide receiving portion (156) of the imaging device (126, 128), the feeder (120) configured to press the operating button (160); at least one collision sensor configured to detect a collision between the feeding device (120) and the imaging device (126, 128); at least one operating system (170) configured to control the operation of said supply device (120); It is equipped with the operating system (170) is configured to drive the feeding device (120) to at least six predetermined positions until it impacts the imaging device (126, 128) and detect a collision with the imaging device (126, 128) by using the collision sensor; The operating system (170) is configured to determine the position of the operation button (160) and / or the slide receiving portion (156) by evaluating the detected collision by using the operating system (170), and the slide imaging equipment (110).
8. The slide imaging facility (110) according to claim 7, wherein the slide imaging facility (110) is configured to carry out the method according to any one of claims 1 to 6.
9. The slide imaging equipment (110) of claim 7 or 8, wherein the operation button (160) is an ejection button or includes an ejection button, and the imaging device (126, 128) is configured to eject a slide tray when the ejection button is pressed.
10. The slide imaging equipment (110) of any one of claims 7 to 9, comprising at least one first imaging device (126) and at least one second imaging device (128), each imaging device configured to generate an image of a sample mounted on a slide (140), and the supply device (120) configured to selectively supply the slide (140) to the first imaging device (126) or the second imaging device (128).
11. The slide imaging equipment (110) of any one of claims 7 to 10, wherein the supply device (120) comprises at least one robot arm (154), the supply device (120) comprises a protrusion (182) configured to press the operating button (160), the protrusion (182) being spear-shaped or finger-shaped, and the protrusion (182) being arranged on the robot arm (154).
12. The slide imaging equipment (110) of any one of claims 7 to 11, comprising at least one storage device (118) capable of loading a plurality of slides (140) and configured to store the slides (140), the supply device (120) configured to supply the slides (140) from the storage device (118) to the imaging devices (126, 128), and the supply device (120) configured to transport the slides (140) from the imaging devices (126, 128) to the storage device (118).
13. A computer program comprising instructions, A computer program, the instructions causing the slide imaging equipment (110) to perform the method according to any one of claims 1 to 6 when the program is executed by the slide imaging equipment (110) described in any one of claims 7 to 12.
14. A computer-readable storage medium containing instructions, comprising: A computer-readable storage medium, the instructions causing the slide imaging equipment (110) to perform a method according to any one of claims 1 to 6 when the instructions are executed by the slide imaging equipment (110) described in any one of claims 7 to 12.
15. A non-transitory computer-readable medium containing instructions, A non-transitory computer readable medium, the instructions, when executed by one or more processors, causing the one or more processors to perform the method of any one of claims 1 to 6.
Citation Information
Patent Citations
Microscope
EP0245089A2
Method and apparatus for automated assay of biological specimens
EP0534247A2
Self teaching robot
US20020068992A1
Automated robot teach tool and method of use
US20090302795A1
Automated tissue section capture, indexing and storage system and methods
US20190301980A1