Slide scan device and operating method therefor

Abstract

The aim of an embodiment of the present disclosure is to provide a slide scan device. In an embodiment, the slide scan device may comprise: a slide tray slot for accommodating one or more slide trays; a fork for transporting a slide tray; a preview camera for acquiring a preview image for the slide tray; and an optical device for acquiring a scan image for one or more slides arranged on the slide tray.

Description

Slide scanning device and method of operation thereof

[0001] The present disclosure relates to a slide scanning device and an operating method thereof.

[0002] Pathology provides a foundation for clinical treatment by observing and analyzing morphological changes in a patient's cells, tissues, and tissue fluids to make diagnoses. Traditionally, pathologists performed cytology or tissue biopsies using a microscope to interpret and diagnose patient slides of cells or tissues. However, with the recent introduction of digital slide scanners, pathology diagnoses are now performed using digital images of scanned cell or tissue stained slides, in addition to microscopic pathology diagnoses.

[0003] Slides containing patient specimens are accommodated in a tray, the slide tray is accommodated in a slide tray slot, the slide tray accommodated in the slide tray slot is scanned by a digital slide scanner or the like, and as the scan is performed, information on the slides accommodated in the slide tray is acquired, so that a pathological diagnosis can be performed.

[0004] Much research is being conducted on the automation of slide scanning devices to obtain information on more slides in a shorter period of time.

[0005] The present disclosure seeks to provide an automated slide scanning device.

[0006] One embodiment of the present disclosure is to provide an automated slide scanning device and a slide scanning method using the same.

[0007] A slide scanning device according to one embodiment of the present disclosure

[0008] Slide tray slots accommodating one or more slide trays;

[0009] A fork for carrying the above slide tray;

[0010] A preview camera for obtaining a preview image for the above slide tray; and

[0011] An optical device for acquiring scan images of one or more slides placed on the slide tray.

[0012] may include.

[0013] In one embodiment, the fork comprises:

[0014] It may be possible to extract a slide tray accommodated in the slide tray slot and transfer it to at least one of an obtainable position of the preview image and an obtainable position of the scan image.

[0015] In one embodiment, the scanned image of one or more of the slides,

[0016] It may be obtained after obtaining the above preview image.

[0017] In one embodiment, the slide scanning device comprises:

[0018] Further comprising a shuttle positioned at a position corresponding to the optical device;

[0019] The above shuttle,

[0020] It may be moved in the xy plane to obtain scan images for one or more of the above slides.

[0021] In one embodiment, the optical device comprises:

[0022] Includes a scan camera, a telescope, a z-axis travel unit, and an objective lens,

[0023] The above objective lens is,

[0024] It may be moved along the z-axis according to an auto-focus algorithm by the above z-axis moving part.

[0025] In one embodiment, the slide scanning device comprises:

[0026] display;

[0027] memory for storing one or more instructions; and

[0028] At least one processor for executing one or more instructions stored in the memory,

[0029] The at least one processor executes the one or more instructions,

[0030] The display can be controlled to output an image of at least one of the preview image and the scanned image of the one or more slides.

[0031] In one embodiment, the display comprises:

[0032] It may be provided on the door of the above slide scanning device.

[0033] In one embodiment, the door comprises:

[0034] It may be installed so that it can be slid.

[0035] One embodiment of the present disclosure is a slide scanning method of the slide scanning device,

[0036] Step of obtaining a scan request for the first slide tray;

[0037] A step of transferring the first slide tray, which is accommodated in the slide tray slot, to an obtainable position of the preview image;

[0038] A step of the above preview camera obtaining a preview image of the first slide tray;

[0039] The step of the fork moving the first slide tray to an obtainable position of the scan image; and

[0040] The method comprises the step of obtaining a scan image of one or more slides placed on the first slide tray by the optical device.

[0041] In one embodiment, the step of the fork moving the first slide tray to an obtainable position of the scan image is:

[0042] The fork may include a step of transferring the first slide tray to the shuttle.

[0043] In one embodiment, the step of obtaining scan images of one or more slides placed on the first slide tray by the optical device comprises:

[0044] (a) a step of extracting the ROI (region of interest) of the first slide;

[0045] (b) a step of performing IAF (Image Autofocusing) at the first point of the ROI to determine a reference focus position (RFP);

[0046] (c) performing LAF (Laser Autofocusing) at a second point of the ROI based on the determined RFP; and

[0047] (d) may include a step of obtaining a scan image for the ROI of the first slide by repeating the above step (c) for the entire ROI of the first slide.

[0048] In one embodiment, the first point may be a point determined as a point where a sample exists in the ROI.

[0049] In one embodiment, the first point may be determined by the autofocus performing device as the center of gravity point of the sample or the center point of the ROI.

[0050] In one embodiment, the step of acquiring the scan image may further include: identifying a center of gravity of the sample; determining whether a sample exists at the center of gravity; and, if it is determined that a sample does not exist at the center of gravity, determining a point where the sample exists closest to the center of gravity as the first point.

[0051] In one embodiment, the step of acquiring the scan image may further include: identifying coordinates of a center point of the ROI; determining whether a sample exists at the center point of the ROI; and, if it is determined that a sample does not exist at the center point of the ROI, determining a point where a sample exists closest to the center point of the ROI as the first point.

[0052] In one embodiment, the RFP may be determined based on a peak level of a laser reflected from the upper surface of the cover glass, the lower surface of the cover glass, the upper surface of the slide glass, or the lower surface of the slide glass.

[0053] One embodiment of the present disclosure includes a program stored on a recording medium to cause a computer to execute a method according to one embodiment of the present disclosure.

[0054] One embodiment of the present disclosure includes a computer-readable recording medium having recorded thereon a program for executing a method according to one embodiment of the present disclosure on a computer.

[0055] One embodiment of the present disclosure includes a computer-readable recording medium having recorded thereon a database used in one embodiment of the present disclosure.

[0056] According to one embodiment of the present disclosure, scan information of a large number of slides can be obtained quickly and easily.

[0057] FIG. 1 is an exemplary diagram of a slide scanning device according to one embodiment of the present disclosure.

[0058] FIG. 2 is a schematic diagram of the internal structure of a slide scanning device according to one embodiment of the present disclosure.

[0059] FIG. 3 is an exemplary diagram of a slide scanning device for obtaining a preview image according to one embodiment of the present disclosure.

[0060] FIG. 4 is a drawing illustrating a method of extracting or inserting a slide tray from a slide tray slot according to one embodiment of the present disclosure.

[0061] FIG. 5 is a drawing showing a state in which a slide tray according to one embodiment of the present disclosure is transported by a shuttle.

[0062] FIG. 6 is a diagram illustrating a method for obtaining a scan image according to one embodiment of the present disclosure.

[0063] FIG. 7A is a diagram for explaining a method for determining a reference focus position (RFP) according to one embodiment of the present disclosure.

[0064] FIG. 7b is an example graph of peak information according to one embodiment of the present disclosure.

[0065] FIG. 8 is a diagram illustrating a camera module performing an autofocus procedure according to one embodiment of the present disclosure.

[0066] FIG. 9 is a flowchart of a method for scanning a slide by a slide scanning device according to one embodiment of the present disclosure.

[0067] FIG. 10 is a block diagram of a slide scanning device according to one embodiment of the present disclosure.

[0068] FIG. 11 is an exemplary diagram of a slide scanning device according to one embodiment of the present disclosure.

[0069] FIG. 12 is a drawing illustrating a slide tray according to one embodiment of the present disclosure.

[0070] FIG. 13 is a drawing illustrating a slide according to one embodiment of the present disclosure.

[0071] FIG. 14 is a drawing illustrating a slide scanning device according to one embodiment of the present disclosure.

[0072] FIG. 15 is a diagram illustrating a method for extracting a Region Of Interest (ROI) according to one embodiment of the present disclosure.

[0073] FIG. 16 is a flowchart illustrating a method for scanning an ROI according to one embodiment of the present disclosure.

[0074] FIG. 17A is a diagram illustrating a method for determining a first point for performing passive auto-focus according to one embodiment of the present disclosure.

[0075] FIG. 17b is a diagram illustrating a method for determining a first point for performing passive auto-focus according to one embodiment of the present disclosure.

[0076] To clarify the technical idea of ​​the present disclosure, embodiments of the present disclosure will be described in detail with reference to the attached drawings. In describing the present disclosure, if a detailed description of a related known function or component is determined to unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. Components having substantially the same functional configuration among the drawings are given the same reference numbers and symbols as possible even if they are shown in different drawings. For convenience of explanation, devices and methods are described together when necessary. Each operation of the present disclosure does not necessarily have to be performed in the described order and may be performed in parallel, selectively, or individually.

[0077] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, the applicant may arbitrarily select terms, and in such cases, their meanings will be described in detail in the description of the relevant embodiments. Therefore, the terms used in this specification should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0078] Throughout this disclosure, singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms such as "comprise" or "have" should be understood to indicate the presence of a feature, number, step, operation, component, part, or combination thereof, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In other words, when it is said throughout this disclosure that a part "comprises" a certain component, unless specifically stated otherwise, this does not mean that other components may be included, but rather that other components may be excluded.

[0079] Expressions such as "at least one" modify the entire list of elements, not individual elements of the list. For example, "at least one of A, B, and C" and "at least one of A, B, or C" refer to only A, only B, only C, both A and B, both B and C, both A and C, all of A, B, and C, or any combination thereof.

[0080] In addition, terms such as “...unit”, “...module”, etc. described in the present disclosure mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0081] Throughout this disclosure, when a part is said to be "connected" to another part, this includes not only cases where the parts are "directly connected," but also cases where the parts are "electrically connected" with other elements intervening. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes other components, unless otherwise specifically stated.

[0082] The expression “configured to” as used throughout this disclosure can be used interchangeably with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” does not necessarily mean something that is “specifically designed to” in terms of hardware. Instead, in some contexts, the expression “a system configured to” can mean that the system, together with other devices or components, is “capable of.” For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor (e.g., an embedded processor) for performing those operations, or a generic-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in memory.

[0083] Terms including ordinal numbers, such as "first," "second," etc., may be used to describe various components, but the components are not limited by the terms. The terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component." The term "and / or" encompasses any combination of multiple related items or any one of multiple related items.

[0084] Throughout this disclosure, a slide is a thin, flat piece of glass used to hold a specimen, and may include a slide glass, a glass slide, a specimen slide, a microscope slide, a slide glass with a specimen placed on it and then covered with a cover glass, etc. Of course, the present invention is not limited to the above examples.

[0085] Throughout this disclosure, a slide tray is a case for transporting or storing slides, and may also be referred to as a tray, glass plate, slide plate, slide holder, glass holder, slide case, glass case, slide folder, glass folder, etc.

[0086] The term "sample" as used herein is used interchangeably with "sample" and can include anything known in the art to be observable by optical or light microscopes. For example, a "sample" can include, but is not limited to, cells or tissues (e.g., cells or tissues of animals, plants, fungi, protozoa, bacteria, etc. - including both normal and pathological cells / tissues), cell components (e.g., nucleus, cytoplasm, chloroplasts, mitochondria, etc.), microorganisms (e.g., bacteria, protozoa, some algae, and fungi, etc.), organoids, etc.

[0087] The term "about" as used herein means within 10%, preferably within 5%, and more preferably within 1% of a given value or range.

[0088] FIG. 1 is an exemplary diagram of a slide scanning device according to one embodiment of the present disclosure.

[0089] Referring to FIG. 1, the slide scanning device (100) may include a door (110). In one embodiment, the door (110) may include a display (120) and a handle (130). For example, the display (120) may include a touch monitor or a touch screen. Additionally, the display (120) may be fixed to a bracket, such that the bracket may function as a door. In one embodiment, the door (110) may be installed in a sliding manner.

[0090] In one embodiment, the information displayed on the display (120) may be designed to vary depending on whether the door (110) is fully open, open, or closed. That is, the slide scanning device (100) may identify at least one of the closed, open, and fully open states of the door (110), and control the display (120) to output a user interface (UI) corresponding to the state of the door based on the state of the door. For example, when the door (110) is open or fully open, the slide trays may be exposed to the outside or safety may not be guaranteed due to movement of the equipment, so the scanning operation of the slide scanning device may be controlled to be impossible. Accordingly, when the door (110) is open or fully open, information indicating that the scanning process of the slide is impossible due to the open state of the door may be displayed on the display (120).

[0091] According to one embodiment, unlike conventional equipment that required a large installation space by providing scan information of slides on a separate monitor, a slide scanning device that can be miniaturized by having the monitor integrated into the form of a door can be provided.

[0092] FIG. 2 is a schematic diagram of the internal structure of a slide scanning device according to one embodiment of the present disclosure.

[0093] Referring to FIG. 2, the slide scanning device (100) may include a slide tray slot (210) that accommodates one or more slide trays (260) therein, a fork (220) that carries the slide tray, a fork support (270) to which the fork (220) is slidably connected so that the fork can move up and down, a preview camera (230) that obtains a preview image of the slide tray, an optical device (240) that obtains a scan image of one or more slides placed on the slide tray, a shuttle (250) that is positioned corresponding to the optical device to obtain a scan image and moves in the xy plane according to a predetermined algorithm during slide scanning, a lighting unit (280), etc. However, not all of the components illustrated in FIG. 2 are essential components of the slide scanning device (100). The slide scanning device (100) may be implemented by more components than the components illustrated in FIG. 2, or by fewer components than the components illustrated in FIG. 2.

[0094] In one embodiment, the slide tray slot (210) is a device for storing slide trays, and may be referred to as a cabinet for storing slide trays, a slide tray cabinet, a slide tray storage cabinet, a slide tray rack, etc. The slide tray slot (210) may include a plurality of slide tray storage portions arranged in a vertical stack.

[0095] In one embodiment, the fork (220) can take out the slide tray (260) accommodated in the slide tray slot (210) and move it to a position where a preview image can be obtained. In addition, when the preview image is acquired, the fork (220) can again accommodate the slide tray (260) in the slide tray receiving portion of the slide tray slot (210).

[0096] In one embodiment, the fork (220) can remove the slide tray (260) accommodated in the slide tray slot (210) and move it to a position where scanned images of the slides can be obtained. In addition, when the scanned images are acquired, the fork (220) can again accommodate the slide tray (260) in the slide tray receiving portion of the slide tray slot (210).

[0097] In one embodiment, the slide scanning device (100) can obtain a preview image for the slide tray (260), recognize information about the slide tray, the slide, or the pathology sample from the preview image, obtain a slide scan image based thereon, and store the slide scan image in association with the information about the slide tray, the slide, or the pathology sample. That is, the scan images of the slides can be obtained after the preview images are obtained. That is, the slide scanning device (100) can obtain a preview image by photographing the slide tray (260) transferred to an obtainable position of the preview image by the fork (220) using the preview camera (230), and can obtain a scan image after transferring the slide tray (260) to an obtainable position of the scan image via the fork (220).

[0098] In one embodiment, while the slide tray (260) is placed on the fork (220), preview images of a plurality of slides can be acquired using the preview camera (230), and scan images of pathology samples contained in each slide can be acquired using the optical device (240).

[0099] In one embodiment, the fork (220) can move the slide tray (260) to a position where a preview image can be acquired by the preview camera (230), and after acquiring the preview image, the fork (220) can move the slide tray (260) back to a position where an image of the pathology sample can be acquired using the optical device (240). In another embodiment, a first fork can move the slide tray (260) to a position where a preview image can be acquired by the preview camera (230), and after acquiring the preview image, a second fork (separate from the first fork) can move the slide tray (260) to a position where an image of the pathology sample can be acquired using the optical device (240).

[0100] In one embodiment, the preview camera (230) can obtain preview images of a plurality of slides placed on the fork (220). For example, the preview camera (230) can obtain an entire image of the slide tray (260) as a preview image. As another example, the preview camera (230) can obtain images of some of the slides placed on the slide tray (260) as a preview image.

[0101] In one embodiment, the fork (220) may be stopped at a predetermined distance from the preview camera (230) so that the preview camera (230) can obtain preview images of a plurality of slides. That is, for example, the fork (220) may move the slide tray (260) to a predetermined position where the preview camera (230) can obtain an entire image of the slide tray (260). In one embodiment, the preview camera (230) may have a resolution suitable for obtaining information desired to be derived from the preview image. For example, the preview camera (230) may be configured as a camera with a lower resolution than the optical device (240).

[0102] In one embodiment, the preview camera (230) may include one or more lens(es), an image sensor, and a focus control unit. The preview camera (230) may use the focus control unit to adjust the focus on the slide tray and acquire an image passing through one or more lens(es) using the image sensor.

[0103] In one embodiment, the fork (220) can move the slide tray (260) to a predetermined position so that the optical device (240) can scan an area of ​​the slide. For example, the fork (220) can transport the slide tray (260) over the shuttle (250).

[0104] In one embodiment, the optical device (240) can acquire scan images of pathology samples included in each slide. The specific configuration of the optical device (240) will be described in more detail later with reference to FIG. 8. In one embodiment, the optical device (240) can divide a slide into multiple regions, acquire scan images for each region, and then combine them to acquire an image of the entire slide. For example, the optical device (240) can divide the slide region into a square tile shape and acquire scan images for each region. As another example, the optical device (240) can divide the slide region into multiple lines, acquire scan images using a line scan method, and then combine them. As another example, the optical device (240) can acquire an image of the entire slide at once. However, the method by which the optical device (240) acquires an image of the slide is not limited to this, and two or more of the methods exemplified above can be combined and used.

[0105] In one embodiment, the optical device (240) may include one or more lens(es), an image sensor, and a focus control unit. The optical device (240) may use the focus control unit to adjust the focus for each area of ​​the slide to be scanned, and may acquire an image passing through one or more lens(es) using the image sensor. The method by which the optical device (240) acquires a scanned image will be described in more detail below with reference to FIGS. 6 to 8.

[0106] In one embodiment, the fork (220) can accommodate a slide tray (260) on its upper surface. The fork (220) may be provided with a fixing means (not shown) capable of fixing the slide tray (260) on its upper surface. For example, the fork (220) may be provided with a means for fixing all or part of the slide tray (260) in a vertical or horizontal direction.

[0107] In one embodiment, the shuttle (250) can be moved so that the optical device (240) can scan the next area after completing a scan of one area of ​​the slide. For example, the shuttle (250) can be moved along the scan direction when the optical device (240) performs LAF (Laser AutoFocus). In another example, when the optical device (240) uses a line scan method, the shuttle (250) can be moved together with the slide tray (260) for the line scan. For example, the shuttle (250) on which the slide tray (260) is placed can be moved according to the line scan speed of the optical device (240).

[0108] In another embodiment, the fork (220) or shuttle (250) may not move, and the preview camera (230) or optical device (240) may move to a predetermined position to acquire a preview image or a slide image. For example, after the position of the fork (220) or shuttle (250) is fixed at a predetermined position, the preview camera (230) or optical device (240) may be mounted on a movable arm and moved to a position to acquire a preview image or a slide scan image.

[0109] In one embodiment, the slide scanning device (100) may include an illumination unit (280) for providing illumination to the slide tray (260) and / or the slides. The illumination unit (280) may provide illumination from the top or bottom of the slide tray (260). In another embodiment, the illumination unit may provide illumination to both the top and bottom of the slide tray (260).

[0110] In one embodiment, the acquired preview image or scan image may be output to the display (120) of FIG. 1.

[0111] FIG. 3 is an exemplary diagram of a slide scanning device for obtaining a preview image according to one embodiment of the present disclosure.

[0112] Referring to FIG. 3, when the slide tray is positioned at an obtainable position (300) of the preview image, the preview camera (230) can obtain a preview image of the slide tray.

[0113] In one embodiment, when the slide scanning device (100) obtains a scan request for the first slide tray, the slide scanning device (100) may use the fork (220) to transfer the first slide tray accommodated in the slide tray slot (210) to an obtainable position (300) of the preview image, and may use the preview camera (230) to obtain a preview image of the transferred first slide tray. In addition, when acquisition of the preview image is completed, the slide scanning device (100) may control the fork (220) to again accommodate the first slide tray in the slide tray slot (210), or control the fork (220) to transfer the first slide tray to an obtainable position of the scan image so as to immediately obtain the scan image.

[0114] FIG. 4 is a drawing illustrating a method of extracting or inserting a slide tray from a slide tray slot according to one embodiment of the present disclosure.

[0115] Referring to FIG. 4, a method of extracting or inserting a slide tray (260) from or into a slide tray receiving portion of a slide tray slot (210) is illustrated. In one embodiment, the slide scanning device (100) may extract the slide tray (260) from the slide tray slot (210) using a fork (220) when extracting the slide tray from the slide tray slot to obtain a preview image, to obtain a scan image, or for other reasons.

[0116] In one embodiment, the slide tray slot (210) may include a plurality of slide tray receiving portions. For example, the slide tray slot (210) may include 20 slide tray receiving portions capable of receiving slide trays. When the slide scanning device (100) obtains an extraction request for a first slide tray (e.g., a preview image acquisition request), the device may identify which slide tray receiving portion the first slide tray is received in, and may extract the slide tray received in the corresponding slide tray receiving portion using the fork (220).

[0117] Likewise, when the slide scanning device (100) receives a storage request (e.g., a scan completion signal) for the first slide tray, it can identify a slide tray receiving compartment in which the first slide tray is to be stored and control the fork (220) to insert the first slide tray into the corresponding slide tray receiving compartment.

[0118] FIG. 5 is a drawing showing a state in which a slide tray according to one embodiment of the present disclosure is transported by a shuttle.

[0119] Referring to FIG. 5, the slide scanning device (100) can place the slide tray on the shuttle (250) using the fork (220) to obtain a scan image of the slide tray. In one embodiment, the fork (220) can move in the z-axis by the fork support (270) to transport the slide tray. Therefore, when the slide scanning device (100) receives a request to obtain a scan image of the slides, the fork (220) can place the slide tray on the shuttle (250). For example, the lowest point to which the fork (220) can move may be the height at which the shuttle exists. After the slide tray is placed on the shuttle (250), in order to obtain a scan image of the slides, the slides must be moved to a position corresponding to the optical device (240) and the lighting unit (280). Therefore, after the slide tray is placed on the shuttle (250), the shuttle (250), not the fork (220), can move on the xy plane. In this regard, more detailed explanation will be given later with reference to Fig. 6.

[0120] FIG. 6 is a diagram illustrating a method for obtaining a scan image according to one embodiment of the present disclosure.

[0121] Referring to FIG. 6, the slide scanning device (100) can obtain a scan image of the slide tray by controlling the shuttle (250) to move in the xy plane after the slide tray is fixed to the shuttle (250) by the fork (220). In one embodiment, the scan image can be obtained after an automatic focusing process.

[0122] Autofocus methods for automatically focusing an image can be broadly divided into active and passive systems. Active systems use an external measuring device, where the camera measures the distance to the subject and focuses based on this information. These systems determine focus through direct distance measurement, typically using laser, infrared, or ultrasonic sensors. The camera uses a physical measuring device to accurately measure the distance to the subject and adjusts the lens' focus based on this information. Typical examples include infrared (IR)-based systems and Laser AutoFocus (LAF). While this method offers relatively fast response times, it suffers from relatively low precision. In particular, for thin, transparent samples, ambiguity in the focus position can occur at a specific magnification objective, further reducing precision. This is because light reflected from the top and bottom of the transparent object enters the receiver together, resulting in similar focus points by the condenser lens.

[0123] In contrast, the passive method automatically focuses by analyzing the image data itself without using an external measuring device (such as a laser or ultrasonic sensor). The objective lens, which is a certain distance away from the sample, moves a certain distance based on its current position while simultaneously obtaining images of the sample using the image sensor. The image sharpness is calculated from these multiple images to determine the image sharpness according to the moving position. The position with the greatest image sharpness becomes the focus position of the sample. When the observation object moves from the current position to the position with the greatest image sharpness, the object is in focus. This method can precisely find the focus position, but it is slow to find the focus because it obtains multiple images while moving a certain distance and analyzes them to find the focus position. A representative passive method is the Image AutoFocus (IAF) method.

[0124] Typically, the LAF method can quickly detect a reference focus position (RFP). However, the initially detected RFP may not be the optimal focus position due to the thickness or flatness of the sample. Accordingly, the slide scanning device (100) according to one embodiment of the present disclosure may adopt a method of determining the RFP through the initial focus area (IAF) and then performing LAF based on this. This will be described in more detail below with reference to FIGS. 7A and 7B .

[0125] In one embodiment, a more precise and faster scanning procedure can be performed by searching for an optimal focal length before scanning a Region Of Interest (ROI) by taking into account the flatness of the apparatus (e.g., a conveyor, a tray, etc.), vibration of the apparatus, thickness variation of the sample within the slide, etc.

[0126] FIG. 7a is a diagram for explaining a method for determining a reference focus position (RFP) according to one embodiment of the present disclosure, and FIG. 7b is an example graph of peak information according to one embodiment of the present disclosure.

[0127] Referring to FIG. 7A, a slide (750) may include a slide glass (760) which is a transparent lower plate having a thickness of about 1 T (Thickness) = 1 mm, a sample placed on the slide glass (760), and a cover glass (770) having a thickness of about 0.1 T to 0.2 T covering the sample. In one embodiment, the slide scanning device (100) may determine a reference focus position (RFP) for performing LAF through IAF before scanning a region of interest (ROI) of the slide. For example, the z-axis position of the objective lens where the sharpest image is acquired may be confirmed by performing IAF, and then determined as the RFP. At this time, since the objective lens moves together with the LAF module by one motor (i.e., moved together as a set of focus transfer parts; this will be described in detail in FIG. 8), the RFP can be searched by measuring the peak level of the laser by performing LAF at a point where a sharp image is acquired through IAF. For example, the slide scanning device (100) can detect the RFP by measuring the peak level of the laser reflected from the upper surface (772) of the cover glass (770), the lower surface (774) of the cover glass (770), the upper surface (762) of the slide glass (760), and the lower surface (764) of the slide glass (760) and checking the z value of the LAF module at that time. In one embodiment, the slide scanning device (100) can detect the RFP by measuring the peak level of the laser reflected from the lower surface (774) of the cover glass (770), which is the central boundary surface where the sample is located, or the upper surface (762) of the slide glass (760). For example, the slide scanning device (100) can measure the peak level of the laser reflected from the lower surface (774) of the cover glass (770) and determine the z value at that time as the RFP.

[0128] In a more specific embodiment, the slide scanning device (100) can perform IAF at the first point (710) to determine the RFP. For example, the slide scanning device (100) can acquire multiple images by adjusting the height of the objective lens at the first point (710) and then determine the focus position for acquiring the clearest image. For example, the slide scanning device (100) can determine the first focus (740) as the focus for acquiring the clearest image at the first point (710), and the position of the objective lens or LAF module (hereinafter collectively referred to as “focus transfer unit set”) at this time can be determined as the reference focus position (RFP). In one embodiment, the reference focus position (RFP) may be a distance between the focus transport set and the sample for obtaining the first focus (740), or alternatively, a distance between the focus transport set and the upper surface of the cover glass, a distance between the focus transport set and the lower surface of the cover glass, a distance between the focus transport set and the upper surface of the slide glass, or a distance between the focus transport set and the lower surface of the slide glass. In one embodiment, the reference focus position (RFP) may be a distance between the focus transport set and the upper surface of the cover glass or a distance between the focus transport set and the lower surface of the slide glass.

[0129] In one embodiment, the slide scanning device (100) may acquire the sharpest image for the first point (710) at the first focus (740) (i.e., the image obtained by performing IAF), and determine the z-position for the laser peak obtained through the laser emitting unit and the light receiving unit at the first focus (740) as the RFP. For example, according to FIG. 7b, if the z-position for the laser peak of the LAF for the first point obtained in the above example is 1350, this is determined as the RFP.

[0130] In one embodiment, when the RFP is determined at the first point (710), the slide scanning device (100), more specifically, the optical device (240), may move to the second point (720) to acquire an image in another region of the ROI and perform LAF based on the RFP. In one embodiment, the optical device (240) may move along a reference focal plane (730) determined under the assumption that the RFP determined according to the first focus (740) is an optimal focus position in the entire region of the ROI. The optical device (240) may move along the reference focal plane (730) to the second point (720) and then perform LAF, thereby adjusting the z-position of the focus transfer unit set to match the RFP, and then acquire an image. By performing LAF for all regions of the ROI in the above manner, a high-quality scan image may be acquired for the ROI.

[0131] In another embodiment, the optical device (240) may be fixed, with only the objective lens moving in the z-axis, and the slide tray may be moved by the shuttle (250) to allow the optical device (240) to perform LAF at a second point and scan. Similarly, the movement of the slide tray may proceed along the reference focal plane (730).

[0132] In the embodiment according to FIG. 7b, when the RFP is determined to be the z-position 1350 of the laser peak, if the position of the focus transfer part set deviates from RFP 1350 when performing LAF at the second point, it is adjusted to the RFP position, and by repeating this method at each point in the remaining area of ​​the ROI, a faster and clearer image can be obtained.

[0133] In one embodiment, the slide scanning device (100) may determine the center point of the ROI area or the center of gravity of the sample to determine the first point at which a passive autofocus procedure is to be performed. If a sample is present at the center point of the ROI area or the center of gravity of the sample, the slide scanning device (100) may determine that point as the first point and perform the passive autofocus procedure at the first point.

[0134] For example, the slide scanning device (100), for example, the optical device (240), can identify the centroid of the sample in the ROI, and if it determines that the sample is present at the centroid, it can determine the centroid point as the first point to perform IAF. However, if the sample does not exist at the point, evaluation of the focus position is impossible, and thus passive automatic focusing, for example, IAF, cannot be performed. Accordingly, if the sample does not exist in the centroid or the central area of ​​the ROI, or if the detected sample presence area is partial, the slide scanning device (100), for example, the optical device (240), can search for the location coordinates of the closest sample from the center coordinates or the center coordinates of the ROI. For example, the slide scanning device (100) can identify a white pixel area in a binarized image as a location where the sample exists. In one embodiment, the slide scanning device (100) can convert the location coordinates where the sample exists from image reference coordinates to feed-system reference coordinates and perform a passive autofocus procedure, such as IAF, at those coordinates.

[0135] That is, in one embodiment, the slide scanning device (100), for example, the optical device (240), can identify the center of gravity of the sample, determine whether the sample exists at the center of gravity, and if the sample exists, determine that point as the first point, and if the sample does not exist, determine the point where the sample exists closest to the center of gravity as the first point.

[0136] In another embodiment, the slide scanning device (100), such as the optical device (240), can identify the coordinates of the center point of the ROI, determine whether a sample exists at the center point of the ROI, and if the sample exists, determine that point as the first point, and if the sample does not exist, determine the point where the sample exists closest to the center point of the ROI as the first point.

[0137] FIG. 8 is a diagram illustrating a camera module performing an autofocus procedure according to one embodiment of the present disclosure.

[0138] Referring to FIG. 8, the optical device (240) may include a scan camera (810), a barrel (820), a z-axis moving unit (830), an objective lens (840), etc. In one embodiment, the z-axis moving unit (830) includes a motor and can move the objective lens (840) in the z-axis direction using the motor. At this time, the scan camera (810) and the barrel (820) may be fixed without moving in the z-axis direction. In addition, LAF modules (852, 854) may be provided on both sides of the objective lens (840). For example, an LAF laser light-emitting unit (852) may be provided on the left side of the objective lens (840), so that when the LAF laser light-emitting unit (852) emits light, the light is reflected from a sample, a slide glass, a cover glass, etc., and the reflected light can be detected by the LAF light-receiving unit (854). Accordingly, peak information such as Fig. 7b can be obtained.

[0139] In one embodiment, the LAF module (842, 854) may be moved by a motor together with the objective lens (840). Accordingly, the objective lens (840) and the LAF module (852, 854) may be referred to as a focus transfer unit set (850). In addition, a control unit (856) that controls the motor and the focus transfer unit set may be provided next to the objective lens.

[0140] FIG. 9 is a flowchart of a method for scanning a slide by a slide scanning device according to one embodiment of the present disclosure.

[0141] Referring to FIG. 9, in operation 910, the slide scanning device may obtain a scan request for the first slide tray. In one embodiment, the slide scanning device (100) may obtain the scan request for the first slide tray based on a user input touching the display (120). For example, the user input touching the display (120) may include a user input requesting to start scanning for the first slide tray, a user input requesting to obtain a preview image for the first slide tray, a user input requesting to start scanning for all slide trays placed in slide tray slots, including the first slide tray, etc.

[0142] In operation 930, the slide scanning device may use a fork to move a first slide tray accommodated in a slide tray slot to an obtainable position of a preview image. In one embodiment, the fork may be moved in the z-axis by a fork support (270), and the fork support (270) may be moved in the x-axis. Accordingly, the fork (220) may be moved in the x-axis and the z-axis. In one embodiment, the fork support (270) may move in the x-axis so that the fork (220) may take out the first slide tray accommodated in the slide tray slot, and then the fork (220) may move in the z-axis so as to move the first slide tray to an obtainable position of a preview image.

[0143] In operation 950, the slide scanning device may obtain a preview image of the first slide tray using a preview camera. In one embodiment, the slide scanning device (100) may obtain a preview image of the first slide tray and recognize information about the slide tray, the slide, or the pathology sample from the preview image. The information recognized from the preview image may include identification information about the first slide tray (e.g., a number, a barcode, or a QR code), direction indication information about the slide tray, identification information about each slide receiving compartment, information about the slide, information about the pathology sample, etc. The information about the slide may include whether a slide is present in each slide receiving compartment, whether the slide is damaged in each slide receiving compartment, whether the slide is received in the correct position in each slide receiving compartment, or information about an area of ​​interest in which a pathology sample is present within the slide. In addition, the information about the pathology sample may include the specimen site of the pathology sample, the time of specimen collection, whether staining was performed, the name of the staining reagent, patient information corresponding to the pathology sample, etc.

[0144] In one embodiment, the slide scan device (100) can control the display (120) so that the acquired preview image, information acquired from the preview image, etc. are output to the display (120).

[0145] In operation 970, the slide scanning device may use a fork to move the first slide tray to an obtainable position for a scan image. In one embodiment, the fork (220) may move the first slide tray so that the first slide tray is placed on the shuttle (250).

[0146] In operation 990, the slide scanning device may acquire scan images of one or more slides placed on a first slide tray using an optical device. In one embodiment, the optical device (240) may be synchronized with a shuttle (250) to acquire scan images of one or more slides while moving the objective lens in the z-axis and moving the shuttle (250) in the xy-plane.

[0147] FIG. 10 is a block diagram of a slide scanning device according to one embodiment of the present disclosure.

[0148] Referring to FIG. 10, the slide scanning device (100) may include a transceiver (1010), an input device (1020), a memory (1030), and a processor (1040). However, not all of the components illustrated in FIG. 10 are essential components of the slide scanning device (100). The slide scanning device (100) may be implemented with more components than the components illustrated in FIG. 10, or may be implemented with fewer components than the components illustrated in FIG. 10. In addition, the transceiver (1010), the input device (1020), the memory (1030), and the processor (1040) may be implemented in the form of a single chip.

[0149] In one embodiment, the transceiver (1010) can communicate with a slide tray slot, user terminal, or other electronic device connected wired or wirelessly to the slide scanning device (100).

[0150] In one embodiment, the input device (1020) may include a touch display, a preview camera, a scan camera, an image acquisition device, and the like.

[0151] The memory (1030) may be installed and stored with various types of data, such as programs and files, such as applications. The processor (1040) may access and use data stored in the memory (1030), or store new data in the memory (1030). In one embodiment, the memory (1030) may include a database. For example, the memory or database may normally store preview images of a slide tray or scanned images of an ROI.

[0152] The processor (1040) controls the overall operation of the slide scanning device (100) and may include at least one processor, such as a CPU or GPU. The processor (1040) may control other components included in the slide scanning device (100) to perform operations for operating the slide scanning device (100). For example, the processor (1040) may execute a program stored in the memory (1030), read a stored file, or store a new file. In one embodiment, the processor (1040) may obtain a scan request for a first slide tray by executing a program stored in the memory (1030), identify a slide tray receiving portion in which the first slide tray is received, control a fork to transfer the first slide tray received in the slide tray slot to an obtainable position of a preview image, control a preview camera to obtain a preview image of the first slide tray, control the fork to transfer the first slide tray to an obtainable position of a scan image, and obtain scan images for one or more slides placed on the first slide tray while moving the shuttle. In one embodiment, the processor (1040) may extract a region of interest (ROI) of the first slide by executing a program stored in the memory (1030), perform an Image Autofocusing (IAF) at a first point of the ROI to determine a Reference Focus Position (RFP) value, and perform a Laser Autofocusing (LAF) based on the RFP to scan the ROI of the first slide.

[0153] An embodiment of the present disclosure may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically contains computer-readable instructions, data structures, or program modules, and includes any information delivery media.

[0154] FIG. 11 is an exemplary diagram of a slide scanning device according to one embodiment of the present disclosure.

[0155] Referring to FIG. 11, the slide scanning device (2000) can obtain a preview image for a slide tray, recognize information about a slide tray (2400), a slide (2500), or a pathology sample from the preview image, obtain a slide scan image based thereon, and store the slide scan image in association with information about the slide tray, the slide, or the pathology sample.

[0156] In one embodiment, the slide scanning device (2000) may include a preview camera (2100), a scan camera (2200), and a slide tray stage (2300). For example, when a slide tray (2400) is placed on the slide tray stage (2300), preview images of a plurality of slides may be acquired using the preview camera (2100), and images of pathology samples accommodated in each slide (2500) may be acquired using the scan camera (2200).

[0157] In one embodiment, the slide tray stage (2300) can move the slide tray (2400) to a position where a preview image can be acquired by the preview camera (2100), and after acquiring the preview image, the slide tray stage (2300) can move the slide tray (2400) back to a position where an image of the pathology sample can be acquired using the scan camera (2200). In another embodiment, the first slide tray stage (2300) can move the slide tray (2400) to a position where a preview image can be acquired by the preview camera (2100), and after acquiring the preview image, a second slide tray stage (not shown, separate from the first slide tray stage (2300)) can move the slide tray (2400) to a position where an image of the pathology sample can be acquired using the scan camera (2200).

[0158] In one embodiment, the preview camera (2100) can obtain preview images of a plurality of slides placed on the slide tray stage (2300). For example, the preview camera (2100) can obtain an entire image of the slide tray (2400) as a preview image. As another example, the preview camera (2100) can obtain images of some of the slides placed on the slide tray (2400) as preview images.

[0159] In one embodiment, the preview camera (2100) may be positioned a predetermined distance from the slide tray stage (2300) to obtain a preview image of a slide tray (2400) containing a plurality of slides. The preview camera (2100) may have a resolution suitable for obtaining information desired from the preview image. For example, the preview camera (2100) may be configured as a camera with a lower resolution than the scan camera (2200).

[0160] In one embodiment, the preview camera (2100) may include one or more lens(es), an image sensor, and a focus control unit. The preview camera (2100) may use the focus control unit to adjust the focus on the slide tray, and may acquire an image passing through one or more lens(es) using the image sensor.

[0161] In one embodiment, the scan camera (2200) can acquire scan images of pathology samples included in each slide (2500). The scan camera (2200) can divide a slide (2500) into multiple regions, acquire scan images for each region, and then combine them to acquire an image for the entire slide (2500). For example, the scan camera (2200) can divide the slide (2500) into a square tile shape and acquire scan images for each region. As another example, the scan camera (2200) can divide the slide region into multiple lines, acquire scan images using a line scan method, and then combine them. As another example, the scan camera (2200) can acquire an image for the entire slide at once. However, the method by which the scan camera (2200) acquires an image of the slide (2500) is not limited to the above embodiments.

[0162] In one embodiment, the scan camera (2200) may include one or more lens(es), an image sensor, and a focus control unit. The scan camera (2200) may use the focus control unit to adjust the focus for each area of ​​the slide to be scanned, and may acquire an image passing through one or more lens(es) using the image sensor. In one embodiment, the scan camera (2200) may perform an auto-focus process using both an active auto-focus method and a passive auto-focus method, and then scan a specific area of ​​the slide. This will be described in more detail below with reference to FIGS. 7A to 7B and FIG. 16.

[0163] In one embodiment, the slide tray stage (2300) can accommodate a slide tray (2400) on its upper surface. In one embodiment, the slide tray stage (2300) can be provided with a fixing means (not shown) capable of fixing the slide tray (2400) on its upper surface. For example, the slide tray stage (2300) can be provided with a means for fixing all or part of the slide tray (2400) in a vertical or horizontal direction.

[0164] In one embodiment, the slide tray stage (2300) can move the slide tray (2400) to a position where the preview camera (2100) or the scan camera (2200) can acquire an image. For example, the slide tray stage (2300) can move the slide tray (2400) to a predetermined position where the preview camera (2100) can acquire an entire image of the slide tray (2400). Additionally, the slide tray stage (2300) can move the slide tray (2400) to a predetermined position so that the scan camera (2200) can scan an area of ​​the slide (2500). Additionally, after the scan camera (2200) completes scanning an area, the slide tray (2400) can be moved so that it can scan a next area.

[0165] In another embodiment, the first slide tray stage (2300) can move the slide tray (2400) to a position where the preview camera (2100) or the scan camera (2200) can acquire an image. For example, the first slide tray stage (2300) can move the slide tray (2400) to a predetermined position where the preview camera (2100) can acquire an entire image of the slide tray (2400). Thereafter, the second slide tray stage (not shown, separate from the first slide tray stage (2300)) can move the slide tray (2400) to a predetermined position so that the scan camera (2200) can scan an area of ​​the slide (2500). Additionally, the second slide tray stage (not shown) can move the slide tray (2400) so that the scan camera (2200) can scan a next area after it has completed scanning an area.

[0166] In one embodiment, when the scan camera (2200) uses a line scan method, the slide tray stage (2300) can move the slide tray (2400) for line scanning. For example, the slide tray (2400) can be moved according to the line scan speed of the scan camera (2200).

[0167] In another embodiment, the slide tray stage (2300) may not move, and the preview camera (2100) or the scan camera (2200) may move to a predetermined position to acquire a preview image or a slide image. For example, the slide tray stage (2300) may be fixed in position, and the preview camera (2100) or the scan camera (2200) may be mounted on a movable arm and moved to a position to acquire a preview image or a slide scan image.

[0168] In one embodiment, the slide scanning device (2000) may further include an illumination unit (2600) for providing illumination to the slide tray (2400) and / or the slide (2500). For example, as in the example of FIG. 11, the illumination unit (2600) may provide illumination from the upper surface of the slide tray (2400). In another example, the illumination unit (2600) may provide illumination from the lower surface of the slide tray (2400). In yet another example, the illumination unit (2600) may provide illumination to both the upper and lower surfaces of the slide tray (2400).

[0169] FIG. 12 is a drawing illustrating a slide tray according to one embodiment of the present disclosure.

[0170] Referring to FIG. 12, the slide scanning device (2000) can recognize information about the slide tray (2400) from a preview image acquired through the preview camera (2100).

[0171] In one embodiment, the slide tray (2400) may be configured with a plurality of slide receiving portions (2450) each capable of receiving a slide, and may include identification information (2410, 2420) for the slide tray (2400), direction indication information (2430, 2440) for the slide tray (2400), and identification information (2460) for each slide receiving portion. For example, the identification information (2410, 2420) for the slide tray (2400) may be printed in letters or numbers, or may be included in the form of a barcode or QR code. If the identification information (2410, 2420) for the slide tray (2400) is printed in letters or numbers, it may be identified from a preview image using optical character recognition (OCR) technology, and if it is in the form of a barcode or QR code, it may be recognized to obtain the identification information for the corresponding slide tray (2400). The slide scanning device (2000) can recognize identification information (2410, 2420) for the slide tray (2400) to accurately identify the slide tray (2400) currently being scanned.

[0172] In one embodiment, the slide scanning device (2000) can identify whether the slide tray (2400) is positioned on the slide tray stage (2300) from the preview image. For example, if a mark is placed at a predetermined position on the slide tray stage (2300) and the mark is placed at a position where the mark is obscured when the slide tray (2400) is placed on the slide tray stage (2300), the presence of the slide tray (2400) can be simply determined based on whether the mark is recognized in the preview image. Furthermore, if the mark is placed so that the mark on the slide tray stage (2300) is obscured only when the slide tray (2400) is correctly placed, it can also be determined whether the slide tray (2400) is positioned correctly based on whether the mark is recognized in the preview image. As another example, the slide scanning device (2000) can determine the presence of a slide tray (2400) by recognizing identification information (2410, 2420) for the slide tray (2400) or direction indication information (2430, 2440) of the slide tray (2400) in the preview image.

[0173] In one embodiment, the direction indication information (2430, 2440) of the slide tray (2400) may include an indication for the up-down or left-right direction, and may be displayed as an arrow for the corresponding direction, or may be displayed with a predetermined identifier on one side of a specific direction. For example, referring to FIG. 12, the direction indication information (2430) of the slide tray (2400) may be displayed by an arrow expressed in the up-down direction of the slide tray (2400). As another example, the direction indication information (440) for the up-down and left-right directions may be displayed by displaying a predetermined identifier (e.g., a black dot in FIG. 12) on the upper left corner of the slide tray (2400). The slide scanning device (2000) may recognize the direction indication information (2430, 2440) for the slide tray (2400) by recognizing the direction of the arrow in the preview image, or by recognizing a specific identifier at a predetermined location. In one embodiment, the slide scanning device (2000) can use the direction indication information (2430, 2440) for the slide tray (2400) to recognize the direction in which the slide tray (2400) is placed on the slide tray stage (2300).

[0174] In one embodiment, the identification information (2460) for each slide receiving portion (2450) of the slide tray (2400) may be represented by a number, the number of dots, the length of a line, etc. For example, the identification information (2460) for each slide receiving portion (2450) may be represented by a number, and the slide scanning device (2000) may recognize the identification information (2460) for each slide receiving portion (2450) from a preview image using OCR technology. As another example, if the identification information (2460) for each slide receiving portion (2450) is expressed as a predetermined symbol, such as the number of dots, the length of a line, etc., the identification information for each slide receiving portion (2450) may be recognized by recognizing the symbol from the preview image.

[0175] In one embodiment, the slide scanning device (2000) can recognize the corresponding slide receiving portion (2450) using identification information (2460) when a specific slide (2500) received in the slide tray (2400) in the preview image is damaged or has come out of the slide receiving portion (2450). For example, when the fourth slide from the left in the upper slide row is damaged, the slide scanning device (2000) can recognize the identification information (2460) of the corresponding slide receiving portion (2450) and recognize that the slide at position 4 is damaged.

[0176] FIG. 13 is a drawing illustrating a slide according to one embodiment of the present disclosure.

[0177] Referring to FIG. 13, the slide scanning device (2000) can recognize information about each slide (2500) accommodated in the slide tray (2400) from a preview image acquired through a preview camera (2100). In one embodiment, the slide (2500) may include an information display unit (2510) that displays information about the slide (2500) and a sample receiving unit (2550) that accommodates a pathology sample. The information display unit (2510) of the slide (2500) may display information about the slide (2500) using letters, numbers, symbols, barcodes, or QR codes. For example, when the information is displayed using letters and / or numbers, the information can be recognized using OCR technology, when it is expressed using a predetermined symbol, the information can be recognized using camera vision technology, and when it is displayed using a QR code or barcode, the information can be recognized by decoding it.

[0178] In one embodiment, the information description section (2510) of the slide (2500) may include information about a patient corresponding to the slide (2500), information about a pathology sample, information about a scanning method, and the like. For example, information about a patient corresponding to the slide (2500) may include patient name, patient age, patient's resident registration number, patient ID (or patient number), department, disease name, and the like. Information about a pathology sample may include the specimen site of the sample (e.g., liver tissue), whether it is stained, the name of the staining reagent, the time of sample collection, and the like. In addition, information about the scanning method may include information about the priority of the sample (e.g., whether it is an urgent test), scan resolution, scan area, scan method, and the like.

[0179] In one embodiment, the slide scanning device (2000) can recognize the location of the pathology sample on the slide (2500) from the preview image to determine in advance the region of interest (ROI) to be scanned by the scan camera (2200). For example, the slide scanning device (2000) can recognize the location of the pathology sample in the sample receiving portion (2550) of the slide (2500), store it, and then cause the scan camera (2200) to scan the region of interest to acquire an image of the pathology sample. For example, the region of interest can be set as a rectangle containing the pathology sample, as in the example of FIG. 13, or can be set along the outline of the pathology sample, depending on the scanning method.

[0180] FIG. 14 is a drawing illustrating a slide scanning device according to one embodiment of the present disclosure.

[0181] Referring to FIG. 14, the slide scanning device (2000) may include a preview camera (2100), a scan camera (2200), a slide tray stage (2300), a lighting unit (2600), a processor (2710), a display unit (2720), an input unit (2730), a communication unit (2740), and a storage unit (2750). However, not all of the components illustrated in FIG. 14 are essential components of the slide scanning device (2000).

[0182] In one embodiment, the processor (2710) may be configured as a general-purpose processor or a dedicated processor. The processor (2710) may read and execute program modules stored in the storage (2750). For example, the processor (2710) may control the preview camera (2100), the scan camera (2200), the slide tray stage (2300), the lighting unit (2600), etc., by executing the program modules stored in the storage (2750). In addition, the processor (2710) may output the current status of the slide scanning device (2000), the progress status of the slide scan, and / or the image of a slide that has been scanned to the display unit (2720) by executing the program modules stored in the storage (2750). In addition, the processor (2710) may transmit and receive information with an external device via the communication unit (2740) by executing the program modules stored in the storage (2750). Additionally, the processor (2710) can receive user input from the input unit (2730) by executing a program module stored in the storage unit (2750). In one embodiment, the input unit (2730) can be configured in the form of a touch screen on the display unit (2720).

[0183] In one embodiment, the storage unit (2750) stores program modules, such as an image recognition module (2810), a scanner control module (2820), and an image storage module (2830), which can be executed via a processor. For example, the storage unit (2750) can be configured as a memory, an SSD, an HDD, or a combination thereof.

[0184] In one embodiment, the image recognition module (2810) can recognize information about the slide tray (2400), the slide (2500), and / or the pathology sample from the preview image acquired through the preview camera (2100). For example, the image recognition module (2810) can distinguish a slide tray area, a slide area, an information description section (2510) and a sample receiving section (2550) of the slide area, extract information about the slide tray (2400) from the slide tray area, extract information about the slide from each slide area, and recognize information about the corresponding pathology sample from the information description section (2510) and the sample receiving section (2550) of each slide area.

[0185] In one embodiment, the image recognition module (2810) can recognize information about a slide tray (2400) from a preview image. Here, the information about the slide tray (2400) can include whether the slide tray (2400) is present on the slide tray stage (2300), identification information about the slide tray (2400), direction information about the slide tray (2400), or identification information (2460) of each slide receiving portion (2450) of the slide tray (2400). For example, whether the slide tray (2400) is present on the slide tray stage (2300) can be determined by recognizing a mark located on the upper surface of the slide tray stage (2300) (which is obscured when the slide tray (2400) is placed in the correct position). For example, identification information for the slide tray (2400), direction information of the slide tray (2400), or identification information (2460) of each slide receiving portion (2450) of the slide tray (2400) can be recognized by letters, numbers, symbols, barcodes, or QR codes displayed on the upper surface of the slide tray (2400).

[0186] In one embodiment, the image recognition module (2810) can recognize information about a slide (2500) in a preview image. Here, the information about the slide (2500) can include information about whether a slide (2500) exists in each slide receiving portion (2450), whether the slide (2500) in each slide receiving portion (2450) is damaged, whether the slide (2500) is received in the correct position in each slide receiving portion (2450), or information about a region of interest in which a pathological sample exists in the slide (2500). For example, the presence, damage, and correct position of the slide (2500) in each slide receiving portion (2450) can be obtained through analysis of the preview image, and for the convenience of image analysis, specific marks can be added to predetermined positions of each slide receiving portion and / or the slide. For example, information on a region of interest in which a pathology sample exists within a slide (2500) can be set as a region of interest including the outline of the pathology sample by recognizing the outline of the pathology sample in the sample receiving portion (2550) of each slide (2500).

[0187] In one embodiment, the image recognition module (2810) can recognize information about a pathology sample from a preview image. Here, the information about the pathology sample may include the specimen site of the pathology sample, the time of specimen collection, whether or not it was stained, the name of the staining reagent, or patient information corresponding to the pathology sample. In addition, the patient information corresponding to the pathology sample may include the patient's name, patient's age, the patient's resident registration number, the patient's number, the patient's department, or the patient's disease name. In one embodiment, the information about the pathology sample may include information about the priority of the pathology sample, scan resolution, scan area, or scan method. For example, the information about the pathology tissue (or pathology sample) may be obtained by recognizing letters, numbers, symbols, barcodes, or QR codes described in the information description section (2510) of each slide (2500).

[0188] In one embodiment, the scanner control module (2820) controls the preview camera (2100) to acquire a preview image, and uses information about the slide tray (2400), the slide (2500), and / or the pathology sample recognized by the image recognition module (2810) to control the slide scanning device (2000) to acquire a scan image of the pathology sample. For example, the scanner control module (2820) may use information about the slide tray (2400), the slide (2500), and / or the pathology sample to control the scan camera (2200), the slide tray stage (2300), and the illumination unit (2600) to acquire a scan image.

[0189] In one embodiment, the scanner control module (2820) may determine the lens magnification, scan resolution, scan area size, scan method, illumination intensity of the illumination unit (2600), illumination method, etc. of the scan camera (2200) by considering the disease name of the patient corresponding to the pathology sample, the body tissue name of the pathology sample, whether or not to stain, the name of the staining reagent, the time of specimen collection, etc., and control the slide scanning device (2000) based on the determination. In another embodiment, the scanner control module (2820) may control the slide scanning device (2000) based on information about the scan method recognized by the image recognition module (2810) in the information description unit (2510) of the slide (2500) (e.g., priority, scan resolution, scan area, scan method, etc.).

[0190] In one embodiment, the scanner control module (2820) may provide the user with a preview image along with information about the slide tray (2400), slides (2500), and / or pathology samples, and obtain user input regarding slide scanning choices from the user. In this case, the scanner control module (2820) may control the slide scanning based on the user's selections. For example, the user may choose to scan only some of the slides contained in the slide tray. As another example, the user may choose to scan specific slides contained in the slide tray first. As another example, the user may select a scanning method for specific slides.

[0191] In one embodiment, the image storage module (2830) may store a scanned image obtained by the scanner control module (2820) in association with information about a pathology sample obtained by the image recognition module (2810). For example, the image storage module (2830) may store the scanned image by including, as macro information, the patient name, the patient's age, the patient's resident registration number, the patient ID (or patient number), the department, the disease name, etc. obtained by the image recognition module (2810). For example, the image storage module (2830) may store the scanned image by including, as macro information, the body tissue name of the sample (e.g., liver tissue), whether it is stained, the name of the staining reagent, the time of specimen collection, etc. obtained by the image recognition module (2810). For example, the image storage module (2830) may store the scanned image by including, as macro information, the priority (e.g., whether urgent examination is required), scan resolution, scan area, scan method, etc. obtained by the image recognition module (2810).

[0192] In one embodiment, the image storage module (2830) may store the scanned image by including the scan resolution, scan area, scan method, etc. used in the process of obtaining the scanned image from the scanner control module (2820) as macro information.

[0193] In one embodiment, the image storage module (2830) may transmit a scanned image obtained by the scanner control module (2820) to an external device using the communication unit (2740) based on information about a pathology sample obtained by the image recognition module (2810). For example, if the pathology sample is a subject of an urgent examination according to information about the pathology sample, the image storage module (2830) may transmit a scanned image of the pathology sample to the external device in association with the information about the pathology sample. As a result, a medical professional who has requested an urgent examination can be provided with an image of the pathology sample in priority.

[0194] FIG. 15 is a diagram illustrating a method for extracting a Region Of Interest (ROI) according to one embodiment of the present disclosure.

[0195] Referring to FIG. 15, the slide scanning device (2000) may perform a process of acquiring a preview image to determine a ROI area where actual scanning will be performed among the slides positioned on the slide tray. In one embodiment, the slide scanning device (2000) may capture an entire image of the slide tray using a preview camera (2100).

[0196] In one embodiment, the slide scanning device (2000) can correct an image obtained using a preview camera (2100) through a perspective transformation and lens distortion correction process according to the camera installation. In addition, the slide scanning device (2000) can extract an image for each slide positioned on the slide tray based on the corrected image. For example, each slide image of the corrected preview image can be cropped.

[0197] In one embodiment, the slide scanning device (2000) can recognize the background portion and the sample (or sample) portion of each extracted slide image to extract the outline and ROI area of ​​the sample (or sample). For example, the slide scanning device (2000) can extract the outline and ROI area of ​​the sample (or sample) by utilizing the color, brightness difference, etc. of the background portion and the sample (or sample) portion of each slide image.

[0198] In one embodiment, the slide scanning device (2000) can convert the image reference coordinates for the extracted ROI area into the transport system reference coordinates. The converted transport system reference coordinates can be used by the slide scanning device (2000) to perform auto-focus and scan. The method by which the slide scanning device (2000) performs auto-focus and scans will be described in more detail below with reference to FIGS. 7A to 7B and FIGS. 16 to 17B.

[0199] FIG. 16 is a flowchart illustrating a method for scanning an ROI according to one embodiment of the present disclosure.

[0200] Referring to FIG. 16, the slide scanning device (2000) can perform steps 610 to 670. However, the fact that the slide scanning device (2000) performs steps 610 to 670 includes that a detailed device included in the slide scanning device (2000) but not specifically mentioned herein performs steps 610 to 670. In FIG. 16, the slide scanning device (2000) is described as performing steps 610 to 670, but it is also understood that another device external to the slide scanning device (2000) can perform steps 610 to 670. Furthermore, although the embodiments of the present disclosure are described as being performed by the slide scanning device (2000), they may be performed by a detailed device included in the slide scanning device (2000) or an external device. For example, steps 610 to 670 may be performed by an auto-focus device included in the slide scanning device (2000), or steps 610 to 670 may be performed by an auto-focus device external to the slide scanning device (2000).

[0201] In step 610, the slide scanning device (2000) can extract the ROI of the first slide. In one embodiment, the method of extracting the ROI of the first slide may refer to the description described above with reference to FIG. 15. For example, the slide scanning device (2000) may acquire an image of a slide tray through a preview camera, perform perspective transformation and lens distortion correction according to the installation of the preview camera on the image of the slide tray, extract individual images of the corrected image, and extract the outline and ROI area of ​​the sample by utilizing the color difference, brightness difference, etc. between the background part and the sample part for the individual images. Accordingly, the ROI of the first slide may also be acquired.

[0202] In step 630, the slide scanning device (2000) may perform a passive autofocus procedure at a first point of the extracted ROI to determine a Reference Focus Position (RFP). For example, the passive autofocus procedure may include an Image Autofocus (IAF) procedure. The IAF procedure is a method of selecting the image with the best focus among multiple images acquired by adjusting the positions of the subject and the lens. Since the IAF method searches for a focal plane based on image data input to an actual camera, it can search for an optimal focal plane more accurately than Laser Autofocusing (LAF), thereby performing a precise autofocus procedure. However, the IAF method takes longer than the LAF method. Accordingly, performing the IAF autofocus procedure at all points may be inefficient.

[0203] In one embodiment, the first point may be a point within the ROI where the sample is present. For example, the slide scanning device (2000) may determine the center point of the ROI or the center of gravity of the sample as the first point. If the sample is not present at the center point of the ROI or the center of gravity of the sample, the slide scanning device may determine another point where the sample is present as the first point. This will be described in more detail later with reference to FIG. 17b.

[0204] In one embodiment, the slide scanning device (2000) can search for a focus position based on image data input to an actual camera, and determine a reference focus position (RFP) corresponding to the searched focus position. Here, the reference focus position (RFP) is generally a focus position that serves as a reference when performing LAF, and is a focus position searched by measuring the intensity level of the laser reflected from the boundary surface or the peak of the laser through the LAF laser module. The reference focus position (RFP) will be described in more detail later with reference to FIGS. 7A and 7B.

[0205] In step 650, the slide scanning device (2000) may perform an active autofocus procedure (e.g., LAF) based on a reference focus position (RFP). In one embodiment, the slide scanning device (2000) may perform LAF using a reference focus position (RFP) determined at a first point. For example, when the RFP is determined at the first point, the slide scanning device (2000) is moved to a second point, and LAF is performed based on the RFP at the second point to acquire an image.

[0206] In step 670, the slide scanning device (2000) can repeatedly perform LAF for each point of the entire ROI of the first slide by repeating step 650, thereby obtaining a scan image for the ROI. According to one embodiment, the slide scanning device (2000) can perform a more precise focus adjustment process than performing only an active auto-focus procedure based on performing a passive auto-focus procedure at the first point, and can perform an active auto-focus procedure based on a predetermined RFP in an area other than the first point or all areas including the first point, thereby enabling a higher speed scan than performing only a passive auto-focus procedure.

[0207] FIG. 17A is a diagram illustrating a method for determining a first point for performing passive auto-focus according to one embodiment of the present disclosure.

[0208] Referring to FIG. 17A, the slide scanning device (2000) can determine the center point of the ROI area or the center of gravity of the sample to determine the first point at which the passive autofocus procedure is to be performed. If a sample exists at the center point of the ROI area or the center of gravity of the sample as shown in FIG. 17A, the slide scanning device (2000) can determine the point as the first point and perform the passive autofocus procedure at the first point.

[0209] For example, in Fig. 17a, a slide scanning device (2000), such as a camera module (2800), can identify the centroid of a sample in a ROI, and if it determines that a sample exists at the centroid, it can determine the centroid point as the first point at which IAF is to be performed. However, if a sample does not exist at the point, focal plane evaluation is impossible, and thus passive automatic focusing, such as IAF, cannot be performed. Accordingly, a method for determining the first point when a sample does not exist at the centroid or the central area of ​​the ROI will be described in detail later in Fig. 17b.

[0210] FIG. 17b is a diagram illustrating a method for determining a first point for performing passive auto-focus according to one embodiment of the present disclosure.

[0211] Referring to FIG. 17b, if the sample does not exist at the center of gravity (Centroid) of the sample or the center of the ROI, or if the detected sample presence area is partial, the coordinates where the actual tissue is located cannot be searched for with simple coordinates such as the center coordinates or the center of gravity coordinates of the ROI. Accordingly, the slide scanning device (2000), for example, the camera module (2800), can search for the location coordinates where the sample exists at the closest location from the center coordinates or the center of gravity coordinates of the ROI. For example, the slide scanning device (2000) can identify a white pixel area in a binarized image as the location where the sample exists. In one embodiment, the slide scanning device (2000) can convert the location coordinates where the sample exists from the image reference coordinates to the feed-system reference coordinates, and perform a passive auto-focus procedure such as IAF at the corresponding coordinates.

[0212] That is, in one embodiment, the slide scanning device (2000), for example, the camera module (2800), can identify the center of gravity of the sample, determine whether the sample exists at the center of gravity, and if the sample exists, determine that point as the first point, and if the sample does not exist, determine the point where the sample exists closest to the center of gravity as the first point.

[0213] In another embodiment, the slide scanning device (2000), such as the camera module (2800), can identify the coordinates of the center point of the ROI, determine whether a sample exists at the center point of the ROI, and if the sample exists, determine that point as the first point, and if the sample does not exist, determine the point where the sample exists closest to the center point of the ROI as the first point.

[0214] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that the present disclosure can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0215] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. A slide tray slot that accommodates one or more slide trays; A fork for carrying the above slide tray; A preview camera for obtaining a preview image for the above slide tray; and An optical device for acquiring scan images of one or more slides placed on the slide tray. A slide scanning device comprising:

2. In the first paragraph, the fork, A slide scanning device characterized in that the slide tray accommodated in the slide tray slot is extracted and transferred to at least one of an obtainable position of the preview image and an obtainable position of the scanned image.

3. In the first paragraph, the scanned image of one or more of the slides is A slide scanning device characterized in that it is acquired after acquiring the above preview image.

4. In paragraph 1, The above slide scanning device, Further comprising a shuttle positioned at a position corresponding to the optical device; The above shuttle, A slide scanning device characterized in that it moves in the xy plane to obtain scan images for one or more of the slides.

5. In paragraph 1, The above optical device, Includes a scan camera, a telescope, a z-axis travel unit, and an objective lens, The above objective lens is, A slide scanning device characterized in that it moves along the z-axis according to an auto-focus algorithm by the z-axis moving unit.

6. In the first paragraph, the slide scanning device, display; memory for storing one or more instructions; and At least one processor for executing one or more instructions stored in the memory, The at least one processor executes the one or more instructions, A slide scanning device that controls the display to output an image of at least one of the preview image and the scanned image of one or more slides.

7. In the 6th paragraph, the display, A slide scanning device characterized in that it is provided on the door of the above slide scanning device.

8. In paragraph 7, the door, A slide scan device characterized in that it is installed in a slidable manner.

9. A slide scanning method of a slide scanning device according to any one of clauses 1 to 8, Step of obtaining a scan request for the first slide tray; A step of transferring the first slide tray, which is accommodated in the slide tray slot, to an obtainable position of the preview image; A step of the above preview camera obtaining a preview image of the first slide tray; The step of the fork moving the first slide tray to an obtainable position of the scan image; and A method comprising the step of obtaining scan images of one or more slides placed on the first slide tray by the optical device.

10. In the 9th paragraph, the step of moving the first slide tray to an obtainable position of the scan image by the fork is as follows: A method comprising the step of transferring the first slide tray to the shuttle by the fork.

11. In the 9th paragraph, the step of obtaining a scan image of one or more slides placed on the first slide tray by the optical device comprises: (e) a step of extracting the ROI (region of interest) of the first slide; (f) a step of performing IAF (Image Autofocusing) at the first point of the ROI to determine a reference focus position (RFP); (g) a step of performing LAF (Laser Autofocusing) at a second point of the ROI based on the determined RFP; and (h) A method comprising the step of obtaining a scan image for the ROI of the first slide by repeating the step (c) for the entire ROI of the first slide.

12. In the slide scanning device, memory; and At least one processor that executes one or more instructions stored in the memory, At least one processor, (a) Extract the ROI (region of interest) of the first slide, (b) Performing IAF (Image Autofocusing) at the first point of the ROI to determine the reference focus position (RFP), (c) Perform LAF (Laser Autofocusing) at the second point of the ROI based on the determined RFP, (d) A slide scanning device that obtains a scan image for the ROI of the first slide by repeating the above step (c) for the entire ROI of the first slide.

13. A program stored on a computer-readable recording medium that causes a computer to execute the method of any one of clauses 9 to 11.

Images