Intelligent quality control for histological images

Abstract

A method for automated intelligent quality control can be provided. Such a method can include applying a quality control process to a plurality of histology images. Such a quality control process can include, for each of a set of artifacts, determining whether a histology image to which the quality control process is applied has the artifact. Such a quality control process can also include, for each artifact from the set of artifacts, based on determining that the histology image to which the quality control process is applied has the artifact, determining whether to generate a flag indicating that the histology image to which the quality control process is applied has the artifact.

Description

Technical Field

[0001] The embodiments described herein generally relate to the generation of histological images, and more specifically, to the automatic identification of artifacts in such images. Background Technology

[0002] The usability of histological images can be reduced by various artifacts, such as out-of-focus images, writing on the slide, and / or incomplete imaging of the tissue on the underlying slide. However, in many cases, the practices used to address these artifacts have significant drawbacks. For example, in some situations, histological images may only be sampled for artifacts, resulting in the vast majority of laboratory-generated images remaining unexamined before actual diagnostic use. Therefore, there is a need for automated quality control techniques that can identify and / or respond to these types of artifacts in all laboratory-generated images. Summary of the Invention

[0003] The disclosed techniques can be implemented in various ways. For example, in some aspects, a method for performing automated quality control on histological images can be provided. Such a method may include applying a quality control process to a plurality of histological images. Such a quality control process may include, for each artifact in a set of artifacts, determining whether the histological image to which the quality control process is applied has the artifact. Such a quality control process may also include, for each artifact from the set of artifacts, determining, based on the determination that the histological image to which the quality control process is applied has the artifact, whether to generate a flag indicating that the histological image to which the quality control process is applied has the artifact. In such a method, for a first image from a plurality of histological images, applying the quality control process to the image may include: determining that the image has a first artifact from the set of artifacts; determining whether to generate a flag indicating that the image has the first artifact; and generating a flag indicating that the first image has the first artifact.

[0004] Other types of implementations are also possible, including in the form of systems and computer-readable media for performing such methods, and those skilled in the art will immediately recognize these implementations in light of this disclosure. Therefore, the exemplary methods provided in this invention should be understood as illustrative only and should not be considered limiting. Attached Figure Description

[0005] By studying the accompanying drawings, one can partially understand the details of the disclosed technology regarding its structure and operation, wherein the same reference numerals refer to the same parts, and in the drawings:

[0006] Figure 1A Examples of processor-supporting devices are shown that can be used in conjunction with the various embodiments described herein;

[0007] Figure 1B An example line scan camera with a single line array is shown according to an embodiment;

[0008] Figure 1C An example line scan camera with three line arrays is shown according to an embodiment;

[0009] Figure 1D An example line scan camera with multiple line arrays is shown according to an embodiment;

[0010] Figure 2 Provides advanced illustrations of methods that can be used to automatically apply quality control processes to histological images;

[0011] Figure 3 An interface that can be provided on the scanner's display is shown;

[0012] Figure 4 A schematic diagram of a laboratory containing multiple scanners is shown;

[0013] Figure 5 The methods that can be executed when providing a dashboard interface are shown; and

[0014] Figure 6 A method that can be used to determine whether an artifact flag has been generated is shown. Detailed Implementation

[0015] This document discloses techniques that can be used to intelligently perform quality control processes on histological images. After reading this specification, those skilled in the art will understand how the disclosed techniques can be implemented in various alternative embodiments and applications. However, although various embodiments of the disclosed techniques are described herein, it should be understood that these embodiments are presented by way of example and illustration only, and not as limitations. Therefore, this detailed description of the various embodiments should not be construed as limiting the scope or breadth of protection provided by this document or any related documents.

[0016] 1. Example Scanning System

[0017] Figure 1AThis is a block diagram illustrating an example slide scanning system 100 with a processor that can be used in conjunction with the various embodiments described herein. As those skilled in the art will understand, alternative forms of the scanning system 100 may also be used. In the illustrated embodiment, the scanning system 100 is presented as a digital imaging apparatus, comprising one or more processors 104, one or more memories 106, one or more motion controllers 108, one or more interface systems 110, one or more movable stages 112 each supporting one or more glass slides 114 on which one or more samples 116 are disposed, one or more illumination systems 118 illuminating the samples 116, and an imaging system 101 comprising imaging optics 103 (e.g., one or more objectives 120 each defining an optical path 122 along an optical axis, one or more objective positioners 124, one or more optional incident illumination systems 126 (e.g., included in a fluorescence scanning embodiment), one or more focusing optics 128), and one or more line scan cameras 130 and / or one or more area scan cameras 132, each camera defining a separate field of view 134 on the samples 116 and / or glass slides 114. The various components of the scanning system 100 are communicatively coupled via one or more communication buses 102. Although there may be multiples of each of the various elements in the scanning system 100, for the sake of simplicity in the following description, these elements will be described in the singular unless it is necessary to describe them in the plural form to convey appropriate information.

[0018] Processor 104 may include, for example, a central processing unit (CPU) and a separate graphics processing unit (GPU) capable of processing instructions in parallel, or a multi-core processor capable of processing instructions in parallel. Additional separate processors may also be provided to control specific components or perform specific functions, such as image processing. For example, additional processors may include auxiliary processors for managing data input, auxiliary processors for performing floating-point mathematical operations, dedicated processors (e.g., digital signal processors) with an architecture suitable for fast execution of signal processing algorithms, slave processors subordinate to the main processor (e.g., back-end processors), and additional processors for controlling the line scan camera 130, stage 112, objective lens 120, and / or display (e.g., a console including a touch panel display integrated with the scanning system 100). Such additional processors may be separate discrete processors or may be integrated into a single processor.

[0019] Memory 106 provides storage for data and instructions for a program executable by processor 104. Memory 106 may include one or more volatile and / or non-volatile computer-readable storage media for storing data and instructions. These media may include, for example, random access memory (RAM), read-only memory (ROM), hard disk drive, removable storage drive (e.g., including flash memory), etc. Processor 104 is configured to execute instructions stored in memory 106 and communicate with various components of scanning system 100 via communication bus 102 to perform the overall functions of scanning system 100.

[0020] The communication bus 102 can be configured to transmit analog electrical signals and / or digital data. Therefore, communication via the communication bus 102 from the processor 104, motion controller 108, and / or interface system 110 can include both electrical signals and digital data. The processor 104, motion controller 108, and / or interface system 110 can also be configured to communicate with one or more of the various components of the scanning system 100 via a wireless communication link.

[0021] The motion control system 108 is configured to precisely control and coordinate the X, Y, and / or Z movements of the stage 112 (e.g., in the XY plane), the X, Y, and / or Z movements of the objective lens 120 (e.g., along the Z-axis orthogonal to the XY plane, via the objective lens positioner 124), the rotational movement of the turntable described elsewhere herein, the lateral movement of the push / pull assembly described elsewhere herein, and / or any other moving parts of the scanning system 100. For example, in a fluorescence scanning embodiment including the incident illumination system 126, the motion control system 108 may be configured to coordinate the movement of filters, etc., in the incident illumination system 126.

[0022] Interface system 110 enables scanning system 100 to interface with other systems and operators. For example, interface system 110 may include a console (e.g., a touch panel display) to provide information directly to the operator via a graphical user interface and / or enable the operator to make direct input via touch sensors. Interface system 110 may also be configured to facilitate communication and data transfer between scanning system 100 and one or more external devices (e.g., printers, removable storage media, etc.) directly connected to scanning system 100 and / or one or more external devices (e.g., image storage systems, scanner manager (SAM) servers and / or other management servers, operator stations, user stations, etc.) indirectly connected to scanning system 100 via one or more networks.

[0023] Illumination system 118 is configured to illuminate at least a portion of sample 116. Illumination system 118 may include, for example, one or more light sources and illumination optics. The light source may include a variable-intensity halogen light source with concave mirrors to maximize light output and a KG-1 filter to suppress heat. The light source may include any type of arc lamp, laser, or other light source. In an embodiment, illumination system 118 illuminates sample 116 in transmission mode, such that line scan camera 130 and / or area scan camera 132 detect the light energy transmitted through sample 116. Alternatively or additionally, illumination system 118 may be configured to illuminate sample 116 in reflection mode, such that line scan camera 130 and / or area scan camera 132 detect the light energy reflected from sample 116. Illumination system 118 may be configured to probe sample 116 in any known optical microscopy mode.

[0024] In an embodiment, the scanning system 100 includes an epi-illumination system 126 to optimize the scanning system 100 for fluorescence scanning. It should be understood that the epi-illumination system 126 can be omitted if the scanning system 100 does not support fluorescence scanning. Fluorescence scanning is the scanning of a sample 116 comprising fluorescent molecules, which are photon-sensitive molecules capable of absorbing light of a specific wavelength (i.e., being excited). These photon-sensitive molecules also emit light at even higher wavelengths (i.e., emitting). Because this photoluminescence phenomenon is very inefficient, the amount of emitted light is typically very low. This low amount of emitted light typically makes conventional techniques for scanning and digitizing the sample 116 (e.g., transmission mode microscopy) difficult to implement.

[0025] Advantageously, in embodiments of the scanning system 100 utilizing fluorescence scanning, a line scan camera 130 comprising multiple linear sensor arrays (e.g., a time-delay integration (TDI) line scan camera) is used. This enhances the photosensitivity of the line scan camera 130 by exposing the same region of the sample 116 to each of the multiple linear sensor arrays of the line scan camera 130. This is particularly useful when scanning weakly fluorescent samples with low emission levels. Therefore, in fluorescence scanning embodiments, the line scan camera 130 is preferably a monochromatic TDI line scan camera. Monochromatic images are ideal in fluorescence microscopy because they more accurately represent the actual signals from the various channels present on the sample 116. As those skilled in the art will understand, fluorescent samples can be labeled with a variety of fluorescent dyes that emit light at different wavelengths, also referred to as “channels.”

[0026] Furthermore, since various fluorescent samples exhibit a wide wavelength spectrum for sensing at both the low and high end of the signal range, it is desirable that the low and high end signal range sensed by the line scan camera 130 is also wide. Therefore, in the fluorescence scanning embodiment, the line scan camera 130 may include a monochrome 10-bit 64-line array TDI line scan camera. It should be noted that in this embodiment, the line scan camera 130 may employ various bit depths.

[0027] The movable stage 112 is configured to perform precise XY movement under the control of the processor 104 or motion controller 108. The movable stage 112 can also be configured to perform Z movement under the control of the processor 104 or motion controller 108. The movable stage 112 is configured to position the sample 116 at a desired location during image data acquisition by the line scan camera 130 and / or the area scan camera 132. The movable stage 112 is also configured to accelerate the sample 116 to a substantially constant speed in the scanning direction and then maintain said substantially constant speed during image data acquisition by the line scan camera 130. In an embodiment, the scanning system 100 may employ a high-precision and tightly coordinated XY grid to assist in the positioning of the sample 116 on the movable stage 112. In an embodiment, the movable stage 112 is a linear motor-based XY stage equipped with high-precision encoders on both the X and Y axes. For example, a very precise nano-encoder can be used on an axis perpendicular to the scanning direction and in the same plane as the scanning direction. The stage 112 is also configured to support a glass slide 114 on which the sample 116 is placed.

[0028] Sample 116 can be anything that can be detected by an optical microscope. For example, glass microscope slides 114 are frequently used as observation substrates for samples including tissues and cells, chromosomes, deoxyribonucleic acid (DNA), proteins, blood, bone marrow, urine, bacteria, beads, biopsy material, or any other type of biological material or substance, which can be dead or alive, stained or unstained, labeled or unlabeled. Sample 116 can also be an array of any type of DNA or DNA-related material, such as complementary DNA (cDNA) or ribonucleic acid (RNA), or proteins deposited on any type of slide or other substrate, including any and all samples commonly referred to as microarrays. Sample 116 can be a microtiter plate (e.g., a 96-well plate). Other examples of sample 116 include integrated circuit boards, electrophoresis records, petri dishes, films, semiconductor materials, forensic materials, and machined parts.

[0029] Objective lens 120 is mounted on objective lens positioner 124, which in this embodiment employs a very precise linear motor to move objective lens 120 along the optical axis defined by objective lens 120. For example, the linear motor of objective lens positioner 124 may include a 50-nanometer encoder. Under the control of processor 104, motion controller 108 coordinates and controls the relative positions of stage 112 and objective lens 120 in the X, Y, and / or Z axes in a closed-loop manner. The processor employs memory 106 to store information and instructions, including computer-executable programming steps for the overall operation of scanning system 100.

[0030] In this embodiment, objective 120 is a plan-field apochromatic (“APO”) infinity-corrected objective suitable for transmission-mode illumination microscopy, reflection-mode illumination microscopy, and / or incident-mode fluorescence microscopy (e.g., Olympus 40X, 0.75 NA or 20X, 0.75 NA). Advantageously, objective 120 is capable of correcting chromatic aberration and spherical aberration. Because objective 120 is infinity-corrected, focusing optics 128 can be positioned in an optical path 122 above objective 120, where the light beam passing through objective 120 is collimated. Focusing optics 128 focuses the light signal acquired by objective 120 onto the light-response elements of line scan camera 130 and / or area scan camera 132, and may include optical components such as filters, magnification switching lenses, etc. Objective 120, in combination with focusing optics 128, provides total magnification for scanning system 100. In one embodiment, the focusing optics 128 may include a tube lens and an optional 2X magnification switch. Advantageously, the 2X magnification switch allows the native 20X objective lens 120 to scan the sample 116 at 40X magnification.

[0031] Line scan camera 130 includes at least one line array of pixels 142. Line scan camera 130 can be monochrome or color. Color line scan cameras typically have at least three line arrays, while monochrome line scan cameras can have a single line array or multiple line arrays. Any type of single or multiple line arrays can also be used, whether they are packaged as part of a camera or custom-integrated into an imaging electronics module. For example, a three-line array (“red-green-blue” or “RGB”) color line scan camera or a ninety-six-line array monochrome TDI can also be used. TDI line scan cameras typically provide a significantly better signal-to-noise ratio (“SNR”) in the output signal by summing intensity data from previously imaged areas of the sample, resulting in an SNR increase proportional to the square root of the integral series. TDI line scan cameras include multiple line arrays. For example, TDI line scan cameras can provide 24, 32, 48, 64, 96, or more line arrays. The scanning system 100 also supports line arrays manufactured in various formats, including some with 512 pixels, some with 1,024 pixels, and others with up to 4,096 pixels. Similarly, line arrays with various pixel sizes can also be used in the scanning system 100. A key requirement for selecting any type of line scan camera 130 is that the movement of the stage 112 can be synchronized with the line frequency of the line scan camera 130, so that the stage 112 can move relative to the line scan camera 130 during digital image acquisition of the sample 116.

[0032] In this embodiment, image data generated by the line scan camera 130 is stored in a portion of memory 106 and processed by processor 104 to generate a series of digital images of at least a portion of sample 116. The series of digital images may be further processed by processor 104, and the processed series of digital images may also be stored in memory 106.

[0033] In embodiments having two or more line scan cameras 130, at least one of the line scan cameras 130 may be configured to act as a focus sensor, which operates in conjunction with at least one of the other line scan cameras 130 configured to act as an imaging sensor. The focus sensor may be logically positioned on the same optical axis as the imaging sensor, or the focus sensor may be logically positioned before or after the imaging sensor relative to the scanning direction of the scanning system 100. In this embodiment where at least one line scan camera 130 acts as a focus sensor, image data generated by the focus sensor may be stored in a portion of memory 106 and processed by processor 104 to generate focus information such that the scanning system 100 can adjust the relative distance between the sample 116 and the objective lens 120 to maintain focus on the sample 116 during scanning. Additionally, in embodiments, at least one line scan camera 130 acting as a focus sensor may be oriented such that each of the plurality of individual pixels 142 of the focus sensor is positioned at a different logical height along the optical path 122.

[0034] In operation, the various components of the scanning system 100 and the programming module stored in the memory 106 enable the automatic scanning and digitization of the sample 116 mounted on a glass slide 114. The glass slide 114 is securely placed on a movable stage 112 of the scanning system 100 for scanning the sample 116. Under the control of the processor 104, the movable stage 112 accelerates the sample 116 to a substantially constant speed for sensing by the line scan camera 130. After scanning a segment of image data, the movable stage 112 decelerates and brings the sample 116 to a substantially complete stop. The movable stage 112 then moves orthogonally to the scanning direction to position the sample 116 for scanning subsequent segments of image data (e.g., adjacent segments). Additional segments are then scanned until the entire portion or the entire sample 116 has been scanned.

[0035] In an embodiment, computer-executable instructions (e.g., programming modules and software) are stored in memory 106 and, when executed, enable scanning system 100 to perform various functions described herein (e.g., displaying a graphical user interface, performing disclosed processes, controlling components of scanning system 100, etc.). In this specification, the term "computer-readable storage medium" is used to refer to any medium used to store computer-executable instructions and to provide these instructions to scanning system 100 for execution by processor 104. Examples of such media include memory 106 and any removable or external storage medium (not shown) directly (e.g., via a universal serial bus (USB), wireless communication protocols, etc.) or indirectly (e.g., via wired and / or wireless networks) coupled in communication with scanning system 100.

[0036] Figure 1B A line scan camera 130 with a single line array 140 is shown, which can be implemented as a charge-coupled device (“CCD”) or complementary metal-oxide-semiconductor (“CMOS”) array. The single line array 140 includes a plurality of individual pixels 142. In the illustrated embodiment, the single line array 140 has 4,096 pixels 142. In alternative embodiments, the line array 140 may have more or fewer pixels. For example, common formats for line arrays include 512, 1,024, and 4,096 pixels. The pixels 142 are arranged linearly to define a field of view 134 of the line array 140. The size of the field of view 134 varies depending on the magnification of the scanning system 100.

[0037] Figure 1C A line scan camera 130 with three line arrays 140 is shown, each of which can be implemented as a CCD array. The three line arrays 140 are combined to form a color array 150. In an embodiment, each individual line array in the color array 150 detects a different color intensity, including, for example, red, green, or blue. Color image data from each individual line array 140 in the color array 150 are combined to form a single field of view 134 of color image data. Other types of color line scan cameras, such as 4-line RGB / monochrome or RGB / NIR, Bayer masks, and prism-based 3-channel R / G / B sensors, can also be used in embodiments employing the disclosed techniques in color imaging scenarios.

[0038] Figure 1D A line scan camera 130 with multiple line arrays 140 is shown, each of which can be implemented as a CCD array. Multiple line arrays 140 are combined to form a TDI array 160. Advantageously, a TDI line scan camera can provide a significantly better SNR in its output signal by summing intensity data from previously imaged regions of a sample, resulting in an SNR increase proportional to the square root of the number of line arrays 140 (also called integration stages). TDI line scan cameras can include even more line arrays 140. For example, common formats for TDI line scan cameras include 24, 32, 48, 64, 96, 120, and even more line arrays 140.

[0039] 2. Quality control process and application

[0040] Figure 2 The document provides high-level illustrations of methods for automatically applying quality control processes to histological images, such as those generated by [examples of methods]. Figure 1A-1DThe scanner creation is described in the context of the process. As shown in the image, after process start 201, it can be determined 202 whether the histological image to which the process is applied has artifacts. As discussed in more detail below, this can be accomplished, for example, by processing the histological image using one or more machine learning models trained for a specific type of artifact, to determine whether the image should be considered to have artifacts identified by the machine learning model trained to do so. If this determination 202 is that the image has artifacts, a second determination 203 can be made regarding whether a flag indicating that the histological image being processed has artifacts is generated. If generated, the flag 204 indicating that the image has the artifacts can be generated, for example, by providing an indication in the interface or storing the flag in a computer-readable storage device. Alternatively, if the image does not have artifacts, or if determination 203 does not generate a flag despite the image having artifacts, it can be checked 205 whether the image should be analyzed for further artifacts. For example, in the case of performing a quality control process using a set of machine learning models, each trained for a specific type of artifact, check 205 can be made by determining whether there are any models that have not yet been used to process the histological image. If there are additional artifacts that need to be processed, Figure 2 The method can proceed to the next artifact (e.g., by incrementing a counter indicating the artifact currently being processed for the histological image) and repeat. Alternatively, the method can be shown as 207 completed (e.g., the quality control process can then be moved to the scanner (e.g., Figure 1A (The next image in a batch of histological images created by the scanning device).

[0041] In including, for example Figure 2 In the embodiments of the method shown, the processing performed in the method can be applied in various ways to address artifacts identified in histological images. For illustration, consider... Figure 3 It shows that it can be based on, for example, the above. Figure 2 The information generated by the methods discussed in this context is provided on the interface of the scanner's display (e.g., a display integrated into the scanner itself, or a separate display connected to the scanner). Figure 3 As shown, this interface can provide markers for identifying specific artifacts in a particular histological image, such as... Figure 3 The "F" mark shown indicates that slide 15 is out of focus. Additionally, it includes, for example, the use of... Figure 2 The interface generated by this method can also provide other information, such as the status of the slide relative to scanning and / or quality control. For example, such as... Figure 3 As shown, the slide may have a state indicating that an image has been created for the slide, but the slide has not yet passed quality control (e.g., Figure 3The slide can also be shown as having the following state: indicating that a histological image corresponding to the slide is currently being created (e.g., ...). Figure 3 The “S” state shown) or the histological image has not yet been scanned (e.g., Figure 3 (The "W" state is shown). It can also provide status information beyond that specific slide. For example, when the instrument is configured to load a slide turret for scanning, if the turret has one or more slots without slides, these slots can be displayed in their own states, such as... Figure 3 The "E" shown indicates an "empty" state. Other states (e.g., different markers that can be used to indicate different artifacts affecting a particular slide image) and other functions beyond simply displaying the state (e.g., displaying scanner trends, such as the total number of artifacts detected over time or the number of specific artifacts, allowing the user to view images generated from a particular slide, etc.) are also possible and can be provided in different embodiments. Therefore, Figure 3 The interface should be understood as illustrative only and should not be considered restrictive.

[0042] For example, you can use, for example Figure 2 The information generated by the method shown can also be applied in other ways, for example, by allowing users to obtain a broader view of the entire laboratory operation. For illustration, consider... Figure 4 The diagram illustrates a laboratory comprising multiple scanners 401-404 capable of creating histological images, and an information technology manager computer 405 that can provide information about all scanners to a dashboard interface 406. In some cases, for example... Figure 5 The method shown provides such a dashboard interface 406, which includes aggregating 501 flags generated based on images created on scanners 401-404 by applying quality control processes, and then using those aggregated flags to generate the dashboard interface 502. Figure 5 As shown, the aggregated flags can be matched with stored remediation data to reveal potential remediation measures that 503 users can apply to address the artifacts causing the marking. Examples of this type of remediation data are provided below in Table 1, which identifies illustrative artifacts that can be marked and the steps that can be taken to resolve them.

[0043]

[0044] Table 1: Example Artifacts and Potential Remediations

[0045] As another example of a potential feature that can be provided by the dashboard interface 406, in some cases, the dashboard interface can provide trend information, such as notifying the user 504 that an increase in artifacts has exceeded a user-specified trend threshold. This could be, for example, an increase in the total number (or percentage) of artifacts marked in lab-generated images, the number (or percentage) of artifacts of a specific type, and / or an increase in the number (or percentage) of artifacts (or artifacts of a specific type) marked in images generated by a specific scanner. This trend information can also be used to provide remedial steps in a manner similar to that described above in the context of Table 1. For example, a trend of increasing digital artifacts (e.g., out-of-focus images, or missing or truncated tissue) in images generated by a specific scanner might lead to a notification to the user that he or she should have the affected scanner repaired or replaced.

[0046] It should be understood that the above text is in Figures 3 to 5 The application described in this context is not intended to be an exhaustive description; for example, methods may be used (e.g., Figure 2 All potential applications of the data generated by the method shown. For example, in some cases, instead of notifying the user of potential remedies, remedies can be applied automatically (e.g., if an image is marked as having image banding artifacts, a system implemented using the disclosed technique can automatically rescan the image while measuring slide opacity for white balance using a new calibration point). In some cases, a hybrid manual / automatic remedy can also be supported (e.g., if a scanner is identified as generating an increased number of marked images, a system based on this disclosure can automatically generate a service call for the scanner while also providing notification that the marked images should be regenerated with a different scanner). As another example of the types of variations that can be supported in some cases, aggregated marking information can also be stored and / or provided to a data repository for downstream analysis. In embodiments that provide potential remedies, more complex logic than that described above can also be used to provide these potential remedies. For example, in the case where a slide is marked as having trapped air, the remedy identification logic can consider whether the slide is associated with a retrospective case. If so, a potential remedy to modify the laboratory's storage procedures can be provided. Otherwise, a potential remedy to modify the mounting step in the laboratory's histological workflow can be provided. Other variations are also possible, and those skilled in the art can implement this disclosure without excessive experimentation. Therefore, the exemplary variations described above, such as those in… Figures 3 to 5 The examples given in this context should be considered illustrative only and should not be construed as limiting the protection provided by this document or any related documents.

[0047] 3. Additional variants

[0048] It should be understood that while the above disclosure provides various examples of alternative methods that can be taken when implementing the disclosed techniques, these alternative methods are intended to be illustrative only and are not intended to provide an exhaustive description of all potential variations. For illustration, consider determining whether to label artifacts other than defocus. While in some cases the methods described above in the context of defocus (e.g., identifying the affected area and comparing the area of ​​the affected area to a threshold to determine whether to label the image) can be applied to other artifacts (e.g., image banding artifacts, vibration artifacts, venetian blind artifacts, stain intensity, presence of dirt and debris on the slide, excessive mounting medium, etc.), other methods are also possible. For example, in some cases, instead of relying on machine learning models, image analysis can be used, such as edge detection and filtering using horizontal lines across the entire image, to identify digital banding artifacts, and similar line detection can be applied to identify vibration artifacts, although in this case it is preferable to generalize to consider lines of any orientation or location. As another example of a potential variation, in the case of identifying missing or truncated tissue, an initial image can be captured by the scanner before scanning the slide, and the bounding box of the tissue in the initial image can be compared with the bounding box of the tissue in the scanned image to determine whether the scanned image is marked as having missing or truncated tissue (e.g., if the initial image shows more tissue than the scanned image shows, the scanned image can be marked for this artifact).

[0049] Even when using machine learning to identify the affected regions and then assess whether those regions are sufficient to label the image, variations are possible. For example, semantic segmentation (e.g., using deep learning models such as DeepLabV3, as described in Chen et al.'s "Rethinking Dilated Convolutions for Semantic Image Segmentation") can... https: / / arxiv.org / abs / 1706.05587 (Originally obtained from [source], the full text of which is hereby incorporated herein by reference) and ResNet-101 (as described by He et al. in "Deep Residual Learning for Image Recognition"), which can be used in [the following text is missing from the original] https: / / arxiv.org / abs / 1512.03385 The core of this document (obtained from [source name], the full text of which is hereby incorporated herein by reference) can be used to identify regions in images with artifacts (e.g., handwriting or other occlusions) at the single-pixel level, rather than relying on the above-mentioned [section / section]. Figure 6 The tiling method described in the context of [the context].

[0050] Beyond determining how to label specific artifacts, variations are also possible in how the disclosed technique can be implemented. For example, in some cases, in addition to labeling artifacts, the disclosed technique may include the ability to capture artifacts for subsequent record preservation or analysis purposes (e.g., a portion of an image identified as having handwriting obscuring tissue can be stored, such that any information conveyed by the handwriting will be preserved even if the slide is cleaned and rescanned to obtain a clear image of the tissue itself). Similarly, in some cases, this can be achieved in dashboard and / or scanner interfaces (e.g., as described above). Figure 3 and 4 Additional functionality is provided in those discussed in the context of [the previous discussion]. For example, in implementations that use thresholds to determine reports (e.g., a severity threshold to determine whether an area is affected, a region threshold to determine whether an affected area should trigger a flag, a trend threshold to determine whether the user should be notified of changes in artifact detection), the ability for the user to modify one or more of those thresholds may be provided by the scanner interface, dashboard interface, or both. Where this type of threshold modification is supported, functionality may also be provided for selectively applying threshold changes to a specific scanner, a specific batch of slides, or even a single slide itself.

[0051] In some cases, there may also be provisions regarding implementation, for example Figure 2 Variations of the advanced methods shown. For example, although following, for example... Figure 2 Some embodiments of the method shown can determine whether each artifact exists in each image, but the disclosed technique can also be implemented such that once certain types of artifacts (e.g., missing tissue) are identified, the analysis of the image can be stopped, allowing the identified artifacts to be resolved (e.g., by rescanning the slide, which may render other artifacts identified in the original image meaningless). Similarly, while some embodiments can identify and generate artifact markers on an artifact-by-artifact basis, some embodiments of the disclosed technique can also identify all artifacts in an image and then generate markers for those artifacts simultaneously, rather than identifying and generating markers for each artifact before moving to the next artifact.

[0052] The physical components that can be used to implement the disclosed techniques may also vary. For example, while in some cases artifact markers in an image can be identified and generated on a scanner that captures the image, the image may also be sent to a separate processor (e.g., a centralized computer in a laboratory or a cloud server accessible via a wide area network) to identify artifacts and generate markers. Different parts of the various methods may also be performed by different physical devices. For example, in some cases, the image may be sent to a remote system (e.g., a cloud server) to apply a machine learning model (e.g., for semantic segmentation or to determine a severity score), and the result may then be returned to a laboratory (or scanner) to be compared with a threshold to determine whether a marker should be generated. Other architectures and methods for performing the processing tasks described herein are also possible and can be implemented by those skilled in the art without excessive experimentation; therefore, the above-described variations should be understood as illustrative only and should not be considered limiting.

[0053] 5. Additional non-restrictive examples

[0054] The above description of the disclosed embodiments is provided to enable those skilled in the art to practice or use the invention. Various modifications to these embodiments will become apparent to those skilled in the art, and the general principles described herein can be applied to other embodiments without departing from the spirit or scope of the invention. Therefore, it should be understood that the description and drawings given herein represent currently preferred embodiments of the invention and thus illustrate the broad subject matter contemplated by the invention. It should also be understood that the scope of the invention fully encompasses other embodiments that may be apparent to those skilled in the art, and therefore the scope of the invention is not limited. Other variations are also possible, and these variations will immediately become apparent to those skilled in the art in light of this disclosure. For example, the following examples are provided as specific (but not limiting) illustrations of various methods that can be taken when implementing the disclosed technology.

[0055] Example 1

[0056] A method includes automatically applying a quality control process to a plurality of histological images by executing computer-executable instructions, wherein: the quality control process includes, for each artifact from a set of artifacts: determining whether the histological image to which the quality control process is applied has the artifact; and, based on determining that the histological image to which the quality control process is applied has the artifact, determining whether to generate a flag indicating that the histological image to which the quality control process is applied has the artifact; and, for a first image from the plurality of histological images, applying the quality control process to the image includes: determining that the image has a first artifact from the set of artifacts; determining whether to generate the flag indicating that the image has the first artifact; and generating the flag indicating that the first image has the first artifact.

[0057] Example 2

[0058] According to the method of Example 1, wherein: determining that the first image has the first artifact includes determining a region on the first image having the first artifact; and determining whether to generate the flag indicating that the first image has the first artifact includes comparing the area of ​​the region on the first image having the first artifact with an affected region threshold of the first artifact.

[0059] Example 3

[0060] According to the method of Example 2, determining the region on the first image having the first artifact includes, for each of a plurality of portions of the first image: determining the severity of the first artifact in the portion of the first image; and comparing the severity of the first artifact in the portion of the first image with a severity threshold for the first artifact.

[0061] Example 4

[0062] According to the method of Example 3, the method includes providing an interface operable by a user to specify the following: the affected area threshold of the first artifact; and the severity threshold of the first artifact.

[0063] Example 5

[0064] According to any one of Examples 1 to 4, the method comprises: the plurality of histological images including a plurality of sets of histological images; for each set of histological images from the plurality of sets of histological images; the set of histological images corresponds to a scanner from a set of scanners; none of the scanners in the set of scanners corresponds to more than one set of histological images from the plurality of sets of histological images; the histological image in the set of histological images is generated by scanning the slides corresponding to the set of histological images using the scanner corresponding to the set of histological images for each slide from the set of slides corresponding to the set of images; and the method includes generating a dashboard interface based on aggregating each flag, each flag being generated based on applying the quality control process to each histological image from the plurality of sets of histological images.

[0065] Example 6

[0066] According to the method described in Example 5, the method includes displaying potential remedies for at least one artifact from the set of artifacts via the dashboard interface.

[0067] Example 7

[0068] According to any one of Examples 5 to 6, the method includes notifying the user of the increase in artifacts via the dashboard interface based on the increase in artifacts exceeding a user-specified trend threshold.

[0069] Example 8

[0070] The method according to any one of Examples 1 to 7, wherein the method includes displaying a scanner interface on a display of a first scanner, the scanner interface including a mark indicating that the first image has the first artifact, wherein the first image is a histological image created by scanning a slide using the first scanner.

[0071] Example 9

[0072] According to the method of Example 8, the method includes displaying via the scanner interface for at least one slide from the plurality of slides: a histological image corresponding to the slide has been created; and the quality control process has not yet been applied to the histological image corresponding to the slide.

[0073] Example 10

[0074] According to any one of Examples 1 to 9, the set of artifacts includes: image banding artifacts, trapped air, traces of tissue obscuration, tissue loss, and defocus.

[0075] Example 11

[0076] A system includes: a first scanner operable to generate a plurality of histological images based on scanning tissue on each of a plurality of slides; one or more non-transitory computer-readable media storing instructions to apply a quality control process to the one or more histological images upon execution, wherein the quality control process is operable to determine, for each histological image to which the quality control process is applied, for each artifact from a set of artifacts: whether the histological image to which the quality control process is applied has the artifact; and, based on the determination that the histological image to which the quality control process is applied has the artifact, whether to generate a flag indicating that the histological image to which the quality control process is applied has the artifact.

[0077] Example 12

[0078] According to the system described in Example 11, wherein for a first artifact from the set of artifacts: the quality control process determines whether a flag indicating that the histological image to which the quality control process is applied has the first artifact is generated includes: determining that a region on the histological image to which the quality control process is applied has the first artifact; and comparing the area of ​​the region on the image to which the quality control process is applied with the first artifact with a threshold for the affected region of the first artifact.

[0079] Example 13

[0080] According to the system described in Example 12, determining the region with the first artifact on the image to which the quality control process is applied includes, for each of a plurality of portions of the image to which the quality control process is applied: determining the severity of the first artifact in the portion of the image to which the quality control process is applied; and comparing the severity of the first artifact in the portion of the image to which the quality control process is applied with a severity threshold for the first artifact.

[0081] Example 14

[0082] According to the system described in Example 13, the one or more non-transitory computer-readable media store instructions to provide, when executed, an interface operable by a user to specify: the affected area threshold of the first artifact; and the severity threshold of the first artifact.

[0083] Example 15

[0084] According to any one of Examples 11 to 14, the system includes a set of scanners, the set of scanners including the first scanner; each scanner from the set of scanners is operable to generate a set of histological images; the one or more non-transitory computer-readable media include instructions to, upon execution: apply the quality control process to each histological image generated by the set of scanners; and generate a dashboard interface based on the aggregation of each flag, each flag being generated based on the application of the quality control process to each histological image generated by the set of scanners.

[0085] Example 16

[0086] According to the system described in Example 15, the dashboard interface is configured to present potential remedies for at least one artifact from the set of artifacts.

[0087] Example 17

[0088] According to any one of Examples 15 to 16, in the system, wherein the one or more non-transitory computer-readable media include instructions operable to notify a user of the increase in artifacts via the dashboard interface based on the increase in artifacts exceeding a user-specified trend threshold when executed.

[0089] Example 18

[0090] According to any one of Examples 11 to 17, the system wherein the one or more non-transitory computer-readable media includes instructions to display the scanner interface on the display of the first scanner upon execution, the scanner interface presenting each sign generated by applying the quality control process to the plurality of histological images.

[0091] Example 19

[0092] According to the system described in Example 18, wherein the scanner interface is configured to display a state from a set of states for each of the plurality of slides, the set of states including: a state indicating that a histological image corresponding to the slide has not yet been generated; a state indicating that the histological image corresponding to the slide has been generated but the quality control process has not yet been applied; and a state indicating that the histological image corresponding to the slide has been applied to the quality control process.

[0093] Example 20

[0094] According to any one of Examples 11 to 19, the set of artifacts includes: image banding artifacts; trapped air; traces of tissue obscuration; tissue loss; and defocus.

[0095] 6. Explanation

[0096] Any examples or illustrations presented herein should not be construed as limiting the scope of any claims included in this document or any related document. Rather, the protection provided by this document or any related document shall be understood as being limited by the claims of the relevant document, whereby those claim terms that are expressly defined herein are given their express definitions, and terms that are not expressly defined are given the broadest reasonable interpretation available from a general dictionary.

[0097] The combinations described herein, such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof", include any combination of A, B, and / or C, and may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A, B, and C, and any such combination may contain one or more members of its constituent parts A, B, and / or C. For example, a combination of A and B may include one A and multiple B, multiple A and one B, or multiple A and multiple B.

[0098] A statement that something is “based on” another should be understood to mean that the thing is at least partially determined by the other thing on which it is “based.” While a “based on” relationship includes situations where one thing is entirely determined by the other, it should not be understood as limited to situations where one thing is entirely determined by the other unless the phrase used is “entirely based.”

Claims

1. A method comprising automatically applying a quality control process to multiple histological images by executing computer-executable instructions, wherein: -The quality control process includes, for each artifact from a set of artifacts: • Determine whether the histological images obtained using the quality control process have the artifacts described; as well as • Based on the determination that the histological image to which the quality control process is applied has the artifact, determine whether to generate a flag indicating that the histological image to which the quality control process is applied has the artifact; as well as - For a first image from the plurality of histological images, applying the quality control process to the image includes: • Determine that the image has a first artifact from the set of artifacts; • Determine whether to generate the flag indicating that the image has the first artifact; and • Generate the flag indicating that the first image has the first artifact.

2. The method according to claim 1, wherein: Determining that the first image has the first artifact includes determining the region on the first image that has the first artifact; as well as Determining whether to generate a flag indicating that the first image has the first artifact includes comparing the area of ​​the region on the first image with the first artifact with a threshold for the affected area of ​​the first artifact.

3. The method of claim 2, wherein determining the region on the first image having the first artifact includes for each of a plurality of portions of the first image: - Determine the severity of the first artifact in the portion of the first image; and - Compare the severity of the first artifact in the portion of the first image with a severity threshold for the first artifact.

4. The method of claim 3, wherein the method includes providing an interface operable by a user to specify the following: - The threshold value of the affected region of the first artifact; and -The severity threshold of the first artifact.

5. The method according to claim 1, wherein: -The plurality of histological images includes multiple sets of histological images; -For each set of histological images from the plurality of sets of histological images: • The set of histological images corresponds to a scanner from a set of scanners; • No scanner in the set of scanners corresponds to more than one set of histological images from the multiple sets of histological images; • The histological image in the set of histological images is generated by scanning the slides with the scanner corresponding to the set of histological images for each slide from a set of slides corresponding to the set of images; and The method includes generating a dashboard interface based on the aggregation of each flag, each flag being generated based on applying the quality control process to each histological image from the multiple sets of histological images.

6. The method of claim 5, wherein the method includes displaying potential remedies for at least one artifact from the set of artifacts via the dashboard interface.

7. The method of claim 5, wherein the method includes notifying the user of the increase in artifacts via the dashboard interface based on the increase in artifacts exceeding a user-specified trend threshold.

8. The method of claim 1, wherein the method includes displaying a scanner interface on a display of a first scanner, the scanner interface including a marker indicating that the first image has the first artifact, wherein the first image is a histological image created by scanning a slide using the first scanner.

9. The method of claim 8, wherein the method includes displaying, via the scanner interface, at least one slide from the plurality of slides: - Histological images corresponding to the slides have been created; and - The quality control process has not yet been applied to the histological images corresponding to the slides.

10. The method of claim 1, wherein the set of artifacts comprises: -Image banding artifacts; - Retained air; - To conceal traces of tissue; - Organizational deficiencies; as well as -Out of focus.

11. A system comprising: - A first scanner, wherein the first scanner is operable to generate multiple histological images based on scanning tissue on the slides for each of the multiple slides; - One or more non-transitory computer-readable media, the non-transitory computer-readable media storing instructions to apply a quality control process to one or more histological images upon execution, wherein the quality control process is operable to determine, for each histological image to which the quality control process is applied, for each artifact from a set of artifacts: • Whether the histological images obtained using the quality control process have the artifacts; as well as • Based on the determination that the histological image to which the quality control process is applied has the artifact, whether to generate a flag indicating that the histological image to which the quality control process is applied has the artifact.

12. The system of claim 11, wherein for the first artifact from the set of artifacts: - The quality control process determines whether the histological image indicating the application of the quality control process has the first artifact by including the following indicators: • Determine that the region on the histological image to which the quality control process is applied has the first artifact; as well as • Compare the area of ​​the region with the first artifact on the image to which the quality control process is applied with a threshold value for the affected area of ​​the first artifact.

13. The system of claim 12, wherein determining the region having the first artifact on the image to which the quality control process is applied includes for each of a plurality of portions of the image to which the quality control process is applied: - Determine the severity of the first artifact in the portion of the image to which the quality control process is applied; and - The severity of the first artifact in the portion of the image to which the quality control process is applied is compared with a severity threshold for the first artifact.

14. The system of claim 13, wherein the one or more non-transitory computer-readable media store instructions to provide, when executed, an interface operable by a user to specify the following: - The threshold value of the affected region of the first artifact; and -The severity threshold of the first artifact.

15. The system according to claim 11, wherein: - The system includes a set of scanners, the set of scanners including the first scanner; - Each scanner from the set of scanners is capable of operating to generate a set of histological images; - The one or more non-transitory computer-readable media include instructions to, when executed: • Apply the quality control process to each histological image generated by the set of scanners; • The dashboard interface is generated by aggregating each icon, where each icon is generated based on applying the quality control process to each histological image generated by the set of scanners.

16. The system of claim 15, wherein the dashboard interface is configured to present potential remedies for at least one artifact from the set of artifacts.

17. The system of claim 15, wherein the one or more non-transitory computer-readable media include instructions operable to notify a user of the increase in artifacts via the dashboard interface based on an increase in artifacts exceeding a user-specified trend threshold when executed.

18. The system of claim 11, wherein the one or more non-transitory computer-readable media include instructions to display the scanner interface on the display of the first scanner upon execution, the scanner interface presenting each mark generated by applying the quality control process to the plurality of histological images.

19. The system of claim 18, wherein the scanner interface is configured to display a state from a set of states for each of the plurality of slides, the set of states including: - Indicates the state that a histological image corresponding to the slide has not yet been generated; - Indicates the state of the histological image corresponding to the slide that has been generated but the quality control process has not yet been applied; as well as - Indicates the status of the histological image corresponding to the slide, to which the quality control process has been applied.

20. The system of claim 11, wherein the set of artifacts comprises: -Image banding artifacts; - Retained air; - To conceal traces of tissue; - Organizational deficiencies; as well as -Out of focus.

Images