System and method for digitizing slide
The system uses an imaging device and computing device to generate initial slide images, determine quality control metrics, and rescanning flagged slides to address the challenge of balancing time and coverage in slide scanning, enhancing the efficiency and accuracy of the digitization process.
Patent Information
- Application Number
- JP2025084823
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-04-04
- Filing Date
- 2025-05-21
- Publication Date
- 2025-08-13
AI Technical Summary
Balancing the time required to scan a slide with the need to ensure coverage of necessary elements is difficult, necessitating an automated workflow to minimize the chance of missing critical content.
A system and method that includes an imaging device and a computing device to generate initial slide images, determine quality control metrics, flag slides based on these metrics, and rescanning using generated scanning profiles to ensure comprehensive coverage.
Ensures efficient and thorough digitization of slides by flagging and rescanning those that require additional attention, thereby improving the accuracy and completeness of the scanning process.
Smart Images

Figure 2025118983000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates generally to the field of imaging, and more particularly to a system and method for digitizing slides and methods for using the same. [Background technology]
[0002] Balancing the time required to scan a slide with the need to ensure coverage of the necessary elements is sometimes difficult. An automated workflow is needed to minimize the chance of missing the necessary content present on the slide within the desired time. Summary of the Invention [Means for solving the problem]
[0003] In one aspect, a system for digitizing slides is described. The system includes at least one imaging device and a computing device communicatively connected to the imaging device, the computing device configured to scan each of the plurality of slides with the imaging device to generate an initial slide image and associated scanning metadata for each of the plurality of slides, and to determine a quality control metric for each of the plurality of slides as a function of the initial slide image and the associated scanning metadata, wherein determining the quality control metric includes flagging the slide based on the determined quality control metric. The computing device is further configured to generate at least one scanning profile as a function of the scanning metadata based on the quality control metric, and to rescan the flagged slides in the plurality of slides with the imaging device using the at least one scanning profile.
[0004] In another aspect, a method for digitizing slides is described. The method includes scanning each of a plurality of slides using at least one imaging device to generate, for each slide of the plurality of slides, an initial slide image and associated scanning metadata; and determining, by a computing device, a quality control metric for each of the plurality of slides as a function of the initial slide image and the associated scanning metadata. Determining the quality control metric includes flagging the slide based on the determined quality control metric. The method further includes generating, by a computing device, at least one scanning profile as a function of the scanning metadata based on the quality control metric; and rescanning, at the at least one imaging device, the flagged slides in the plurality of slides using the at least one scanning profile.
[0005] These and other aspects and features of non-limiting embodiments of the present invention will become apparent to those skilled in the art upon review of the following description of specific non-limiting embodiments of the present invention in conjunction with the accompanying drawings. For the purpose of illustrating the invention, the drawings show aspects of one or more embodiments of the invention. It is to be understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown in the drawings. [Brief explanation of the drawings]
[0006] [Figure 1] 1 is an exemplary embodiment of a system for digitizing slides. [Figure 2] 1 illustrates an exemplary embodiment of a decision tree for the automatic generation of a scanning profile. [Figure 3A] 1 illustrates an exemplary embodiment of scanning parameters for hardware components. [Figure 3B]1 illustrates an exemplary embodiment of scanning parameters for hardware components. [Figure 4] FIG. 1 is a block diagram of an example embodiment of a machine learning module. [Figure 5] FIG. 1 is a diagram of an exemplary embodiment of a neural network. [Figure 6] FIG. 2 is a block diagram of an exemplary embodiment of a node of a neural network. [Figure 7] FIG. 1 is a flow diagram illustrating an exemplary embodiment of a method for digitizing a slide. [Figure 8] 1 is a block diagram of a computing system that may be used to implement any one or more of the methodologies disclosed herein and any one or more portions thereof. The drawings are not necessarily to scale and may be illustrated with phantom lines, schematic representations, or fragmentary views. In certain instances, details that are not necessary for an understanding of the embodiments or that obscure other details may be omitted. DETAILED DESCRIPTION OF THE INVENTION
[0007] At a high level, aspects of the present disclosure relate to systems and methods for digitizing slides, the system including at least one imaging device, at least one slide storage unit containing a plurality of slides, and a computing device communicatively coupled to the imaging device, the computing device configured to scan each of the plurality of slides with the imaging device to generate, for each of the plurality of slides in the at least one slide storage unit, an initial slide image and associated scanning metadata, and determine, for each of the plurality of slides, a quality control metric as a function of the initial slide image and the associated scanning metadata, wherein determining the quality control metric includes flagging the slide based on the determined quality control metric. The computing device is further configured to generate at least one scanning profile as a function of the scanning metadata based on the quality control metric, and to rescan flagged slides in the plurality of slides at the imaging device using the at least one scanning profile. Exemplary embodiments illustrating aspects of the present disclosure are described below in the context of several specific examples.
[0008] Referring now to FIG. 1 , an exemplary embodiment of a system 100 for digitizing a slide 104 is illustrated. In this disclosure, a “slide” refers to a container or surface that holds a specimen. In some cases, the specimen may include, but is not limited to, a tissue sample, a cell structure, or other biological material. In some embodiments, the slide 104 may include a formalin-fixed, paraffin-embedded slide. In some embodiments, the specimen on the slide 104 may be stained. In some embodiments, the slide 104 may be substantially transparent. In some embodiments, the slide 104 may include a glass slide. In some embodiments, the slide 104 may include a thin, flat, substantially transparent glass slide. In some cases, a cover, such as a transparent cover, may be applied to the slide 104 such that the specimen is disposed between the slide 104 and the cover. For example, but not limited to, the specimen may be compressed between the slide 104 and the corresponding cover. In a non-limiting example, the slide 104 may include a histopathology slide.
[0009] With continued reference to FIG. 1 , the system 100 can include at least one slide storage unit 108. As used in this disclosure, "slide storage unit" refers to a specialized storage or containment system designed to securely hold, organize, and protect a plurality of slides 112. In some cases, the at least one slide storage unit 108 can be configured to keep the plurality of slides 112 free from dust, damage, and contamination. In some cases, the at least one slide storage unit 108 can be designed to allow for easy removal and placement of the plurality of slides 112 for viewing or scanning. In a non-limiting embodiment, the at least one slide storage unit 108 can include a slide basket, which can be a structured container made of a durable material such as plastic, stainless steel, or a combination thereof. In some cases, the slide basket can have a rectangular shape. In some cases, the slide basket casing can resist chemicals, heat, and other environmental factors that may be present during slide removal, processing, or storage.
[0010] Still referring to FIG. 1 , in some cases, at least one slide storage 108, such as a slide basket, may include one or more compartments or slots each configured to hold one slide 104. In some cases, the one or more compartments may be configured to ensure that each slide of the plurality of slides 112 is kept separated from one another, preventing scratching, overlap, or any physical contact that could damage the slide surface or the specimens mounted thereon. Additionally or alternatively, the slide basket may include a handle (e.g., an ergonomic handle) to allow a user, an actuator mechanism, and / or a transport component, such as those described in detail below, to easily lift and transport the slide between system components. In some cases, the entire slide basket may be submerged in a processing solution or water bath. In some embodiments, the slide basket may include a foldable or removable handle for compact storage. Additionally, the slide basket may be breathable; for example, but not limited to, one or more casings of the slide basket may include an open mesh or perforated pattern configured to facilitate rapid drying of slides after cleaning or chemical treatment and to prevent residue from accumulating within the basket.
[0011] Continuing to refer to FIG. 1 , in some embodiments, the bottom of the slide basket may be slightly raised or include small feet, e.g., the bottom of the slide basket may not be flush with the surface. This allows for better air circulation and prevents moisture from building up within the at least one slide storage section 108. In some embodiments, the at least one slide storage section 108 may also include a locking mechanism. In a non-limiting example, a lid with a latch or magnetic seal may be integrated, and the lid may fit tightly onto the top of the slide basket to provide a secure seal. Additionally or alternatively, the lid may include one or more transparent windows or transparent materials to allow visual inspection of the plurality of slides 112 without having to open the at least one slide storage section 108. In this case, the plurality of slides 112 stored within the slide storage section 108 may be held in place and prevented from moving laterally or vertically within the compartment. In some cases, the capacity of the at least one slide storage section may be variable. For example, but not by way of limitation, the at least one slide storage section can include a small slide basket designed to hold approximately 10 slides, or a larger slide basket capable of holding 50 or more slides.
[0012] Still referring to FIG. 1 , at least one slide storage section 108 (e.g., a slide basket) may additionally or alternatively include a basket adapter; for purposes of this disclosure, a “basket adapter” is a device or component within the slide basket configured to enhance the scanning process. In some cases, the basket adapter may include one or more slots, as described above, that vertically support the weight and load of the plurality of slides 112. In some embodiments, the basket adapter may be designed with a uniform tilt angle. For example, without limitation, an angle between 5° and 90° may be applied to the bottom of the slide basket. In some cases, the tilt angle is calibrated to reliably provide the necessary support for the plurality of slides 112 up to a threshold where one or more of the plurality of slides 112 are no longer securely secured within the slide basket, preventing unintentional dislodgement or misalignment during the scanning process, as described below. In some cases, such a basket adapter may facilitate access to transport components. Because the tilt angle of the basket adapter provides a natural guide, the transport components do not need to make any additional adjustments or movements to retrieve or place the slides 104.
[0013] Still referring to FIG. 1 , the slide basket may further include multiple sensors, such as, but not limited to, pressure sensors, proximity sensors, etc., to detect internal conditions (e.g., the presence of multiple slides 112). In a non-limiting example, each sensor of the multiple sensors may be located at the bottom of each slot or compartment in direct or indirect contact with or proximity to multiple slides 112 stored within the slide basket. In some cases, at least one slide storage section 108 is configured to communicate with other devices in the system 100. For example, but not limited to, feedback from the multiple sensors (e.g., whether a slot is occupied) may be relayed to an imaging device or any computing device, as described in more detail below, to enable real-time monitoring of the scanning process and thereby ensure efficient slide retrieval and / or placement.
[0014] With continued reference to FIG. 1 , system 100 includes at least one imaging device 116. As described in this disclosure, an “imaging device” is a device configured to capture visual information in the form of an image or a sequence of images. In one embodiment, imaging device 116 converts an optical image into an electronic signal, which can be processed, stored, or displayed, as described in more detail below. For example, without limitation, the imaging devices described herein may in some cases be used specifically for medical diagnostics in a clinical setting, such as a microscope configured to capture detailed, high-resolution images of a microscopic object to enable accurate diagnosis, disease monitoring, and other biological studies. In a non-limiting example, imaging device 116 may be compatible with any imaging device such as those described in U.S. Patent Application No. 18 / 226,058 (Attorney Docket No. 1519-102USU1), filed July 25, 2023, entitled "IMAGING DEVICE AND A METHOD FOR IMAGE GENERATION OF A SPECIMEN," which is incorporated herein by reference in its entirety.
[0015] With continued reference to FIG. 1 , in one embodiment, the imaging device 116 can include at least one stage 120 configured to receive and hold the slide 104. As used in this disclosure, a “stage” refers to a flat surface (i.e., a uniform surface without significant irregularities or curvatures) on which the slide 104 or specimen is placed for examination. In some cases, the stage can include a slide port with one or more alignment features, which, as described herein, are physical features that secure the received slide 104 in place and / or help align the slide with other components of the imaging device 116. In some embodiments, the alignment features can include components that secure the slide 104, such as clamps, latches, clips, recesses, or other fasteners. In some embodiments, the stage 120 can facilitate the removal and insertion of the slide 104. In some embodiments, the stage 120 can include a transparent surface through which light can pass. In some embodiments, the slide 104 can be placed on such a transparent surface of the stage 120 and / or illuminated by light traveling through such a transparent surface. In some embodiments, the stage 120 may be mechanically connected to an actuator mechanism 124, as described below.
[0016] Still referring to FIG. 1 , in some cases, the imaging device 116 may include an actuator mechanism 124. As used herein, an “actuator mechanism” is a mechanical component configured to change the position of the slide relative to the optical system. In a non-limiting example, the actuator mechanism 124 can be configured to precisely move the stage 120 in the X, Y, and / or Z directions, allowing for detailed examination of different portions of the specimen. In one or more embodiments, the actuator mechanism 124 may be used to change the line of sight so that an image of the slide 104 can be captured, as discussed further in this disclosure. In some embodiments, the actuator mechanism 124 may be mechanically connected to the slide 104, such as the slide 112 in a slide port. In some embodiments, the actuator mechanism 124 may be mechanically connected to the slide port. For example, the actuator mechanism 124 can move the slide port to move the slide 104. For example, but not by way of limitation, the actuator mechanism 124 can move the slide port to change the distance D between the top surface of the slide 104 and the optical components described below.
[0017] 1 , in some embodiments, actuator mechanism 124 can also change the angle between the top surface (e.g., the surface that is directed toward or facing the optics and that slide 104 and / or specimen contact) and the ground. In some embodiments, actuator mechanism 124 can be mechanically connected to a movable element (i.e., any movable or portable object, component, or device) within imaging device 100, such as, but not limited to, slide 112, a slide port, stage 120, or an optics, as described in more detail below. In some embodiments, actuator mechanism 124 can be configured to change the relative position of slide 104 and the optics by moving stage 120, slide 104, and / or the optics in the X, Y, and / or Z directions during slide scanning and / or rescanning processes, as described in more detail below.
[0018] Still referring to FIG. 1 , in some embodiments, the actuator mechanism 124 may include a hydraulic actuator. A hydraulic actuator may be comprised of a cylinder or fluid motor that uses hydraulic power to facilitate mechanical movement. The output of the hydraulic actuator mechanism 124 may include mechanical movement, such as, but not limited to, linear, rotary, or oscillatory movement. In some embodiments, the hydraulic actuator may employ hydraulic fluid. Because liquids are potentially incompressible, hydraulic actuators may exert large forces. Furthermore, because force is equal to pressure multiplied by area, hydraulic actuators may function as force transducers with changes in area (e.g., the cross-sectional area of the cylinder and / or piston). An exemplary hydraulic cylinder may be comprised of a hollow cylindrical tube through which a piston can slide. In some cases, the hydraulic cylinder may be considered single-acting. The term "single-acting" may be used when fluid pressure is applied substantially only to one side of the piston. Thus, a single-acting piston can move in only one direction. In some cases, a spring may be used to provide a return stroke for the single-acting piston. In some cases, the hydraulic cylinder may be double-acting. "Double acting" may be used when pressure is applied to substantially both sides of the piston. The force difference between the two sides of the piston causes the piston to move.
[0019] Still referring to FIG. 1 , in some embodiments, actuator mechanism 124 may include a pneumatic actuator mechanism. In some cases, pneumatic actuators can generate large forces from relatively small changes in gas pressure. In some cases, pneumatic actuators can respond more quickly than other types of actuators, such as hydraulic actuators. Pneumatic actuators can use compressible fluids (e.g., air). In some cases, pneumatic actuators can operate with compressed air. Operation of hydraulic and / or pneumatic actuators includes control of one or more valves, circuits, fluid pumps, and / or fluid manifolds.
[0020] Still referring to FIG. 1 , in some cases, the actuator mechanism 124 may include an electric actuator. The electric actuator mechanism 124 may include either an electromechanical actuator, a linear motor, or the like. In some cases, the actuator mechanism 124 may include an electromechanical actuator. An electromechanical actuator can convert the rotational force of an electric rotary motor into linear motion, generating linear motion through a mechanism. Exemplary mechanisms include, but are not limited to, a rotary-to-translational converter, such as a belt, a screw, a crank, a cam, a linkage, or a scotch yoke. In some cases, control of the electromechanical actuator may include control of an electric motor; for example, a control signal may control one or more electric motor parameters to control the electromechanical actuator. Exemplary, non-limiting electric motor parameters include rotational position, input torque, speed, current, and potential. The electric actuator mechanism 124 may include a linear motor. A linear motor may differ from an electromechanical actuator in that power from a linear motor is output directly as translational motion, rather than being output as rotational motion and converted to translational motion. In some cases, a linear motor may incur less frictional loss than other devices. Linear motors may be designated into at least three different categories, such as flat linear motors, U-channel linear motors, and tubular linear motors. Linear motors may be directly controlled by control signals that control one or more linear motor parameters. Exemplary linear motor parameters include, but are not limited to, position, force, velocity, potential, and current.
[0021] Still referring to FIG. 1 , in some embodiments, the actuator mechanism 124 may include a mechanical actuator mechanism 124. In some cases, the mechanical actuator mechanism 124 may function to perform movement by converting one type of movement, such as rotational movement, into another type of movement, such as linear movement. An exemplary mechanical actuator includes a rack and pinion. In some cases, a mechanical power source, such as a power take-off, may serve as a power source for the mechanical actuator. The mechanical actuator may employ any number of mechanisms, including, for example, but not limited to, gears, rails, pulleys, cables, linkages, etc.
[0022] 1 , in some cases, system 100 may include a transport component 128, where a “transport component,” as described herein according to some embodiments, refers to a device or system configured to move, transport, or position an object (e.g., a slide 104, or any slide of the plurality of slides 112 in at least one slide storage component 108 described above) from one location to another. In some cases, transport component 128 may be disposed between at least one slide storage component 108 and imaging device 116 or any other processing device configured to automate the process of retrieving, positioning, and / or returning slides, ensuring that each slide of the plurality of slides 112 is efficiently moved between at least one slide storage component 108 and imaging device 116.
[0023] Still referring to FIG. 1 , in a non-limiting example, the transport component 128 may include a programmable robotic arm configured to pick up, move, and place the slides 104. In some cases, the transport component 128 may include one or more joints, each of which may allow at least a portion of the transport component to move within a predetermined range in the X, Y, and / or Z directions. The transport component 128 may also include a gripping component disposed at a distal end of the transport component 128, which may be configured to securely hold and release one or more slides. Such gripping components may be fabricated from a soft, non-abrasive material so as not to damage the held slides during slide transfer. In some cases, the transport component 128 may employ a pinch or suction mechanism to pick up and release slides. In other cases, the transport component 128 may be integrated with the actuator mechanism 124 described above, for example, but not by way of limitation, having a mechanism for converting rotational motion into linear motion, which can then be utilized to precisely move or position one or more slides on the stage 120 of the imaging device 116. In a non-limiting example, such a mechanism may include any mechanism that converts rotary motion into linear motion, such as those described in U.S. Patent Application No. 18 / 382,386 (Attorney Docket No. 1519-109USU1), filed October 20, 2023, entitled "APPARATUS AND METHOD OF USE OF A MECHANISM THAT CONVERTS ROTARY MOTION INTO LINEAR MOTION," which is incorporated herein by reference in its entirety.
[0024] 1 , transport component 128 can include one or more sensors as described herein, such as, but not limited to, proximity sensors or force sensors, configured to detect the precise position of the transported slide so that it is accurately positioned for imaging or storage. In some cases, such information may be received from other devices in system 100; for example, transport component 128 may be in communication with imaging device 116, and one or more sensors (e.g., one or more pressure sensors) integrated into imaging device 116 may be configured to detect the positioning of slide 104. Imaging device 116 may send a signal to transport component 128 upon successful slide positioning (i.e., when the slide is properly positioned on stage 120 within the registration features).
[0025] Still referring to FIG. 1 , in some cases, the transport component 128 and / or the actuator mechanism 124 may include one or more cushioning components to minimize vibration or shock during slide transport. The cushioning components may be configured to ensure that the transported slides are undamaged and remain in their intended position at the end of the process. One or more computing devices, as described in detail below, may be configured to control the transport component 128, as well as the actuator mechanism 124 described herein, to follow a predefined path for transporting slides between at least one slide storage section 108 and the imaging device 116. Additionally or alternatively, safety features (e.g., collision detection) may be implemented in the transport component 128 to stop or back up if an obstacle is detected. Other exemplary embodiments of the transport mechanism described herein include, but are not limited to, a belt conveyor system, a linear actuator, a rotary table (i.e., a rotating platform / stage that holds multiple slides), and the like.
[0026] Continuing with reference to FIG. 1 , in some embodiments, imaging device 116 may include optical system 132. As used in this disclosure, an “optical system” is an arrangement of one or more components that operates on or employs electromagnetic radiation, such as light (e.g., visible light, infrared light, ultraviolet light, etc.). Optical system 132 may include one or more optical components 136, and each “optical component” described herein refers to any device or portion of a device that manipulates, interacts with, or is affected by light. In non-limiting examples, optical components 136 may include lenses, mirrors, windows, filters, etc. In some cases, optical system 132 can form an optical image corresponding to an optical object. For example, without limitation, optical system 132 can form an optical image at or on optical sensor 140, which can capture (e.g., digitize) the optical image, as described in more detail below. In some cases, optical system 132 can have at least one magnification. For example, without limitation, optical system 132 may include an objective lens (e.g., a microscope objective lens) and one or more reimaging optics that together produce optical magnification, as described in more detail below. The degree of optical magnification is sometimes referred to as zoom.
[0027] Still referring to FIG. 1 , in some cases, the optical system 132 may be a light source 144. As used in this disclosure, a "light source" is any device configured to emit electromagnetic radiation. In some embodiments, the light source may emit light having substantially one wavelength. In some embodiments, the light source 144 may emit light having a range of wavelengths. Light emitted by the light source 144 as described herein may include, but is not limited to, ultraviolet, visible, and / or infrared light. In a non-limiting example, the light source 144 may include a light-emitting diode (LED), an organic LED (OLED), and / or other light emitter. Such a light source may be configured to illuminate the slide 104 and / or the slide port and / or the specimen on the stage 120. In a non-limiting example, the light source 144 may illuminate the slide 104 and / or the slide port and / or the specimen on the stage 120 from below, as illustrated in FIG. 1 . In another non-limiting example, the light source 144 may illuminate the slide 104 and / or the specimen from above.
[0028] Continuing with reference to FIG. 1 , in some cases, the light source 144 may be connected to one or more optical components 136 as described herein, such as, but not limited to, a collection lens (i.e., an optical component used to collect and focus light emitted by the light source 144 onto the slide 104 and / or specimen). In some cases, the collection lens may be configured to collect and evenly distribute light so that the slide 104 and / or specimen is evenly illuminated, thereby improving image resolution and contrast. In a non-limiting example, an optical component 136, such as a collection lens, may focus the light emitted from the light source 144 into a cone of light that illuminates the slide 104 and / or specimen with a uniform intensity across the entire visible range. In some cases, the collection lens may include an aperture stop (e.g., a variable aperture that can be adjusted to control the amount of light reaching the slide 104 and / or specimen). In some cases, adjusting such an stop can affect the contrast and depth of field of the image.
[0029] Continuing with reference to FIG. 1 , in some cases, the optical components 136 may also include an objective lens. As used in this disclosure, an “objective lens” refers to an optical component that collects light from the slide 104 and / or specimen and focuses the light to generate an optical image within the imaging device 116. In some embodiments, the generated optical image may be magnified by an eyepiece for viewing by a human operator or captured by an optical sensor 140, as described in detail below, for slide scanning and / or digital imaging. In some embodiments, the optical system 132 may include three objective lenses, each of which may include various magnifications ranging from 4x to 100x or more. In some cases, the magnification may be marked on the casing of the optical component. In some embodiments, the optical components 136 may have different numerical apertures (NA), which measure the lens's ability to collect light at a given subject distance and resolve details on the slide 104 and / or specimen. For example, but not by way of limitation, a higher NA indicates a higher resolving power. Exemplary types of objective lenses include, but are not limited to, dry objective lenses, immersion objective lenses, water immersion objective lenses, and the like.
[0030] Still referring to FIG. 1 , in some embodiments, one or more optical components 136 in the optical system 132 can be attached to a nosepiece of the imaging device 116. As used in this disclosure, a “nosepiece” is a portion of the imaging device 116 that holds multiple optical components (e.g., multiple objective lenses), as shown in FIG. 1 . In some cases, the nosepiece may include a rotating nosepiece (also known as a turret), which may include a rotatable part located in a head portion of the imaging device 116 and configured to hold multiple objective lenses. The optical components 136 can be quickly and efficiently swapped with one another using the rotating nosepiece during imaging of multiple slides 112. In some embodiments, the optical system 132 may be parfocal. For example, without limitation, when a first objective lens is focused, switching to a second or third objective lens may require minimal refocusing. In some cases, multiple optical components 136 may be spaced at regular intervals on a rotating nosepiece, with each optical component being a fixed distance from stage 120 .
[0031] It should be noted that the number of optical components 136 in the optical system 132 described above is exemplary and should not be considered limiting. The actual number of optical components is variable, and the optical system 132 can incorporate more or fewer optical components as desired. Those skilled in the art will recognize, upon reviewing this disclosure in its entirety, various modifications, variations, and alternative configurations that may be applicable to the optical system 132 and optical components 136 described herein.
[0032] Continuing with reference to FIG. 1 , as used herein, an “optical sensor” refers to a device that measures light and converts the measured light into one or more signals, which may include, but are not limited to, one or more electrical signals. In some embodiments, optical sensor 140 may include at least one photodetector. As used herein, a “photodetector” refers to a device that is sensitive to light and can thereby detect light. In some embodiments, a photodetector may include a photodiode, a photoresistor, a photosensor, a photovoltaic chip, or the like. In some embodiments, an optical sensor may include multiple photodetectors. Optical sensor 140 may include, but is not limited to, a camera. The optical sensor may communicate electronically with computing device 148, as described in more detail below. “Electronic communication,” as used in this disclosure, refers to a shared data connection between two or more devices. In some embodiments, optical system 132 may include two or more optical sensors. In some cases, optical sensor 140 may be positioned adjacent to optics 136. In a non-limiting example, optical sensor 140 may be attached to a nosepiece, as described above. In another non-limiting example, optical sensor 140 may be located within the head portion of imaging device 116 above optics 136 .
[0033] Still referring to FIG. 1 , in some embodiments, the at least one optical sensor 140 may include a camera. In some cases, the camera may include one or more optical systems as described herein, such as, but not limited to, a spherical lens, an aspherical lens, a reflecting mirror, a polarizer, a filter, a window, an aperture stop, etc. In some embodiments, one or more optical systems associated with the camera may be adjusted to change the camera's zoom, depth of field, and / or focal length, in non-limiting examples. In some embodiments, one or more of such settings may be configured to detect features of a specimen on the slide 104. In some embodiments, one or more of such settings may be configured based on any set of parameters (i.e., a set of values, such as, but not limited to, quantitative and / or numerical values, that identify how an image is captured), such as those disclosed in U.S. Patent Application Publication No. 18 / 226,058. In some embodiments, the camera may capture images with a shallow depth of field.
[0034] 1 , system 100 includes a computing device 148. In one or more embodiments, system 100 and / or computing device 148 may include an application-specific integrated circuit (ASIC) communicatively coupled to a memory. The memory may include read-only memory (ROM) and / or re-programmable ROM, FPGA, or other combinational and / or synchronous sequential or asynchronous digital circuitry for storing parameters described further in this disclosure. In one or more embodiments, the memory may include one or more memory devices that store data and information, such as parameters or metrics, as described herein. The one or more memory devices may include various types of memory, including, but not limited to, volatile and non-volatile memory devices such as ROM, EEPROM, random access memory (RAM), flash memory, and the like.
[0035] Still referring to FIG. 1 , in one or more embodiments, computing device 148 may include a processor adapted to execute software stored in memory to perform the various methods, processes, and modes of operation as described herein. In other embodiments, system 100 may include circuitry. For example, without limitation, system 100 may include programming in software and / or hardware circuitry. Computing device 148 may include or be included in any computing device described herein, including, but not limited to, a microcontroller, microprocessor, digital signal processor (DSP), and / or system-on-chip (SoC) described herein. Computing device 148 may include, be included in, and / or communicate with a mobile device, such as a mobile phone or smartphone. Computing device 148 may include a single computing device operating independently, or may include two or more computing devices operating cooperatively, in parallel, sequentially, etc., where the two or more computing devices may be included together in a single computing device or in two or more computing devices.
[0036] Still referring to FIG. 1, the computing device 148 can interface or communicate with one or more additional devices via a network interface device, as described in further detail below. The network interface device may be utilized to connect the processor to one or more of various networks and one or more devices. Examples of network interface devices include, but are not limited to, a network interface card (e.g., a mobile network interface card, a LAN card), a modem, and any combination thereof. Examples of networks include, but are not limited to, a wide area network (e.g., the Internet, an enterprise network), a local area network (e.g., a network associated with an office, building, campus, or other relatively small geographic space), a telephone network, a data network associated with a telephone / voice provider (e.g., a mobile communications provider's data and / or voice network), a direct connection between two computing devices, and any combination thereof. The network can employ wired and / or wireless communication modes. In general, any network topology may be used. Information (e.g., data, software, etc.) may be communicated to and / or from computers and / or computing devices.
[0037] Continuing to refer to FIG. 1 , computing device 148 may include, for example, but not limited to, a computing device or cluster of computing devices at a first location and a second computing device or cluster of computing devices at a second location. Computing device 148 may include one or more computing devices specialized for data storage, security, traffic distribution for load balancing, etc. Computing device 148 may distribute one or more computing tasks, as described below, across multiple processors of the computing device, which may operate in parallel, serial, redundant, or any other manner used to distribute tasks or memory among computing devices. Computing device 148 may be implemented using a “shared-nothing” architecture in which data is cached at the workers, which, in one embodiment, may enable scalability of system 100 and / or computing device 148.
[0038] Continuing to refer to FIG. 1 , computing device 148 may be designed and / or configured to repeatedly perform any method, method step, or sequence of method steps in any embodiment described in this disclosure, in any order, and to any degree. For example, computing device 148 may be configured to repeatedly perform a single step or sequence until a desired or ordered result is achieved. The repetition of a step or sequence of steps may be performed iteratively and / or recursively using the output of a previous iteration as input for a subsequent iteration, aggregating the input and / or output of an iteration to produce an aggregate result, decreasing or decrementing one or more variables, such as global variables, and / or dividing a large processing task into a set of smaller processing tasks that are addressed iteratively. Computing device 148 may perform any step or sequence of steps described in this disclosure in parallel, such as performing a step two or more times simultaneously and / or nearly simultaneously, using two or more parallel threads, processor cores, etc., and the division of tasks among parallel threads and / or processes may be performed according to any protocol suitable for dividing tasks among iterations. Those skilled in the art will recognize, upon reviewing this disclosure in its entirety, various ways in which steps, sequences of steps, processing tasks, and / or data may be subdivided, shared, or otherwise processed using iterative, recursive, and / or parallel processing.
[0039] Continuing with reference to FIG. 1 , as used in this disclosure, “communicatively connected” means connected by a connection, attachment, or coupling that allows for the receipt and / or transmission of information between two or more entities. For example, but not limited to, this connection may be wired or wireless, direct or indirect, and between two or more components, circuits, devices, systems, imaging devices, etc., allowing for the reception and / or transmission of data and / or signals. The data and / or signals therebetween may include, but are not limited to, electrical, electromagnetic, magnetic, visual, audio, radio and microwave data and / or signals, combinations thereof, and the like, among others. A communicative connection may be achieved, for example, but not limited to, by wired or wireless electronic, digital, or analog communication, directly or through one or more intervening devices or components. Furthermore, a communicative connection may include electrically coupling or connecting at least one output of one device, component, or circuit to at least one input of another device, component, or circuit, for example, but not limited to, via a bus or other facility for intercommunication between computing device elements. A communicative connection may also include an indirect connection via, for example, but not limited to, a wireless connection, a wireless communication, a low-power wide area network, optical communication, magnetic coupling, capacitive coupling, optical coupling, etc. In some instances, the term "communicatively coupled" may be used in this disclosure instead of communicatively connected.
[0040] Still referring to FIG. 1 , a “signal,” as used in this disclosure, refers to any understandable representation of data, for example, from one device to another. Signals may include optical signals, hydraulic signals, pneumatic signals, mechanical signals, electrical signals, digital signals, analog signals, etc. In one or more embodiments, image data of a slide image may be transmitted through one or more signals. In other embodiments, instructions from a human operator of the imaging device 116 may send commands to components of the imaging device 116, such as the optical system 128 and the optical components 132a-132d, through one or more signals. In some cases, signals may be used to communicate with the computing device 148, for example, through one or more ports. In some cases, signals may be transmitted and / or received by the computing device 148, for example, through an input / output port. Analog signals may be digitized, for example, by an analog-to-digital converter. In some cases, analog signals may be processed, for example, with the analog signal processing steps described in this disclosure, before being digitized. In some cases, digital signals may be used to communicate between the imaging device 116 and the computing device 148. In some cases, the digital signal containing the image data of the captured slide image may be communicated via one or more communication protocols, including, but not limited to, Internet Protocol (IP), Controller Area Network (CAN) protocol, serial communication protocol (e.g., Universal Asynchronous Receiver Transmitter [UART]), parallel communication protocol (e.g., IEEE [printer port]), etc.
[0041] Still referring to FIG. 1 , in some embodiments, computing device 148 can perform one or more signal processing steps on a signal. For example, computing device 148 can analyze, modify, and / or synthesize a signal representing data to improve the signal, for example, by improving transmission, storage efficiency, or signal-to-noise ratio. Exemplary methods of signal processing can include analog, continuous-time, discrete, digital, nonlinear, statistical, etc. Analog signal processing can be performed on non-digitized or analog signals. Exemplary analog processing can include passive filters, active filters, summing mixers, integrators, delay lines, companders, multipliers, voltage-controlled filters, voltage-controlled oscillators, phase-locked loops, etc. Continuous-time signal processing can, in some cases, be used to process signals that vary continuously within a domain, e.g., the time domain. Exemplary, non-limiting continuous-time processes can include time-domain processing, frequency-domain processing (Fourier transform), and complex frequency-domain processing. Discrete-time signal processing can be used when a signal is sampled at non-continuous or discrete time intervals (i.e., quantized in time). Analog discrete time signal processing can process signals using the following exemplary circuits: sample and hold circuits, analog time division multiplexers, analog delay lines, analog feedback shift registers.
[0042] Still referring to FIG. 1 , in some embodiments, digital signal processing may be used by the computing device 148 to process digitized discrete-time sampled signals. Generally, digital signal processing may be performed by the computing device 148 or other specialized digital circuitry, such as, but not limited to, an ASIC, FPGA, or DSP. Digital signal processing may be used to perform any combination of typical arithmetic operations, such as fixed-point, floating-point, real-valued, complex-valued, multiplication, addition, etc. Digital signal processing may also operate circular buffers and look-up tables. Further non-limiting examples of algorithms that may be implemented according to digital signal processing techniques include fast Fourier transforms (FFTs), finite impulse response (FIR) filters, infinite impulse response (IIR) filters, and adaptive filters such as Wiener filters and Kalman filters. Statistical signal processing may be used to exploit statistical properties to process signals as random functions (i.e., stochastic processes). For example, in some embodiments, signals may be modeled with probability distributions representative of noise, which may be used to reduce noise in the processed signals.
[0043] Continuing with reference to FIG. 1 , computing device 148 is configured to scan each slide of the plurality of slides 112 using imaging device 116 as described above to generate an initial slide image 152. In some cases, optical sensor 140 may be utilized for such purposes. As used in this disclosure, a “slide image” refers to a digital representation or photograph of the contents of slide 104 (e.g., the specimen contained therein). In one embodiment, once a slide is placed on slide port and / or stage 120, a view may be generated by optical system 132 using one or more optical components 136 and captured by optical sensor 140, such as a camera, or other imaging system within imaging device 116. A digital image of such a view (i.e., slide image) may represent the visual contents of slide 104 at a particular magnification and resolution. In some cases, the slide image may include a whole slide image (WSI) of slide 104. In other cases, the slide image may include only a portion of the WSI.
[0044] Still referring to FIG. 1 , in one or more embodiments, a slide image may include image data, where “image data” as used herein refers to information representing at least one physical scene, space, and / or object. Image data may include, for example, information representing a specimen, a slide 104, or even a region of a specimen or slide 104. In some cases, the image data may be generated by a camera, such as any of the cameras described herein. “Image data” may be used interchangeably with “image” throughout this disclosure, and image is used as a noun. In some cases, a digital image may be represented as a bitmap. In other cases, a slide image may be composed of any medium capable of representing a physical scene, space, or object. Alternatively, when “image” is used as a verb in this disclosure, it refers to the generation and / or formation of an image.
[0045] Still referring to FIG. 1 , in some embodiments, a slide image may include one or more digital files in a format such as, but not limited to, JPEG, TIFF, PNG, or any other specialized digital format. In some embodiments, a slide image may be captured at a particular magnification level. In some embodiments, a slide image may include annotations, such as, but not limited to, one or more markers, labels, or notes to highlight particular regions of interest or features. In some embodiments, a slide image may also include metadata. In some cases, the metadata may include, but is not limited to, information related to one or more optical component magnification levels, imaging data (e.g., image resolution, image color model, parameter sets, etc.), specimen details (e.g., patient information), and other relevant data.
[0046] Still referring to FIG. 1 , in some embodiments, the imaging device 116 may be configured to generate an initial slide image 152 of the slide 104 as a Z-stack (i.e., a series of slide images captured at different focal planes along the Z-axis (vertical axis) of the specimen on the slide 104). A Z-range, which is the upper and lower focal planes or range of interest of the specimen, may be identified, and the spacing, or step size, between each slide image in the Z-stack may be determined. In some cases, the Z-range and step size may be manually entered by a human operator or may be provided by a parameter set, as described above. In other cases, the Z-range and step size may be determined by the computing device 148, as described in more detail below. In some cases, a smaller step size allows for a more detailed 3D reconstruction, but results in a larger number of slide images in the region of interest. The imaging device 116 may be configured to capture a slide image at each focal plane throughout the Z-range and compile them into a single slide image comprising the 3D reconstruction of the specimen.
[0047] Still referring to FIG. 1 , in a non-limiting example, the initial slide image 152 may include a “merged image,” which, for purposes of this disclosure, is a processed multi-layer scan composed of multiple slide images taken at various focal depths as described above. In one or more embodiments, the initial slide image 152 may include a multi-layer scan having multiple overlaid slide images. For example, without limitation, multiple slide images captured at a location x may be merged to create a first section of the initial slide image 152. In one or more embodiments, the merged image may include an extended field of depth (EFOD) image. In some cases, the initial slide image 152 may be captured by the imaging device 116 using a specimen image generation method such as that described in U.S. Patent Application No. 18 / 226,058. In some embodiments, the at least one optical sensor 140 may include an image sensor. Exemplary, non-limiting image sensors include digital image sensors, such as, but not limited to, charge-coupled device (CCD) sensors and complementary metal-oxide semiconductor (CMOS) sensors. In some embodiments, the optical sensor 140 may be sensitive in the non-visible range of electromagnetic radiation, such as, but not limited to, infrared.
[0048] Still referring to FIG. 1 , in a non-limiting example, metadata such as a plurality of scanning metadata 156 associated with the initial slide image 152 is also generated by the computing device 148 along with the initial slide image 152. As used in this disclosure, “scanning metadata” refers to supplemental information generated or recorded during the scanning process for each slide of the plurality of slides 112, as described above. In some embodiments, the scanning metadata 156 may provide context, details, and parameters related to the scanning process and the scanned slide. In some cases, the scanning metadata 156 associated with the initial slide image 152 may include one or more timestamps (e.g., the exact date and time the slide was scanned). In a non-limiting example, the plurality of slides 112 may be stored in chronological order within at least one slide storage unit 108 after being scanned. In some cases, the scanning metadata 156 may include a string describing a scanner identifier (ID). For example, details about the imaging device used, including, but not limited to, the model number, manufacturer, and / or any unique identification number, may be encoded and incorporated into the scanning metadata 156.
[0049] Still referring to FIG. 1 , in some cases, scanning metadata 156 associated with initial slide image 152 may include data such as image resolution (i.e., the pixel density of the slide image, measured in dots per inch [DPI]). Image resolution may indicate the level of detail captured in the associated slide image. In some cases, scanning metadata 156 may include bit depth (i.e., the number of bits of information stored for each pixel); for example, without limitation, the range of colors or shades of gray that can be represented may be determined by computing device 148 as a function of bit depth. Additionally or alternatively, in some cases, scanning metadata may include a color profile (i.e., the color space used), such as, but not limited to, sRGB, RGB, RYB, CMY, CMYK, HSL, HSV, etc.
[0050] 1 , scanning metadata 156 associated with initial slide image 152 may include data related to compression details (e.g., details of the compression algorithm, compression rate, etc.). In some cases, computing device 148 may be configured to compress the slide image generated during scanning of each slide of plurality of slides 112. In one embodiment, computing device 148 may encode initial slide image 152 to reduce file size and storage requirements while maintaining essential visual information necessary for further processing steps, as described below. In one embodiment, compressing and / or encoding the slide image may facilitate fast transmission of the image. In some cases, computing device 148 may implement one or more lossless compression algorithms (i.e., preserving the original image quality of the slide image), such as Huffman coding, Lempel-Ziv-Welch (LZW), or Run-Length Encoding (RLE), to identify and remove redundancy in each slide image without losing information. In one embodiment, compressing and / or encoding each slide image may include converting the file format of each slide image to PNG, GIF, lossless JPEG2000, or the like. In other cases, one or more lossy compression algorithms, such as, but not limited to, JPEG's Discrete Cosine Transform (DCT) or JPEG2000's Wavelet Transform, may be implemented by computing device 148 to compress and / or encode each slide image with a high compression ratio but lower image quality (i.e., discarding less important information in each slide image, resulting in a smaller file size).
[0051] Still referring to FIG. 1 , in some cases, scanning metadata 156 associated with initial slide image 152 may include a value representing a scan mode (e.g., grayscale, color, or any other mode). In some cases, scanning metadata 156 may also include image settings (e.g., exposure settings, including details regarding exposure during the scanning process, which may affect the brightness and contrast of the resulting initial slide image 152). In some cases, scanning metadata 156 may include imaging device settings. In one embodiment, scanning metadata 156 may include one or more focus parameters. In non-limiting examples, information regarding focus settings may include focal depth, Z-stack information, focus offset, lens specifications, correction data, etc. In another embodiment, scanning metadata 156 may include a magnification level (e.g., the level of magnification used during the scan). In further embodiments, scan duration (i.e., the time required to complete slide 104 or the previous slide), if applicable, may be determined and / or recorded. Scanning metadata 156, including scan duration, may indicate the depth or quality of the selected imaging device.
[0052] Continuing with reference to FIG. 1 , in some embodiments, scanning metadata 156 associated with initial slide image 152 may include post-processing information. In non-limiting examples, any corrections or enhancements may be made to the slide image after it is generated, such as brightness adjustments, contrast enhancements, noise reduction, etc. In some cases, scanning metadata includes a slide label or slide identifier; for example, a unique identifier or label associated with the scanned slide 104 may be incorporated into scanning metadata 156 to aid in cataloging and retrieval, as described herein. Additionally or alternatively, operator details, such as, but not limited to, information related to a human operator or system administrator responsible for any processing steps as described herein, may be included in scanning metadata 156 for accountability and quality control purposes. In some cases, scanning metadata 156 may include calibration data (i.e., information regarding a calibration performed on the imaging device prior to slide scanning). In other cases, scanning metadata 156 may further include environmental condition data (i.e., details about the surrounding environment during scanning), such as, but not limited to, temperature levels, humidity levels, etc.
[0053] Still referring to FIG. 1 , in a further embodiment, the scanning metadata 156 associated with the initial slide image 152 may include an error log. In some cases, the scanning metadata 156 may include errors or problems encountered during the scanning process, along with potential solutions. In a non-limiting example, the scanning metadata 156 may include a contaminant presence indicator, where a “contaminant presence indicator” as described herein is a metric, signal, or set of data points derived from the generated initial slide image that suggests or confirms the presence of contamination (in the optical path of the imaging device 116) caused by one or more contaminants. In some cases, contaminants (i.e., any debris, foreign matter, particle, or material that potentially interferes with the clarity, quality, or accuracy of the initial slide image 152) may include, but are not limited to, dust from the environment, residue from cleaning fluids, oils from human contact, or even specimens such as cells or tissue inadvertently deposited on the slide 104, stage 120, or any optical component 136 of the optical system 132 of the imaging device 116.
[0054] 1 , in a non-limiting example, the contaminant presence indicator may include a binary output, for example, “TRUE / POST” (i.e., POSITIVE) for “contaminant present” or “FALSE / NEG” (i.e., NEGATIVE) for “contaminant not present.” In some cases, the scanning metadata 156 may include a contaminant presence indicator and / or a contaminant cleaning protocol (i.e., a predefined series of steps or actions designed to clean, remove, or mitigate contaminants, as described above), where the contaminant presence indicator and contaminant cleaning protocol may be consistent with the contaminant presence indicator and contaminant cleaning protocol as described in U.S. Patent Application No. 18 / 382,345 (Attorney Docket No. 1519-105USU1), filed October 20, 2023, entitled “SYSTEM AND METHODS FOR DETECTING AND CLEANING CONTAMINANTS FROM AN IMAGING OPTICAL PATH,” which is incorporated herein by reference in its entirety.
[0055] Continuing with reference to FIG. 1 , computing device 148 is configured to determine quality control metrics 160 for each slide of plurality of slides 112 as a function of scanning metadata 156 associated with initial slide image 148. As used in this disclosure, "quality control metrics" refers to a standardized measure or set of measures used to evaluate and ensure the quality of generated slide images. In one embodiment, quality control metrics 160 may include an assessment of the clarity, accuracy, and overall quality of initial slide image 152. Those skilled in the art will recognize, upon reviewing this disclosure in its entirety, the importance of ensuring digitized slides of the highest possible quality suitable for research, diagnostic, archival, or other intended purposes. In a non-limiting example, quality control metrics 160 may be used to determine whether initial slide image 152 meets predefined criteria and whether corrective action or adjustment is required, as described in more detail below.
[0056] 1 , in some cases, quality control metric 160 may include a numerical value representing an overall score for initial slide image 152. The overall score may be derived from various individual metrics, such as, but not limited to, sharpness, contrast ratio, noise level, and color accuracy, as described below. In some cases, such a numerical value may be compared to a threshold value to determine whether initial slide image 152 meets desired quality standards. In other cases, quality control metric 160 may include a categorical rating, such as “Excellent,” “Good,” “Fair,” or “Poor,” based on a combination of individual metrics determined based on initial slide image 152 and assigned to the corresponding slide 104.
[0057] Continuing with reference to FIG. 1 , in some embodiments, the quality control metric 160 may be determined by the computing device 148 based on the image sharpness of the initial slide image 152, which measures the sharpness and crispness of the initial slide image 152. In a non-limiting example, a higher sharpness value for the initial slide image 152 may indicate a clearer image with more detail, leading to a higher score. In some embodiments, the quality control metric 160 may be determined as a function of the contrast ratio of the initial slide image 152. In some cases, the contrast ratio (e.g., contrast-to-noise ratio [CNR] or signal-to-noise ratio [SNR]) may assess the difference in brightness or color between the lightest and darkest areas of the initial slide image 152. In some embodiments, the quality control metric 160 may be determined based on the color accuracy of the initial slide image 152 (i.e., an assessment of how accurately the initial slide image 152 reproduces the colors of the original slide 104). Other exemplary embodiments of slide image metrics may include, but are not limited to, slide image resolution, image noise level, brightness / color uniformity, dynamic range, and the like.
[0058] Still referring to FIG. 1 , in some cases, the quality control metric 160 for each slide of the plurality of slides 112 may be determined based on the quality of focus (of the imaging device 116). In some cases, the scanning metadata 156 associated with the initial slide image 152 may provide one or more data points related to the quality of focus. In some embodiments, the quality control metric 160 may be determined based on the depth of field (DoF). In a non-limiting example, a larger DoF indicates that more of the specimen is simultaneously in focus. In some cases, determining the quality control metric 160 may include generating a depth map (i.e., an image or image channel containing information related to the surface distance from the viewpoint to a scene object, such as a specimen contained in slide 104). In some embodiments, information related to chromatic aberration, which occurs when different wavelengths of light emitted by the light source 144 are focused at slightly different positions, may be used to determine the quality control metric 160, as described herein. In a non-limiting example, the quality control metric 160 may include a higher score if chromatic aberration is minimized (i.e., color fringing is minimal). In other embodiments, the quality control metric 160 may be determined based on a planarity score, where, for purposes of this disclosure, a "planarity score" refers to a value or metric that quantifies the flatness or evenness of a surface (e.g., the surface of the slide 104, the surface of the stage 120, the surface of the slide port, etc.) derived from the initial slide image 152. In a non-limiting example, the planarity score and methods for calculating the planarity score may be described in U.S. Patent Application No. 18 / 382,345, as noted above. While ideally, the entire field of view is in focus, the center of the initial slide image 152 may be in focus while the edges are blurred, or vice versa. In some cases, the quality control metric 160 may include a higher score if the initial slide image 152 has uniform focus across the entire field of view.
[0059] Still referring to FIG. 1 , in one embodiment, the quality control metric 160 may be determined based on the degree of depiction quality of a region of interest in the initial slide image 152. In some cases, the region of interest may include a region of the slide image containing the most specimen-related detail. The computing device 148 may determine the degree of image blur using one or more image processing techniques, such as those described herein. In a non-limiting example, the computing device 148 may perform blur detection by taking a Fourier transform, or an approximation such as a fast Fourier transform (FFT), of the image and analyzing the distribution of low and high frequencies in the resulting frequency-domain depiction of multiple slide images. The number of high-frequency values below a threshold level may indicate blur. In another non-limiting example, blur detection may be performed by convolving the slide image, a channel of the slide image, or the like, with a Laplacian kernel, and for example, but not by way of limitation, the computing device 148 may generate a numerical score (i.e., the quality control metric 160) reflecting the number of abrupt changes in intensity present in each slide image, with a high score indicating sharpness and a low score indicating blur. In some cases, blur detection can be performed using a gradient-based operator that measures an operator based on the gradient or first derivative of the image, with the hypothesis that abrupt changes indicate sharp edges in the initial slide image 152 and therefore a low degree of blur. In some cases, blur detection may be performed using a wavelet-based operator that exploits the ability of discrete wavelet transform coefficients to describe the frequency and spatial content of an image. In some cases, blur detection may be performed using a statistics-based operator that utilizes some image statistics as texture descriptors to calculate the focus level. In other cases, blur detection may be performed using discrete cosine transform (DCT) coefficients to calculate the focus level of the image from its frequency content. Additionally or alternatively, the computing device 148 may be configured to rank the slide images for the plurality of slides 112 according to their degree of depiction quality.
[0060] Still referring to FIG. 1 , additionally or alternatively, quality control metrics 160 may be determined based on artifact detection; for example, without limitation, computing device 148 may be configured to analyze initial slide image 152 for anomalies, inconsistencies, or patterns that are not expected to be part of the actual specimen or slide 104. The amount of contaminants (e.g., unwanted particles) may be used to calculate a score. In some cases, scratches on the slide and / or inspection surface may also be detected by computing device 148 and used in determining quality control metrics 160. Advanced image analysis algorithms, such as, but not limited to, morphological operations (such as dilation, erosion, opening, or closing), Fourier transforms, scale-invariant feature transforms (SIFT), image segmentation, edge detection (such as Sobel, Canny, or Prewitt operators), may be used for artifact detection as described herein. In some cases, one or more machine learning processes, as described in more detail below, may be incorporated into the analysis of the initial slide images 152, such as, but not limited to, a convolutional neural network (CNN), a support vector machine (SVM), a random forest, or the like, may be implemented by the computing device 148 to determine a quality control metric 160 for each slide of the plurality of slides 112. In a non-limiting example, determining the quality control metric 160 may include determining the quality control metric as a function of a contaminant presence indicator, such as described in U.S. Patent Application No. 18 / 382,345.
[0061] 1, determining the quality control metric 160 may include combining the individual metrics and / or scores, as described above, into a single composite score. In some cases, the single composite score may be generated using a weighted average, with weights determined by the computing device 148 according to the importance of each metric. In some cases, the weights may be predetermined manually, for example, by a human operator.
[0062] Still referring to FIG. 1 , determining the quality control metrics 160 may additionally or alternatively include verifying scanning metadata 156 associated with the initial slide image 152. Each slide of the plurality of slides 112 is scanned, and scanning metadata 156, such as, but not limited to, magnification level, lighting settings, focus parameters, and scan timestamp, may be verified to ensure the slide image is generated as intended. In a non-limiting example, light intensity and light source type may be checked to ensure the specimen is properly illuminated, resulting in a clear, accurate slide image with the desired quality control metrics. For specimens that change over time, such as living cells, the scan timestamp may be verified against an optimal scan time. If the scan timestamp associated with the initial slide image 152 significantly deviates from the optimal scan time, the quality control metrics may be determined and / or adjusted accordingly. For example, but not limited to, if cells are best imaged during mitosis and the scan timestamp indicates a post-mitotic scan, the quality control metrics may be lowered to reflect the reduced biological relevance of the generated slide image.
[0063] Continuing with reference to FIG. 1 , in some embodiments, computing device 148 may be configured to generate quality control metadata as a function of quality control metrics 160. As used in this disclosure, “quality control metadata” refers to a set of data providing information about the quality control metrics of a scanned slide image (e.g., initial slide image 152). In some cases, the quality control metadata may be derived from an evaluation of one or more quality control metrics 160, as described above. In non-limiting examples, the quality control metadata may include the score or value of the quality control metric 160, the reason for any deviation from a predetermined threshold, specific areas of the initial slide image 152 flagged for corresponding quality issues, recommendations or actions to improve image quality, etc. In some cases, the quality control metadata may serve as a comprehensive record / profile of the quality assessment process performed by computing device 148, enabling traceability, review, and further analysis, as described in more detail below.
[0064] 1 , in a non-limiting example, for a given set of quality control metrics 160 assessing the sharpness, focus, color accuracy, etc., of an initial slide image 152, computing device 148 may generate quality control metadata as a function of the set of quality control metrics 160 detailing a particular value or score for each of these parameters. In some cases, deviations from expected values and potential reasons for such deviations may be included in the generated quality control metadata. For example, without limitation, the quality control metadata may include one or more deviations indicative of "dust on slide," and corresponding potential causes of the deviations (e.g., "misalignment of the slide on the stage") may also be included in the quality control metadata, as described herein.
[0065] Continuing with reference to FIG. 1 , in some embodiments, determining quality control metrics 160 may also include modifying the generated quality control metadata by computing device 148 as a function of user input 164 received from a user. As used in this disclosure, “user input” refers to feedback, corrections, annotations, or any other form of input provided by a user regarding quality control metrics 160, initial slide image 152, or slide 104 itself. In some cases, a user may include a human operator. In other cases, a user may include a system administrator. In one embodiment, user input 164 may be in the form of manual adjustment of quality control metrics 160, annotation of specific areas of initial slide image 152, comments on recognized errors, suggestions for recalibration or rescanning, etc. In another embodiment, user input 164 may include adjustments to optical component selections (e.g., objective lens, focus parameters of the selected objective lens, position of the slide on stage 120 and / or slide port, position of stage 120, etc.). In other embodiments, user input 164 may include manual cleaning of devices within system 100.
[0066] Still referring to FIG. 1 , in some cases, computing device 148 may include a feedback mechanism, and user input 164 may incorporate human expertise and / or judgments entered by a human operator into the quality assessment process of initial slide image 152, as described herein, such that quality control metrics 160 and quality control metadata rely not only on automated assessment but also on human knowledge. In a non-limiting example, a human operator and / or computing device 148 may notice / detect, based on quality control metrics 164, that a region of initial slide image 152 may appear overexposed, resulting in a loss of detail in that region. Upon reviewing the associated scanning metadata 156, the operator may input user input to adjust the exposure time parameter of imaging device 116 to a shorter time to prevent overexposure. In some cases, computing device 148 may be configured to modify the quality control metadata to reflect that the overexposure was due to incorrect scanning parameters and not a specimen-specific issue. In some cases, user input may be implemented in subsequent scans, as described in more detail below.
[0067] Continuing with reference to FIG. 1 , in some cases, user input 164 may be entered through a user interface of a display device communicatively coupled to computing device 148. In some cases, computing device 148 may request, alert, or instruct the user to provide user input 164 through the user interface. As used in this disclosure, a “display device” is a device used to display content. As used herein, a “user interface” is a means by which a user interacts with a computer system, for example, using input devices and software. A user interface may include a graphical user interface (GUI), a command line interface (CLI), a menu-driven user interface, a touch user interface, a voice user interface (VUI), a form-based user interface, any combination thereof, or the like. A user interface may include a smartphone, smart tablet, desktop, or laptop operated by a user. In one embodiment, a user interface may include a graphical user interface, and as used herein, a “graphical user interface (GUI)” is a graphical form of a user interface that allows a user to interact with an electronic device. A user interface may include an output interface and an input interface. In some embodiments, the output interface may include one or more elements that allow at least one slide storage unit 108, imaging device 116, and / or computing device 148 to communicate information to a human operator. In some embodiments, user input 164 may be entered through an input interface, such as, but not limited to, a mouse, keyboard, touch screen, etc. In a non-limiting example, the output interface may include a display (e.g., a high-resolution display). In some cases, the display may output slide images, associated scanning metadata 156, quality control metrics / metadata, etc. to the user.In a non-limiting example, the display device may display a notification on the user interface indicating the need for user input 164, and the notification may provide details regarding quality control metrics along with quality control metadata generated for the currently processed slide. Additionally or alternatively, the computing device 148 may be integrated with other output systems or devices, such as a speaker (e.g., an alarm), a light source (e.g., a visual indicator), or a tactile feedback mechanism.
[0068] Continuing with reference to FIG. 1 , determining quality control metrics 160 includes flagging slides based on quality control metrics 160, as described above. As used in this disclosure, “flagging” refers to marking or identifying slides that do not meet certain predefined quality criteria based on quality control metrics 160. In some cases, computing device 148 may be configured to highlight slides that may deviate from established quality benchmarks or thresholds. In some embodiments, computing device 148 may be configured to check each quality control metric 160 and / or generated quality control metadata against a database of predefined acceptable thresholds. In some cases, the predefined acceptable thresholds in such a database may be calibrated based on empirical data, industry standards, user-input criteria, etc. In a non-limiting example, each slide image of a plurality of slide images scanned using imaging device 116 for a plurality of slides 112 may be evaluated for pixel intensity distribution, sharpness index, color histogram balance, etc. In some cases, a slide may be flagged by computing device 148 if its sharpness index falls below a specified value, indicating a potential focusing problem, or if its color histogram shows an abnormal distribution, suggesting uneven staining or illumination. Other Exemplary Embodiments
[0069] Still referring to FIG. 1 , in some cases, slides 104 may be flagged if there is a significant deviation in expected scanning parameters, such as, for example, but not limited to, variations in scan speed, which may be due to hardware malfunction or calibration errors in at least one of slide storage 108, transport component 128, imaging device 116, etc. In some cases, if a slide is flagged, it may be manually reviewed by a human operator. In other cases, an automated rescan procedure may be performed on the flagged slide by computing device 148 to correct issues identified based on quality control metrics 160 and / or generated quality control metadata, as described in further detail below. In some cases, scanning metadata 156 associated with flagged slides may be stored in a separate database or directory. In some cases, flagged slides may be transported from imaging device 116 to a second slide storage component using transport component 128 and stored for easy access for subsequent intervention.
[0070] Still referring to FIG. 1 , initial slide image 152, associated scanning metadata 156, quality control metrics 160, quality control metadata, and any data described herein may be stored, for example, in memory of computing device 148 or in database 168. Database 168 may be implemented as, without limitation, a relational database, a key-value database such as a NOSQL database, or any other format or structure for use as a database that one skilled in the art would recognize as appropriate upon reviewing this disclosure in its entirety. Database 168 may alternatively or additionally be implemented using a distributed data storage protocol and / or data structure, such as a distributed hash table. Database 168 may include multiple data entries and / or records, as described above. Data entries in the database may be flagged or linked to one or more additional information elements, which may be reflected in data entry cells and / or in linked tables, such as tables related by one or more indexes in a relational database. Those skilled in the art will recognize upon reviewing this disclosure in its entirety the various ways in which data entries in a database may store, search, organize, and / or reflect the data and / or records, and categories and / or populations of data, used herein, consistent with this disclosure.
[0071] Continuing to refer to FIG. 1, in some cases, the computing device 148 may include: One or more machine learning models may be used to determine the quality control metrics 160. In some embodiments, a computing device may utilize a machine learning module to implement one or more machine learning algorithms and generate one or more machine learning models, as described in more detail below with reference to FIG. 7. However, the machine learning module is exemplary and may not be required to generate machine learning models or perform machine learning tasks as described herein. In one or more embodiments, one or more machine learning models, such as, but not limited to, a slide image classifier, may be generated using training data. The training data may include inputs and corresponding predetermined outputs such that the machine learning model can use the correlations between the provided exemplary inputs and outputs to develop algorithms and / or relationships and determine its own output for the inputs. The training data may include correlations that the machine learning process uses to model relationships between two or more categories of data elements.
[0072] Still referring to FIG. 1 , in some cases, exemplary inputs and outputs of the training data may be obtained from database 168, as described above, or may be provided by a user (e.g., a human operator). In other cases, the machine learning modules described herein may obtain a training set by querying a communicatively connected database containing past inputs and outputs. The training data may include inputs from various types of databases, resources, and / or user inputs, and outputs correlated to each of those inputs, so that the machine learning model can determine an output. The correlations may indicate causal or predictive relationships between the data and may be modeled by the machine learning model as relationships, such as mathematical relationships. In one or more embodiments, the training data may be formatted and / or organized by categories of data elements, for example, by associating the data elements with one or more descriptors that correspond to the categories of the data elements. As a non-limiting example, the training data may include data entered into a standardized form by a person or process, such that the entry of a given data element in a given field of the form maps to one or more descriptors of the category. Elements of the training data may be linked to categories by tags, tokens, or other data elements.
[0073] Still referring to FIG. 1 , in some embodiments, determining quality control metrics 160 may include training a slide image classifier using slide image training data, which may include a plurality of slide images as inputs correlated with a plurality of quality control metrics as outputs, and using the trained slide image classifier to classify initial slide images 152 and associated scanning metadata 156 into scanning states 172. The slide image classifier may be generated using one or more classification algorithms, as described below with reference to FIG. 4 . In a non-limiting example, the slide image classifier may include a CNN, as described below with reference to FIG. 5 . In some cases, the slide image training data may include a plurality of slide images as inputs, each correlated with a scanning state 172 as an output. As used in this disclosure, “scanning state” refers to a designated categorization or state assigned to a slide after being scanned by imaging device 116, as described above. In some cases, scanning state 172 may provide a concise representation of the quality of a scanned slide image (e.g., initial slide image 152). In some cases, scanning status 172 may include a value representing the completion of the scanning process for each slide of the plurality of slides 112. In some embodiments, scanning status 172 may include an indicator of whether the scanned slide meets predefined quality criteria, as described above, or whether further action, such as rescanning, is required. In some embodiments, slides may be flagged based on scanning status 172.
[0074] 1 , in a non-limiting example, the scanning status 172 may include binary scanning states, which may include a first state (e.g., “Scan Complete”) indicating that the slide 104 was scanned successfully, the quality control metrics 160 of the resulting image, i.e., the initial slide image 152, meet all predefined quality benchmarks, such as, but not limited to, optimal sharpness, proper color balance, and the associated metadata 156 of the initial slide image 152 can confirm that there were no hardware or software anomalies during the scanning process. Meanwhile, the binary scanning status may include a second state, e.g., “Re-scan Candidate,” indicating that the initial slide image 152 of the slide 104 does not meet the quality standards and should be re-scanned. In some cases, reasons for a slide being flagged as a "rescan candidate" may include hardware failure (e.g., temporary device malfunction) (in the slide storage unit 108, transport component 128, imaging device 116, and / or computing device 148), misalignment of the slide or stage 120, improper scanning (focusing) parameters, contamination on the slide, etc.
[0075] 1, in a non-limiting example, if a slide 104 is scanned and the resulting initial slide image 152 exhibits a sharpness value above a certain threshold and the associated scanning metadata 156 confirms a consistent scan speed and proper lighting, the slide 104 may be categorized as having a "scanned" status. Conversely, if the initial slide image 152 shows signs of blur, or if the associated scanning metadata 156 indicates an erratic scan speed, suggesting a potential hardware problem, the slide may be flagged as having a "rescan candidate" status, informing the computing device 148 that a subsequent rescan is required to obtain a sharper image.
[0076] Continuing with reference to FIG. 1 , it should be noted that scanning state 172 may include multiple categories aside from binary scanning states, and in a non-limiting example, scanning state 172 may include an “Optimal Scan” state, indicating that the slide scan is of excellent quality with no discernible issues. In some cases, scanning state 172 may include an “Acceptable Scan” state, indicating that the initial slide image may contain one or more minor issues that do not warrant a rescan (because it meets basic criteria) but are recorded for reference. In some cases, scanning state 172 may include a “Marginal Scan” state, indicating that the scan may have some issues that could affect analysis of the slide image but could not be corrected using image processing techniques such as those described herein. In other cases, scanning state 172 may include a “Failed Scan” state, indicating that the slide image for a particular slide has significant issues, making the slide image unsuitable for analysis and requiring a rescan. Those skilled in the art, upon reading this disclosure in its entirety, will recognize the various scanning conditions that system 100 may employ to determine quality control metrics for each slide of plurality of slides 112.
[0077] Continuing with reference to FIG. 1 , computing device 148 is configured to generate at least one scanning profile 176 as a function of scanning metadata 156 based on quality control metrics 160. As used in this disclosure, a “scanning profile” refers to a predefined set of parameters, configurations, and criteria used during the scanning process of a slide or specimen. Computing device 148 may be configured to generate multiple scanning profiles based on quality control metrics associated with multiple slide images 112. In some cases, scanning profile 176 may be utilized by imaging device 116 to generate slide images as described herein. In some embodiments, a scanning profile may be designed to optimize the scanning process described herein based on specific requirements or objectives, such that the resulting slide images are of desired quality and detail. In some cases, computing device 148 may be configured to select one or more scanning profiles during the scanning process of multiple slides 112. In some cases, a human operator may manually select a specific scanning profile for scanning a particular slide with a desired resolution, scanning speed, and other related factors, tailored to the nature of the particular specimen contained on the slide. In a non-limiting example, the scanning profile 176 may dictate how the scanning process of the plurality of slides 112 should be performed.
[0078] 1 , in one embodiment, scanning profiles 176 may include a default scanning profile 180; for purposes of this disclosure, a “default scanning profile” is a standard or primary configuration used by system 100 when no specific scanning profile is selected. In some cases, default scanning profile 180 may be designed to provide a balance between scanning speed and quality; for example, without limitation, default scanning profile 180 may be suitable for a wide range of general scanning tasks on a wide range of specimens. In such cases, default scanning profile 180 may include a high-throughput scanning profile that prioritizes speed over other factors, allowing imaging device 116 to quickly scan a large number of slides in a short period of time (e.g., four minutes per slide).
[0079] Still referring to FIG. 1 , in another embodiment, scanning profile 176 may include a rescan profile generated specifically for rescanning slides (e.g., flagged slides or slides classified as “rescan candidates” as described above). In some cases, a rescan profile may be configured to address issues identified in the initial scan. In a non-limiting example, the rescan profile may employ higher resolution settings, higher focus parameters, slower scan speeds, stronger illumination, etc. to ensure that issues identified in the initial slide images are addressed. If slide 104 is initially scanned under a high-throughput profile and flagged as blurry based on quality control metrics 160 derived from the generated initial scan images, a rescan profile may be executed by computing device 148 to obtain a sharper image. For example, without limitation, the rescan profile may reduce the scan speed and improve the focusing accuracy of imaging device 116.
[0080] Continuing with reference to FIG. 1 , other exemplary embodiments of scanning profiles 176 may include, but are not limited to, a high-resolution scanning profile (tailored for situations where maximum specimen detail is required, such as when cellular structures or complex tissue patterns need to be studied), a low-resolution scanning profile (designed for light-sensitive fluorescently labeled specimens or when minimal light exposure is desired), or a custom scanning profile that allows a user (e.g., a human operator) to define, select, or modify their own parameter set or existing configurations and / or scanning profiles within the system based on quality control metrics 160, quality control metadata, user input 164, etc. In some cases, scanning profiles 176 may be stored in database 168, as described above. In some cases, scanning profiles 176 may be correlated to at least one scanning state 172. For example, but not limited to, a “rescan candidate” state may automatically invoke a “rescan profile,” such as a “high-resolution rescan profile” with increased optical sensor sensitivity and / or focus parameters that prioritizes intricate specimen detail.
[0081] With continued reference to FIG. 1 , it should be noted that the above scanning profiles are exemplary and not exhaustive. Upon reviewing this disclosure in its entirety, those skilled in the art will recognize other embodiments that the scanning profile system 100 may employ that are not expressly described herein, as well as various modifications, alterations, and substitutions that may be made without departing from the scope of the described embodiments. Furthermore, terms such as "high," "low," "standard," "fast," and "slow" are relative and should be interpreted in the context of a particular application and known standards in the industry. Their usage herein is for illustrative purposes and should not be construed as limiting the scope of the present disclosure in any way.
[0082] Continuing with reference to FIG. 1 , scanning profile 176 may be generated based on associated scanning metadata 156. In a non-limiting example, computing device 148 may be configured to retrieve scanning metadata 156 from database 168. To generate scanning profile 176, scanning metadata associated with slide images having scanning statuses such as, but not limited to, “scan complete,” “best scan,” or “acceptable scan” may be used, and these metadata may be configured as reference or standard criteria for generating scanning profile 176. Additionally or alternatively, generating scanning profile 176 may include fine-tuning the associated scanning metadata 156. In some cases, parameters such as, but not limited to, light source intensity, magnification level, and optical sensor sensitivity may be adjusted based on algorithms such as autofocus, image stitching, and color correction that may be implemented by computing device 148.
[0083] 1 , one or more machine learning models may be employed by computing device 148 to generate one or more scanning profiles 176. In a non-limiting example, computing device 148 may receive scanning profile training data (from database 168 and / or a human operator), which may include a plurality of slide images as input, quality control metrics / metadata (e.g., specimen type or desired outcome), each of which may be correlated to a plurality of scanning parameter sets or configuration sets (i.e., scanning profiles) as output, which may be used to train a scanning profile generator generated using a machine learning module as described herein, and which may be used to generate at least one scanning profile 176 based on quality control metrics 160.
[0084] Continuing with reference to FIG. 1 , in some embodiments, generating at least one scanning profile 176 may include identifying at least one missing element 184 from the initial slide image 152 of the flagged slide as a function of the quality control metrics 160. As used in this disclosure, a “missing element” refers to a feature, structure, or detail of a specimen on a slide that was not accurately captured, represented, or identified in the corresponding slide image. In some cases, the missing element 184 may include an omission in the scanning process caused by suboptimal scanning parameters, hardware limitations, software errors, slide quality issues, foreign material, or inherent characteristics of the specimen itself. In a non-limiting example, the missing element 184 may be a cellular structure or abnormality that was not clearly imaged on a histopathology slide, potentially leading to a misreading or misdiagnosis. In another non-limiting example, the missing element 184 may include an unidentified microstructural defect or feature. In a further non-limiting example, the missing element 184 may include a necessary content present on the slide 104 that was not scanned.
[0085] Still referring to FIG. 1 , in some cases, missing elements 184 may be identified based on a threshold analysis of the quality control metrics 160 of each slide of the plurality of slides 112. For example, without limitation, the quality control metrics 160 may be compared to one or more predefined thresholds for various parameters (e.g., sharpness, focus quality, contrast, etc.). If any of these parameters fall below the set threshold, an indicator indicating a potential missing element may be raised. In a non-limiting example, an initial slide image 152 with a sharpness score below an acceptable range may have blurred areas where cellular structures or other details are not discernible. In some cases, missing elements 184 may be identified by comparing the initial slide image 152 to a reference or standard image. Missing elements 184 may be identified as a function of discrepancies identified from such a comparison. In a non-limiting example, if quality control metrics determined based on the initial slide image 152 indicate a significant deviation from the standard, this may indicate the presence of one or more missing elements 184.
[0086] Still referring to FIG. 1 , in some cases, quality control metadata (including a detailed record of multiple scanning parameters and resulting quality scores) generated from quality control metrics 160 as described above may be analyzed by computing device 148 to identify missing elements 184. In a non-limiting example, if the quality control metadata indicates that a particular region of slide 104 was scanned with one or more suboptimal focus settings, computing device 148 may identify the particular region as missing element 184. Additionally or alternatively, advanced image processing algorithms may be used and / or machine learning models may be trained to recognize common patterns associated with missing elements 184, as described herein. In some cases, training data may include multiple slide images, quality control metrics, and metadata as inputs, correlated with multiple missing elements as outputs. In some cases, missing elements 184 may include values describing the likelihood of missing elements in each scanned image of a slide. Additionally, missing elements 184 may be manually defined based on user input 164, as described above. In a non-limiting example, one or more user feedbacks from a user (e.g., pathology or a human operator) may include identification of missing elements 184 in the initial slide image 152. In some cases, the user feedback may be added to quality control metadata or training data to improve the accuracy of missing element detection, as described herein.
[0087] Continuing with reference to FIG. 1 , in some embodiments, generating at least one scanning profile 176 may also include selecting scanning parameters 188 configured to address at least one missing element 184. As used in this disclosure, “scanning parameters” refers to variables or settings that direct how a scan / rescan process is performed. In some cases, the scanning parameters 188 may affect the quality, resolution, speed, and other attributes of the scanned images of the slides 112. In some cases, the scanning parameters may include, but are not limited to, pre-scan device settings, resolution (DPI or PPI), focus parameters, magnification levels, lighting settings (e.g., type of light, light intensity, light angle, etc.), scan speed, filter settings (e.g., selection of filters to enhance or suppress specific wavelengths of light), post-scan image processing algorithms, and scan patterns (i.e., the path or order in which the imaging device 116 captures the specimen (e.g., linear, serpentine, or region of interest [ROI]-based)). Those skilled in the art will recognize, upon review of this disclosure in its entirety, that a variety of scanning parameters may be selected for purposes of scanning or rescanning a slide.
[0088] Still referring to FIG. 1 , in some cases, one or more scanning parameters 188 may be selected or adjusted to address the identified missing element 184. In a non-limiting example, computing device 148 may identify a region of initial slide image 152 as missing element 184 based on quality control metrics 160 due to poor focus, and additional focus parameters may be selected and / or existing focus parameters may be adjusted for that region in a subsequent scan. At least one scanning profile 176 may be generated as a function of multiple scanning parameters 188. In some cases, scanning profile 176 may be generated or modified based on the adjusted scanning parameters. In some cases, imaging device 116 may employ at least one scanning profile 176 to address missing element 184 of initial slide image 152 in subsequent slide images of the slide, as described in more detail below. In a non-limiting example, one or more additional lighting parameters may be selected, and existing lighting parameters may be fine-tuned to correct cellular structure visibility problems caused by insufficient lighting during the initial scanning process as determined based on quality control metrics 160 of the generated initial slide images. Computing device 148 may generate a scanning profile based on these lighting parameters and apply the generated scanning profile during subsequent scans of the slide.
[0089] Continuing with reference to FIG. 1 , in one embodiment, scanning parameters 188 may include imaging device configuration parameters, where “imaging device configuration parameters,” for purposes of this disclosure, refer to specific settings made to imaging device 116, as described herein, that dictate the operational behavior of imaging device 116 during a scan / rescan process. In some cases, system 100 may include multiple imaging devices. In some embodiments, system 100 may include multiple imaging devices, each imaging device of the multiple imaging devices under different settings (i.e., different sets of imaging device configuration parameters). In a non-limiting example, scanning profile 176 may further include imaging device selection from among multiple currently available imaging devices based on quality control metrics 160 and / or quality control metadata. Exemplary imaging device configuration parameters may include, but are not limited to, optical sensor settings (e.g., ISO, exposure time, pixel binning, etc.), aperture size, focal length, zoom level, wavelength selection, stage selection, etc. In a non-limiting example, scanning profile 176 may specify an imaging device with high magnification and optical sensor sensitivity for scanning a slide having a specimen requiring detailed imaging at the cellular level.
[0090] Still referring to FIG. 1 , in some cases, the plurality of scanning parameters 188 may be stored in database 168 as described above. The selection of the plurality of scanning parameters 188 may be performed manually by a human operator based on expert knowledge. In other cases, one or more machine learning models may be trained by computing device 148 using scanning parameter training data, which may include a plurality of missing elements as inputs correlated with a plurality of scanning parameter sets as outputs, and the trained scanning parameter selection model is used to automatically select the plurality of scanning parameters 188 as a function of the missing elements 184. In some cases, the scanning parameter training data may be received from database 168. In some cases, the scanning parameter training data may further include slide images, associated scanning metadata, quality control metrics, and / or quality control metadata as inputs correlated with a plurality of scanning parameters as outputs.
[0091] Continuing with reference to FIG. 1 , computing device 148 is configured to rescan one or more flagged slides 192 within plurality of slides 112 at imaging device 116 using at least one scanning profile 176, as described above. In some cases, flagged slides 192 may include slides associated with a scanning state as “rescan candidates.” In a non-limiting example, flagged slides 192 may include slides that initially had generated slide images exhibiting artifacts, blur, or incomplete image capture during an initial scan due to various reasons, such as mechanical disturbances, suboptimal lighting conditions and / or focus parameters, slide misalignment, etc., as described above. In some cases, rescanning flagged slides 192 may include scanning the flagged slides using scanning profile 176. In some cases, rescanning flagged slides 192 may include adjusting imaging device configuration parameters according to scanning profile 176. In some cases, rescanning flagged slides 192 may include scanning the flagged slides with a second imaging device based on scanning profile 176. In a non-limiting embodiment, rescanning flagged slides 192 may include applying scanning profile 176 to imaging device 116, and applying scanning profile 176 may include modifying scanning metadata 156 based on a plurality of scanning parameters incorporated therein. Subsequent slide images 196 of each flagged slide may be generated by computing device 148 using imaging device 116 to rescan slide 104 using scanning profile 176.
[0092] Continuing with reference to FIG. 1 , in some embodiments, rescanning flagged slides 192 may include scanning identified missing elements 184 using at least one scanning profile 176 to generate slide image fragments. As used in this disclosure, a “slide image fragment” refers to a segmented or partial slide image capture of a slide, particularly one or more missing elements 184 that were of interest for capture. In some cases, unlike a full slide image (or WSI) that provides a comprehensive view of the entire slide, a slide image fragment may focus on capturing only a specified region of interest (ROI), which may include one or more missing elements 184. In cases where flagged slides 192 do not need to be rescanned, and only specific areas that were previously missing or insufficiently captured are scanned, one or more slide image fragments may be generated for each flagged slide using a scanning profile 176, and such scanning profile 176 may, in some cases, include an increased optical zoom level, increased exposure, best focus parameters, etc.
[0093] Still referring to FIG. 1 , additionally or alternatively, subsequent slide images for each flagged slide may be generated by merging slide image fragments with the initial slide image 152, as described above. Once a slide image fragment is captured, it may be seamlessly integrated with the initial slide image 152 to create a composite image that combines features from both scans. In some cases, the merging process may include registering the slide image fragment with a corresponding region of the initial slide image 152; for example, without limitation, computing device 148 may be configured to utilize one or more image registration techniques (e.g., feature-based registration, in which key features in both images are identified and used as reference points for registration). In some cases, a weighted blending algorithm may be employed by computing device 148 to smooth the transition between the boundary between the slide image fragment and the initial slide image 152. In some cases, multi-resolution blending or Laplacian pyramid blending may be used to merge images at multiple resolutions (resulting from scanning slides with different scanning profiles) to ensure that specimen details are preserved in the composite image. Additionally, any brightness or color discrepancies between the slide image fragment and the initial slide image 152 can be adjusted using histogram matching or other color correction algorithms.
[0094] Still referring to FIG. 1 , in a non-limiting example, computing device 148 may combine slide image fragments for each flagged slide with initial slide image 152 through image fusion. As used in this disclosure, “image fusion” refers to a process of gathering important information from multiple images into fewer, typically one, images. One method of image fusion is multi-focus image fusion. In this disclosure, “multi-focus image fusion” refers to an image fusion process that combines input images with different focal depths to create at least one output image consisting of focused image data from the input images. According to some embodiments, multi-focus image fusion may include selecting focused image data from multiple input images. The determination of the focused image data may be performed pixel-by-pixel, for example, according to one or more image fusion metrics. Image fusion is generally divided into two categories: transform domain and spatial domain. Transforms commonly used in image fusion are the discrete cosine transform (DCT) and the multi-scale transform (MST). In some cases, image fusion may be based on the MST. Exemplary MST methods include the Laplacian pyramid transform, gradient pyramid-based transforms, morphological pyramid transforms, and their higher-level counterparts, the discrete wavelet transform, the shift-invariant wavelet transform (SIDWT), and the discrete cosine harmonic wavelet transform (DCHWT). In some cases, DCT-based methods can be more efficient than MST in terms of image transmission and archiving. In some cases, the DCT may be employed for Joint Photographic Experts Group (JPEG)-encoded images. The JPEG system consists of an encoder and decoder pair. In the encoder, the image is divided into 8x8 non-overlapping blocks, and DCT coefficients are calculated for each. Because quantization of DCT coefficients is a lossy process, many small-valued DCT coefficients are quantized to zero, corresponding to high frequencies. DCT-based image fusion algorithms perform better when multi-focus image fusion methods are applied in the compressed domain. In some cases, image fusion methods based on the DCT domain do not require sequential decoding and encoding operations.Exemplary DCT image fusion processes include DCT+Variance, DCT+Corr_Eng, DCT+EOL, and DCT+VOL.
[0095] Still referring to FIG. 1 , in some cases, image fusion methods may additionally or alternatively include processing in the spatial domain. Image fusion may employ the determination and use of focus measures, including variance, energy of image gradient (EOG), Tenenbaum's algorithm (Tenengrad), energy of the Laplacian (EOL), sum-modified-Laplacian (SML), and spatial frequency (SF). Image fusion may include aggregating in-focus regions from multiple input images into an image. In some cases, image edges or boundaries between regions with different in-focus input images may be treated differently. For example, in some cases, a section of an image at a boundary between two (or more) in-focus images may be treated as a weighted average of values from the two (or more) closest-in-focus images. In some cases, one or more machine learning processes, such as those described herein, may be used to assist the image fusion process.
[0096] Referring now to FIG. 2, an exemplary embodiment of a decision tree 200 involved in the automated generation of scanning profiles is illustrated. As described herein, a "decision tree" refers to a data structure that represents and combines one or more decisions or other calculations based on and / or with respect to provided data, as well as previous such decisions or calculations, as nodes of a tree data structure with inputs of some nodes connected to outputs of other nodes. In a non-limiting example, decision logic for selecting at least one scanning profile 176 from a plurality of predetermined or existing scanning profiles may be implemented as a decision tree. In some cases, generating at least one scanning profile 176 as a function of scanning metadata 156 may include selecting at least one scanning profile 176 from a plurality of scanning profiles as a function of quality control metrics 160 and / or traversal of decision tree 200 based on quality control metadata, as described above with reference to FIG. 1.
[0097] Still referring to FIG. 2 , as used in this disclosure, a “node” of a tree is an entity that contains a value or data and, optionally, one or more connections to other nodes. The nodes in decision tree 200 may include at least one root node 204, i.e., a node that receives quality control metadata to decision tree 200, such as processed scan data 156, quality control metrics 160, and / or overall or individual quality scores for scanned slide images. Intermediate nodes 208 a-g within the nodes in decision tree 200 may include, but are not limited to, categories of quality control metadata (e.g., but not limited to, “missing tissue,” “color issues,” “resolution issues,” “focus issues,” etc.). In some cases, intermediate nodes 208 a-g may also include one or more quality control metadata subcategories (e.g., internal nodes 208 e-g connected to node 208 a) (e.g., “faint tissue,” “partial scan,” “missing pieces,” etc.).
[0098] 2, the plurality of nodes in decision tree 200 may include a plurality of terminal nodes 212a-212f, which may alternatively or additionally be referred to herein as "leaf nodes," corresponding to execution results of decision tree 200. In other words, decisions and / or judgments generated by decision tree 200 may be output at at least one terminal node. In a non-limiting example, decision tree 200 may include a plurality of terminal nodes 212a-212f, and each terminal node of the plurality of terminal nodes 212a-212f may include at least one scanning profile as described herein.
[0099] Still referring to FIG. 2, in some cases, the scanning profile may include an "intermediate mag profile (IMP)," which may be designed to address issues related to faint tissue and debris. In one embodiment, the IMP may use an intermediate level of magnification to increase the visibility of faint tissue structures and ensure that no tissue debris is lost during the scanning process. In a non-limiting example, the imaging device 116 may be adjusted to an intermediate level to capture details that may be missed at lower magnifications while avoiding the excessive detail and noise associated with higher magnifications.
[0100] Still referring to FIG. 2, in some cases, the scanning profile may include an "extra probe profile (EPP)," and the EEP may be tailored to situations where missing tissue exists due to partial scanning. In one embodiment, the EEP may employ additional or more sensitive probes to ensure comprehensive scan coverage. In a non-limiting example, the EEP may involve multiple passes or the use of probes with a wider field of view or higher sensitivity to ensure that no portion of the slide is left unscanned.
[0101] Still referring to FIG. 2 , in some cases, the scanning profile may include a “custom color profile (CCP),” which may be configured to address color inconsistencies or discrepancies in the scanned images. In one embodiment, the CCP may enable calibration and adjustment of the color settings of the imaging device 116. In a non-limiting example, the color of a subsequent slide image 196 may be compared to the color of the actual slide to accurately represent the color captured by the imaging device under the CCP. In some cases, color issues may arise when analyzing stained tissue samples where color distinctions are important.
[0102] Still referring to FIG. 2, in some cases, the scanning profile may include a "high magnification profile (HMP)," which may be used to solve problems with image resolution. In one embodiment, the HMP may use a higher magnification setting to capture slide details. In a non-limiting example, the imaging device 116 under the HMP may have a higher magnification level, which will capture slide images with more complex structures and details, resulting in a clearer, more detailed scan.
[0103] Still referring to FIG. 2 , in other cases, the scanning profile may include a “Higher Z Stack Profile (HZSP),” which may be configured to address focusing issues in the initial slide image 152. In one embodiment, the HZSP may involve configuring the imaging device 116 to capture multiple images at different focal planes (e.g., increasing the depth of field and number of focal planes) and then combining them to generate a single focused slide image (i.e., a Z-stack). As a non-limiting example, for thick or uneven samples, which may present focusing issues, different portions of the sample may be in focus at different depths. By employing the HZSP, the imaging device 116 can ensure that all portions of the sample are in focus in the subsequent slide image 196.
[0104] Continuing with reference to FIG. 2 , in some cases, the computing device 148 may generate two or more potentially overlapping decision trees 200. In a non-limiting example, the decision tree 200 may establish links to remote decision modules, devices, systems, etc. The root node 204 of the decision tree 200 may connect to and / or receive output from one or more terminal nodes of another decision tree (not shown). One or more of the nodes, such as intermediate nodes, of the decision tree 200 may be shared with another decision tree, etc. In some embodiments, the decision tree 200 may be configured to use one or more application programming interfaces (APIs) to receive scanning metadata 156 and / or incorporate multiple quality control metrics 160 or quality control metadata. In other embodiments, the decision tree 200 may perform one or more database lookups and / or lookup table lookups.
[0105] Still referring to FIG. 2 , in a non-limiting example, based on the scanning metadata 156 and quality control metrics 160 or metadata, the selection of one or more scanning parameters 188 can be automated as a function of traversal of a decision tree 200, leading to the creation of a custom scanning profile for each rescanned slide, although a user (e.g., a human operator) can customize the determined or selected scanning profile if desired. It should be noted that the scanning profiles as shown in FIG. 2 are only a subset of possible profiles, and the actual range of profiles may vary based on the hardware components available in the imaging device 116. Those skilled in the art will recognize, upon review of this disclosure in its entirety, that the described system 100 can be extended, modified, or adapted to incorporate newer scanning technologies, advanced hardware components, and evolving software algorithms to further improve scan processing and image quality.
[0106] 3A , an exemplary embodiment of scanning parameters for a first imaging device 300a is illustrated. In some cases, a first imaging device (e.g., first imaging device 300a) may be partially comprised of a 4x objective lens 304a and a 40x objective lens 304b. In a non-limiting example, the 4x objective lens 304a may be associated with a 4x scanning profile 308a, and the 40x objective lens 304b may be associated with a 40x scanning profile 308b. In some cases, the 4x scanning profile 308a may be primarily used for broader, general scans of a specimen, capturing a larger area at a lower magnification. In other cases, the 40x scanning profile 308b may provide a higher magnification, enabling a detailed close-up view of a specific ROI within the specimen.
[0107] 3B, an exemplary embodiment of scanning parameters for second imaging device 300b is illustrated. In some cases, a second imaging device (e.g., second imaging device 300b) may be partially comprised of a 6x objective lens 304c and a 60x objective lens 304d. Similarly, 6x objective lens 304c may be associated with 6x scanning profile 308c, and 60x objective lens 304d may be associated with 60x scanning profile 308d. In some cases, 6x scanning profile 308c may be used for moderate magnification scans, providing a balanced field of view that captures a relatively wide area while still providing a decent level of detail. In other cases, 60x scanning profile 308d may provide an extremely high level of magnification, ideal for detailed examination of specific areas within a specimen.
[0108] 4, an exemplary embodiment of a machine learning module 400 that may perform one or more machine learning processes as described in this disclosure is illustrated. The machine learning module may use the machine learning processes to perform the determining, classifying, and / or analyzing steps, methods, processes, etc. as described in this disclosure. As used in this disclosure, a "machine learning process" is a process that automatically uses training data 404 to generate algorithms instantiated in hardware or software logic, data structures, and / or functions executed by a computing device / module to generate output 408 when data is provided as input 412, in contrast to non-machine learning software programs in which the commands to be executed are predetermined by a user and written in a programming language.
[0109] Still referring to FIG. 4 , as used herein, “training data” refers to data containing correlations that a machine learning process can use to model relationships between two or more categories of data elements. For example, without limitation, training data 404 may include multiple data entries, also known as “training examples,” each representing a set of data elements recorded, received, and / or generated together, where the data elements may be correlated by shared presence in a given data entry, proximity in a given data entry, etc. The multiple data entries of training data 404 may exhibit one or more trends in correlations between categories of data elements. For example, without limitation, high values of a first data element belonging to a first category of data elements tend to correlate with high values of a second data element belonging to a second category of data elements, indicating the possibility of a proportional or other mathematical relationship linking values belonging to the two categories. Multiple categories of data elements may be associated in training data 404 according to various correlations. Correlations may indicate causal and / or predictive links between categories of data elements and may be modeled as relationships, such as mathematical relationships, by machine learning processes, as described in further detail below. The training data 404 may be formatted and / or organized by categories of data elements, for example, by associating the data elements with one or more descriptors that correspond to the categories of the data elements. As a non-limiting example, the training data 404 may include data entered into standardized forms by a person or process, such that entry of a given data element in a given field of the form maps to one or more descriptors of the category. Elements of the training data 404 may be linked to descriptors of the category by tags, tokens, or other data elements.For example, but not by way of limitation, the training data 404 may be provided in a fixed-length format, a format that links the location of the data to a category, such as a comma-separated values (CSV) format, and / or a self-describing format, such as Extensible Markup Language (XML) or JavaScript Object Notation (JSON), that allows a process or device to detect the category of the data.
[0110] Alternatively or additionally, with continued reference to FIG. 4 , the training data 404 may include one or more uncategorized elements. That is, the training data 404 may be unformatted or may not include descriptors for some elements of the data. Machine learning algorithms and / or other processes may sort the training data 404 according to one or more categorizations, for example, using natural language processing algorithms, tokenization, detecting correlation values in the raw data, etc. Categories may be generated using correlation and / or other processing algorithms. As a non-limiting example, in a corpus of text, phrases consisting of “n” compounds, such as nouns modified by other nouns, may be identified according to the statistically significant prevalence of n-grams containing such words in a particular order. Such n-grams may be categorized as elements of language, such as “words,” that are tracked similarly to single words, generating new categories as a result of the statistical analysis. Similarly, in a data entry containing text data, a person's name may be identified by reference to a list, dictionary, or other glossary, allowing for ad-hoc categorization by a machine learning algorithm and / or automatic association of the data in the data entry with a descriptor or a given format. The ability to automatically categorize data entries allows the same training data 404 to be applied to two or more different machine learning algorithms, as described in further detail below. The training data 404 used by the machine learning module 400 may correlate any input data, as described in this disclosure, with any output data, as described in this disclosure. As a non-limiting example, the training data may include a plurality of slide images as input correlated with a plurality of scanning conditions as output.
[0111] Continuing with reference to FIG. 4 , the training data may be filtered, sorted, and / or selected using one or more supervised and / or unsupervised machine learning processes and / or models, as described in further detail below, including, but not limited to, the training data classifier 416. The training data classifier 416 may include a “classifier,” which as used in this disclosure is a machine learning model, as defined below, and is a data structure that represents and / or uses a mathematical model, neural net, or program generated by a machine learning algorithm known as a “classification algorithm,” as described in further detail below, for example, that sorts inputs into categories or bins of data and outputs the categories or bins of data and / or their associated labels. The classifier may be configured to output at least one datum that labels or otherwise identifies sets of data, such as those clustered and found to be close under a distance metric, as described below. The distance metric may include any norm, such as, but not limited to, the Pythagorean norm. The machine learning module 400 may generate a classifier using a classification algorithm, defined as a process in which a computing device and / or any module and / or component operating on the computing device derives a classifier from the training data 404. Classification may be performed using, but is not limited to, linear classifiers such as, but not limited to, logistic regression and / or naive Bayes classifiers, nearest neighbor classifiers such as k-nearest neighbor classifiers, support vector machines, least squares support vector machines, Fisher's linear discriminant, quadratic classifiers, decision trees, boosted trees, random forest classifiers, learning vector quantization, and / or neural network-based classifiers. As a non-limiting example, the training data classifier 416 may classify elements of the training data based on certain demographic and phenotypic attributes, leading to the identification of distinct subpopulations within the dataset.These attributes may include age, sex, ethnicity, genetic markers, environmental exposures, medical history, and particular physiological or pathological conditions associated with the specimen.
[0112] Still referring to FIG. 4 , the computing device 148 may be configured to generate a classifier using a Naive Bayes classification algorithm. The Naive Bayes classification algorithm generates a classifier by assigning class labels to problem instances represented as vectors of element values. The class labels are drawn from a finite set. The Naive Bayes classification algorithm may include generating a family of algorithms that assume that, given a class variable, the value of a particular element is independent of the values of other elements. The Naive Bayes classification algorithm may be based on Bayes' theorem, expressed as P(A / B)=P(B / A)P(A)÷P(B), where P(A / B) is the probability of hypothesis A given data B, also known as the posterior probability; P(B / A) is the probability of data B given that hypothesis A is true; P(A) is the probability that hypothesis A is true regardless of the data, also known as the prior probability of A; and P(B) is the probability of the data regardless of the hypothesis. The Naive Bayes algorithm may be generated by first converting training data into a frequency table. The computing device 148 can then calculate a likelihood table by calculating the probabilities of different data entries and classification labels. The computing device 148 can utilize the Naive Bayes equation to calculate the posterior probability of each class. The class with the highest posterior probability is the predicted result. The Naive Bayes classification algorithm can include a Gaussian model that follows a normal distribution. The Naive Bayes classification algorithm can also include a multinomial model used for discrete counts. The Naive Bayes classification algorithm can include a Bernoulli model that is utilized when the vector is binary.
[0113] Continuing with reference to FIG. 4 , the computing device 148 may be configured to generate a classifier using a K-nearest neighbor (KNN) algorithm. As used in this disclosure, a “K-nearest neighbor algorithm” includes a classification method that analyzes how closely out-of-sample features resemble the training data, utilizing feature similarity to classify input data into one or more clusters and / or categories of features represented in the training data. This may be performed by representing both the training data and the input data in vector form and using one or more vector similarity measures to identify classifications in the training data and determine the classification of the input data. The K-nearest neighbor algorithm may include specifying a K value, or a number that instructs the classifier to select the k entries of training data that are most similar to a given sample, determining the most common classifier of entries in the database, and classifying the known sample. This may be performed recursively and / or iteratively to generate a classifier that can be used to further classify the input data as a sample. For example, an initial set of samples may be run to cover initial heuristics and / or "first guesses" at outputs and / or relationships, which may be seeded using, without limitation, expert input received according to any process described herein. As a non-limiting example, the initial heuristics may include ranking associations between inputs and elements of training data. The heuristics may include selecting a few highest-ranking associations and / or training data elements.
[0114] Continuing with reference to FIG. 4 , the generating k-nearest neighbor algorithm generates a first vector output containing data entry clusters, generates a second vector output containing input data, and can calculate the distance between the first and second vector outputs using any suitable norm, such as cosine similarity, Euclidean distance measure, etc. Each vector output can be represented as, without limitation, a tuple of n values, where n is at least two values. Each value in the n-tuple of values can represent a measurement or other quantitative value associated with a given category of data or attribute, examples of which are provided in further detail below. The vectors can be represented in an n-dimensional space using, without limitation, an axis for each category of values represented in the n-tuple of values, such that the vectors have a geometric direction that characterizes the relative amounts of the attributes in the n-tuple compared to each other. Two vectors can be considered equivalent if their directions and / or the relative amounts of the values in each vector are the same compared to each other. Thus, as a non-limiting example, a vector represented as [5, 10, 15] can be treated as equivalent to a vector represented as [1, 2, 3] for purposes of this disclosure. Vectors may be more similar if their directions are more similar, and more dissimilar if their directions are more dissimilar. However, vector similarity may alternatively or additionally be determined using the average of similarities between similar attributes, or any other measure of similarity appropriate for any n-tuple of values, or an aggregation of numerical similarity measures for purposes of a loss function, as described in more detail below. Any vector described herein may be scaled so that each vector represents each attribute along an equivalent scale of values. Each vector may be "normalized," or scaled using the Pythagorean norm:
number
[0115] With further reference to FIG. 4 , training examples for use as training data may be selected from a population of potential examples according to a cohort relevant to the analytical problem, classification task, etc. to be solved. Alternatively or additionally, training data may be selected to span a set of situations or inputs that the machine learning model and / or process is likely to encounter upon deployment. For example, for each category of input data to a machine learning process or model that may exist across a range of values in a population of phenomena, such as, but not limited to, images, user data, processing data, physical data, etc., the computing device, processor, and / or machine learning model may select training examples representing each possible value in such range and / or a representative sample of values in such range. Selecting a representative sample may include selecting training examples in a proportion that matches a statistically determined and / or predicted distribution of such values according to relative frequency, e.g., such that more frequently encountered values in the population of analyzed data are represented by more training examples than less frequently encountered values. Alternatively or additionally, the set of training examples may be compared to a collection of representative values in a database and / or presented to a user, allowing the process to detect, automatically or via user input, one or more values not included in the set of training examples. A computing device, processor, and / or module may automatically generate missing training examples by receiving and / or acquiring missing input and / or output values and associating the missing input and / or output values with corresponding output and / or input values that coexist in the data record with the acquired values, such as provided by a user and / or other device.
[0116] 4, the computer, processor, and / or module may be configured to preprocess the training data. As used in this disclosure, "preprocessing" training data refers to converting the training data from its raw state into a form that can be used to train a machine learning model. Preprocessing may include sanitization, feature selection, feature scaling, data augmentation, etc.
[0117] Still referring to FIG. 4 , the computer, processor, and / or module may be configured to sanitize the training data. As used in this disclosure, “sanitizing” training data refers to a process in which training examples that prevent a machine learning model and / or process from converging to a useful result are removed. For example, without limitation, the training examples may include input and / or output values that are outliers from typically encountered values, so that the machine learning algorithm using the training examples can adapt to unlikely quantities as inputs and / or outputs. For example, values that are more than a threshold number of standard deviations away from the average, mean, or expected value may be removed. Alternatively or additionally, one or more training examples may be identified as having low-quality data, where “low-quality” is defined as having a signal-to-noise ratio below a threshold. Sanitization may include steps such as removing duplicate or other redundant data, interpolating missing data, correcting data errors, standardizing data, and identifying outliers. In a non-limiting example, sanitization may include utilizing an algorithm to identify duplicate entries or a spell-checking algorithm.
[0118] As a non-limiting example, and with further reference to FIG. 4 , images used to train an image classifier or other machine learning model and / or process that takes images as input or produces images as output may be rejected if their image quality is below a threshold. For example, without limitation, a computing device, processor, and / or module may perform blur detection and eliminate one or more blurs. Blur detection may be performed by, as a non-limiting example, taking a Fourier transform or an approximation, such as a fast Fourier transform (FFT), of the image and analyzing the distribution of low and high frequencies in the resulting frequency domain representation of the image. The number of high frequency values below a threshold level may indicate blur. In a further non-limiting example, blur detection may be performed by convolving the image, a channel of the image, or the like, with a Laplacian kernel, which may generate a numerical score reflecting the number of abrupt changes in intensity shown in the image, with a high score indicating sharpness and a low score indicating blur. Blur detection can be performed using gradient-based operators, which measure the operator based on the gradient or first derivative of the image, based on the hypothesis that abrupt changes indicate sharp edges in the image and therefore less blur. Blur detection may also be performed using wavelet-based operators, which exploit the ability of discrete wavelet transform coefficients to describe the frequency and spatial content of an image. Blur detection may also be performed using statistics-based operators, which utilize several image statistics as texture descriptors to calculate the focus level. Blur detection may also be performed using discrete cosine transform (DCT) coefficients to calculate the focus level of an image from its frequency content.
[0119] Still referring to FIG. 4 , a computing device, processor, and / or module may be configured to be preconditioned on one or more training examples. For example, without limitation, if a machine learning model and / or process has one or more inputs and / or outputs that require, transmit, or receive a certain number of bits, samples, or other units of data, elements of one or more training examples used as or compared to the inputs and / or outputs may be modified to have such units of data. For example, a computing device, processor, and / or module may convert a smaller number of units, such as an image with a low number of pixels, into a desired number of units, such as by upsampling or interpolation. As a non-limiting example, an image with a low number of pixels may have 100 pixels, but the desired number of pixels may be 128 pixels. The processor may interpolate the image with a low number of pixels to convert the 100 pixels into 128 pixels. It should also be noted that, after reading this disclosure, one skilled in the art will know various methods for interpolating a smaller number of data units, such as samples, pixels, or bits, into a desired number of such units. In some examples, the set of interpolation rules may be trained with highly detailed inputs and / or outputs, a corresponding set of inputs and / or outputs downsampled to a smaller number of units, and a neural network or other machine learning model trained to predict interpolated pixel values using the training data. As a non-limiting example, sample inputs and / or outputs, such as a sample image having sample-augmented data units (e.g., pixels added between original pixels), can be input to a neural network or machine learning model, which can output a pseudo-replica sample image in which pixels between the original pixels are assigned dummy values based on the set of interpolation rules.As a non-limiting example, in the context of an image classifier, a machine learning model may have a set of interpolation rules trained with a set of high-definition images and images downsampled to a smaller number of pixels, and a neural network or other machine learning model trained using those examples to predict interpolated pixel values in the context of facial images. As a result, inputs with sample-expanded data units (with dummy values added between original data units) can be run through the trained neural network and / or model to fill in values to replace the dummy values. Alternatively or additionally, the processor, computing device, and / or module may utilize a sample expander technique, a low-pass filter, or both. A "low-pass filter," as used in this disclosure, is a filter that passes signals with frequencies below a selected cutoff frequency and attenuates signals with frequencies above the cutoff frequency. The exact frequency response of the filter depends on the filter design. The computing device, processor, and / or module may use averaging, such as luma averaging or chroma averaging, in the image to fill in data units between the original data units.
[0120] In some embodiments, with continued reference to FIG. 4 , a computing device, processor, and / or module may downsample elements of a training example to a desired smaller number of data elements. As a non-limiting example, a high-pixel-count image may have 256 pixels, but a desired number of pixels may be 128. The processor may downsample the high-pixel-count image to convert the 256 pixels to 128 pixels. In some embodiments, the processor may be configured to perform downsampling on the data. Downsampling, also known as decimation, may involve removing every Nth entry, all but every Nth entry, etc., in a sequence of samples, a process known as “compression,” which may be performed, for example, by an N-sample compressor implemented using hardware or software. Anti-aliasing and / or anti-imaging filters, and / or low-pass filters may be used to clean up compression artifacts.
[0121] 4, feature selection includes narrowing and / or filtering training data to exclude features and / or elements, or training data containing such elements, that are not relevant to the purpose for which the trained machine learning model and / or algorithm is trained, and / or collecting features and / or elements, or training data containing such elements, based on their relevance or usefulness to the intended task or purpose for which the trained machine learning model and / or algorithm is trained. Feature selection can be performed using any process described in this disclosure, including, but not limited to, using training data classifiers, filtering outliers, etc.
[0122] Continuing with reference to FIG. 4, feature scaling may include, but is not limited to, normalization of data entries, which may be accomplished by dividing a numeric field by its norm, as performed, for example, in vector normalization. Feature scaling may include maximum absolute value scaling, where each quantitative data is divided by the maximum absolute value of all quantitative data in a set or subset of quantitative data. Feature scaling may be performed by subtracting the minimum value X from the set or subset of values from which each value X is subtracted. min and the result is divided by a range of values, max may include min-max scaling, giving the maximum value in:
number
number
number
number
[0123] With further reference to FIG. 4 , a computing device, processor, and / or module may be configured to perform one or more processes of data augmentation. As used in this disclosure, “data augmentation” is the addition of data to a training set using elements and / or items already contained in the dataset. Data augmentation may be achieved using, without limitation, interpolation, generating modified copies of existing entries and / or examples, and / or one or more generative AI processes, for example, using deep neural networks and / or generative adversarial networks. A generative process may alternatively be referred to in this context as “data synthesis” or creating “synthetic data.” Augmentation may include performing one or more transformations on the data, such as geometric, color space, affine, brightness, cropping, and / or contrast transformations of the image.
[0124] Still referring to FIG. 4 , the machine learning module 400 may be configured to execute a lazy learning process 420 and / or protocol. This may alternatively be referred to as a “lazy loading” or “call-when-needed” process and / or protocol, in which machine learning is performed by combining the input with a training set upon receipt of the input to be converted into an output and deriving an algorithm used to generate the output on demand. For example, an initial set of simulations may be run to cover initial heuristics and / or “first guesses” on outputs and / or relationships. As a non-limiting example, the initial heuristics may include ranking associations between the input and elements of the training data 404. The heuristics may include selecting several highest-ranking associations and / or elements of the training data 404. Lazy learning may be implemented using any suitable lazy learning algorithm, including, but not limited to, a K-nearest neighbor algorithm, a lazy Naive Bayes algorithm, etc., and upon reviewing this disclosure in its entirety, one skilled in the art will recognize a variety of lazy learning algorithms that may be applied to generate outputs as described in this disclosure, including, but not limited to, lazy learning applications of machine learning algorithms, as described in more detail below.
[0125] Alternatively or additionally, and continuing to refer to FIG. 4 , a machine learning process as described herein can be used to generate the machine learning model 424. As used herein, a “machine learning model” is a data structure that represents and / or instantiates a mathematical and / or algorithmic representation of a relationship between inputs and outputs, generated using any machine learning process, including but not limited to any of the processes described above, and stored in memory; once created, the inputs are submitted to the machine learning model 424, which generates an output based on the derived relationship. For example, without limitation, a linear regression model generated using a linear regression algorithm can calculate a linear combination of input data using coefficients derived during the machine learning process to calculate output data. As a further non-limiting example, the machine learning model 424 may be generated by creating an artificial neural network, such as a convolutional neural network, that includes an input layer of nodes, one or more hidden layers, and an output layer of nodes. Connections between nodes may be created through a process of "training" the network, in which elements from a set of training data 404 are applied to the input nodes, and an appropriate training algorithm (such as Levenberg-Marquardt, conjugate gradient, simulated annealing, or other algorithm) is then used to adjust the connections and weights between nodes in adjacent layers of the neural network to produce the desired values at the output nodes. This process is sometimes referred to as deep learning.
[0126] Still referring to FIG. 4 , the machine learning algorithm can include at least one supervised machine learning process 428. At least one supervised machine learning process 428, as defined herein, includes an algorithm that receives a training set relating a number of inputs to a number of outputs and attempts to generate one or more data structures that represent and / or instantiate one or more mathematical relationships relating the inputs to the outputs, each of the one or more mathematical relationships being optimal according to some criteria specified to the algorithm using some scoring function. For example, the supervised learning algorithm can include a plurality of scanning metadata and quality control metrics or metadata, such as those described above, as inputs, a plurality of scanning profiles as outputs, and a scoring function that represents the desired form of the relationship to be found between the inputs and outputs. The scoring function can, for example, attempt to maximize the probability that a given input and / or combination of elements of the inputs is associated with a given output and minimize the probability that a given input is not associated with a given output. The scoring function may be expressed as a risk function that represents the “expected loss” of the algorithm relating inputs to outputs, where the loss is calculated as an error function that represents the degree to which the prediction produced by the relationship is inaccurate when compared to a given input-output pair provided in the training data 404. Those skilled in the art will recognize, upon reviewing this disclosure in its entirety, various possible variations of at least one supervised machine learning process 428 that may be used to determine the relationship between inputs and outputs. The supervised machine learning process may include a classification algorithm as defined above.
[0127] With further reference to FIG. 4 , training a supervised machine learning process can include iteratively updating coefficients, biases, and weights based on, but not limited to, an error function, an expected loss, and / or a risk function. For example, outputs generated by a supervised machine learning model using example inputs from training examples can be compared to example outputs from the training examples, and an error function can be generated based on the comparison, which may include any error function suitable for use with any machine learning algorithm described in this disclosure, including the squared difference between one or more sets of compared values. Such error functions can then be used to update one or more weights, biases, coefficients, or other parameters of the machine learning model through any suitable process, including, but not limited to, a gradient descent process, a least-squares process, and / or other processes described in this disclosure. This may be done iteratively and / or recursively to gradually adjust the weights, biases, coefficients, or other parameters. The updates may be performed using one or more backpropagation algorithms in neural networks. Iterative and / or recursive updates to weights, biases, coefficients, or other parameters as described above may be performed until the currently available training data is exhausted and / or a convergence test is passed, where a "convergence test" is a test on a condition selected as indicating that the model and / or its weights, biases, coefficients, or other parameters have reached a certain level of accuracy. The convergence test may, for example, compare the difference between two or more consecutive error or error function values, with a difference below a threshold being considered to indicate convergence. Alternatively or additionally, one or more error and / or error function values evaluated in a training iteration may be compared to a threshold.
[0128] Still referring to FIG. 4 , a computing device, processor, and / or module may be configured to perform the methods, method steps, sequences of method steps, and / or algorithms described with reference to this figure in any order and to any degree of repetition. For example, a computing device, processor, and / or module may be configured to repeatedly perform a single step, sequence, and / or algorithm until a desired or ordered result is achieved. The repetition of a step or sequence of steps may be performed iteratively and / or recursively using the output of a previous iteration as input for a subsequent iteration, aggregating the inputs and / or outputs of an iteration to produce an aggregate result, decreasing or decrementing one or more variables such as global variables, and / or dividing a large processing task into a set of smaller processing tasks that are addressed iteratively. The computing device, processor, and / or module may execute any step, sequence of steps, or algorithm in parallel, such as performing a step simultaneously and / or nearly simultaneously two or more times using two or more parallel threads, processor cores, etc., and the division of tasks among parallel threads and / or processes may be performed according to any protocol suitable for dividing tasks among iterations. Those skilled in the art will recognize, upon reviewing this disclosure in its entirety, various ways in which steps, sequences of steps, processing tasks, and / or data may be subdivided, shared, or otherwise processed using iterative, recursive, and / or parallel processing.
[0129] 4, the machine learning process can include at least one unsupervised machine learning process 432. An unsupervised machine learning process, as used herein, is a process that draws inferences in a dataset without regard to labels, such that the unsupervised machine learning process is free to discover any structure, relationships, and / or correlations provided in the data. The unsupervised machine learning process 432 may not require a response variable, and the unsupervised machine learning process 432 can be used to find interesting patterns and / or inferences between variables, determine the degree of correlation between two or more variables, etc.
[0130] Still referring to FIG. 4, the machine learning module 400 can be designed and configured to create the machine learning model 424 using techniques for developing linear regression models. The linear regression model may include ordinary least squares regression, which aims to minimize the squared difference between predicted and actual results according to an appropriate norm (e.g., a vector space distance norm) that measures such difference. The coefficients of the resulting linear equation can be modified to improve the minimization. The linear regression model may include a ridge regression method, in which the function being minimized may include, in addition to the least squares function, a term that multiplies the square of each coefficient by a scalar to penalize large coefficients. The linear regression model may also include a least absolute shrinkage selection operator (lasso) model, in which ridge regression is combined with multiplying the least squares term by a factor of 1 divided by twice the number of samples. The linear regression model may also include a multitasking lasso model, in which the norm applied to the least squares term in the lasso model is the Frobenius norm, which corresponds to the square root of the sum of the squares of all terms. The linear regression model may include an elastic net model, a multitask elastic net model, a least-angle regression model, a LARS lasso model, an orthogonal matching pursuit model, a Bayesian regression model, a logistic regression model, a stochastic gradient descent model, a perceptron model, a passive-aggressive algorithm, a robustness regression model, a Huber regression model, or other suitable models that would occur to one of ordinary skill in the art upon reviewing this disclosure in its entirety. In embodiments, the linear regression model may be generalized to a polynomial regression model, thereby finding a polynomial (e.g., quadratic, cubic, or higher order) that provides the best predicted output / actual output fit. As would be apparent to one of ordinary skill in the art upon reviewing this disclosure in its entirety, methods similar to those described above can be applied to minimize the error function.
[0131] Still referring to FIG. 4 , the machine learning algorithm may include, but is not limited to, linear discriminant analysis. The machine learning algorithm may include quadratic discriminant analysis. The machine learning algorithm may include kernel ridge regression. The machine learning algorithm may include support vector machines, including, but not limited to, regression processes based on support vector classification. The machine learning algorithm may include stochastic gradient descent algorithms, including classification and regression algorithms based on stochastic gradient descent. The machine learning algorithm may include nearest neighbor algorithms. The machine learning algorithm may include various forms of latent space regularization, such as variational regularization. The machine learning algorithm may include Gaussian processes, such as Gaussian process regression. The machine learning algorithm may include cross-decomposition algorithms, including partial least squares and / or canonical correlation analysis. The machine learning algorithm may include naive Bayes methods. The machine learning algorithm may include decision tree-based algorithms, such as decision tree classification and regression algorithms. The machine learning algorithm may include ensemble methods, such as bagging meta-estimators, forests of random trees, AdaBoost, gradient tree boosting, and / or voting classifier methods. The machine learning algorithms may include neural net algorithms, including convolutional neural net processing.
[0132] Still referring to FIG. 4 , the machine learning model and / or process may be deployed or instantiated by incorporation into a program, device, system, and / or module. For example, without limitation, the machine learning model, neural network, and / or some or all of its parameters may be stored and / or deployed in any memory or circuit. Parameters such as coefficients, weights, and / or biases may be stored as circuit-based constants, such as arrays of wires and / or binary inputs and / or outputs set to logic “1” and “0” voltage levels in a logic circuit to represent numbers according to any suitable encoding system, including binary complement, or may be stored in any volatile and / or non-volatile memory. Similarly, mathematical operations and data inputs and / or outputs to or from the model, neural network layers, etc. may be instantiated in hardware circuitry and / or in the form of firmware, machine code such as binary opcode instructions, assembly language, or instructions in any higher-level programming language. To instantiate a machine learning process and / or model, any technique for hardware and / or software instantiation of memory, instructions, data structures, and / or algorithms may be used, which may include, but is not limited to, the manufacture and / or configuration of non-reconfigurable hardware elements, circuits, and / or modules, such as, but not limited to, ASICs; the manufacture and / or configuration of reconfigurable hardware elements, circuits, and / or modules, such as, but not limited to, FPGAs; the manufacture and / or configuration of non-reconfigurable memory elements, circuits, and / or modules, such as, but not limited to, non-reconfigurable ROMs; the manufacture and / or configuration of reconfigurable and / or reconfigurable memory elements, circuits, and / or modules, such as, but not limited to, reconfigurable ROMs or other memory technologies described in this disclosure; and / or any combination of the manufacture and / or configuration of any computing device and / or components thereof, as described in this disclosure.Such deployed and / or instantiated machine learning models and / or algorithms may receive inputs from, and generate outputs for, any other processes, modules, and / or components described in this disclosure.
[0133] 4 , any process of training, retraining, deployment, and / or instantiation of a machine learning model and / or algorithm may be performed and / or repeated after initial deployment and / or instantiation to modify, refine, and / or improve the machine learning model and / or algorithm. Such retraining, deployment, and / or instantiation may be performed as a periodic or periodic process, for example, after a quantity measure such as the number of bytes or other measure of data processed, the number of uses or executions of the processes described in this disclosure, and / or according to a software, firmware, or other update schedule. Alternatively or additionally, retraining, deployment, and / or instantiation may be event-based and may be triggered, without limitation, by user input indicating suboptimal or otherwise problematic performance, and / or by an automated field testing and / or audit process that may compare the output of the machine learning model and / or algorithm, and / or error and / or its error function, to any threshold value, convergence determination, etc., and / or compare the output of the processes described herein to similar threshold values, convergence determinations, etc. Event-based retraining, deployment, and / or instantiation may alternatively or additionally be triggered by the receipt and / or generation of one or more new training examples, where the number of new training examples may be compared to a pre-configured threshold, and exceeding the pre-configured threshold may trigger retraining, deployment, and / or instantiation.
[0134] Still referring to FIG. 4 , retraining and / or additional training may be performed using any of the processes for training described above, using any version of a current or previously deployed machine learning model and / or algorithm as a starting point. Training data for retraining may be collected, preprocessed, sorted, classified, sanitized, or otherwise processed according to any of the processes described in this disclosure. Training data may include, but is not limited to, training examples including inputs and associated outputs used, received, and / or generated from any version of any system, module, machine learning model or algorithm, apparatus, and / or method described in this disclosure; such examples may be modified and / or labeled according to user feedback or other processing to indicate desired results, and / or may have actual or measured results from a process being modeled and / or predicted by a system, module, machine learning model or algorithm, apparatus, and / or method as “desired” results to be compared with outputs for the training process as described above.
[0135] The redeployment may be performed using any reconfiguration and / or rewriting of reconfigurable and / or rewritable circuitry and / or memory elements; alternatively, the redeployment may be performed by fabrication of new hardware and / or software components, circuits, instructions, etc., which may be added to and / or replace existing hardware and / or software components, circuits, instructions, etc.
[0136] 4 , one or more of the processes or algorithms described above may be performed by at least one dedicated hardware unit 436. For purposes of this figure, a “dedicated hardware unit” is a hardware component, circuitry, etc., other than the main control circuitry and / or processor that performs the method steps described in the present disclosure, that is specifically designated or selected to perform one or more particular tasks and / or processes described with reference to this figure, such as, but not limited to, preconditioning and / or sanitizing training data and / or training machine learning algorithms and / or models. The dedicated hardware unit 436 may include hardware units capable of efficiently performing repetitive or intensive calculations, such as, but not limited to, matrix-based calculations that update or adjust parameters, weights, coefficients, and / or biases of machine learning models and / or neural networks, using pipelined processing, parallel processing, etc., and such hardware units may be optimized for such processing by including dedicated circuitry for matrix operations and / or signal processing operations, including, for example, multiple arithmetic and / or logic circuit units, such as multipliers and / or adders, that can operate simultaneously and / or in parallel, etc. Such special-purpose hardware units 436 may include, but are not limited to, graphics processing units (GPUs), special-purpose signal processing modules, FPGAs, or other reconfigurable hardware configured to instantiate parallel processing units for one or more specific tasks. A computing device, processor, apparatus, or module may be configured to direct one or more special-purpose hardware units 436 to perform one or more operations described herein, such as evaluation of model and / or algorithm outputs, one-time or iterative updates of parameters, coefficients, weights, and / or biases, and / or any other operations, such as vector and / or matrix operations, described in this disclosure.
[0137] Referring now to FIG. 5, an exemplary embodiment of a neural network 500 is illustrated. A neural network 500, also known as an artificial neural network, is a network of "nodes" or data structures having one or more inputs, one or more outputs, and a function that determines the output based on the inputs. Such nodes may be organized into networks such as, but not limited to, a convolutional neural network, including an input layer of node 504, one or more hidden layers 508, and an output layer of node 512. Connections between nodes may be created through a process of "training" the network, in which elements from a set of training data are applied to the input nodes, and an appropriate training algorithm (such as Levenberg-Marquardt, conjugate gradient, simulated annealing, or other algorithms) is then used to adjust the connections and weights between nodes in adjacent layers of the neural network to produce desired values at the output nodes. This process is sometimes referred to as deep learning. Connections run only from input nodes to output nodes in "feedforward" networks, while "recurrent networks" allow the output of one layer to be fed back to the input of the same or a different layer.
[0138] Continuing with reference to FIG. 5 , in one embodiment, the neural network 500 can include a deep neural network (DNN). As used in this disclosure, a “deep neural network” is defined as a neural network with two or more hidden layers. In a non-limiting example, the DNN can include a convolutional neural network (CNN). Determining the contaminant presence indicator can include training the CNN using slide image training data as described above and using the trained CNN to determine the quality control metric 160 (e.g., scanning state 172). For purposes of this disclosure, a “convolutional neural network” is a neural network in which at least one hidden layer is a convolutional layer that convolves the input to that layer with a subset of the input known as a “kernel,” along with one or more additional layers, such as a pooling layer, a fully connected layer, etc. In some cases, the CNN can include, but is not limited to, an extension of a deep neural network (DNN). The mathematical (or convolutional) operations performed in a convolutional layer may include convolution of two or more functions, where kernels may be applied to input data (e.g., slide images) through a sliding window approach. In some cases, convolution operations enable a computing device to detect local / global patterns, edges, textures, and other features described herein within the slide images. Features may be passed through one or more activation functions, such as, but not limited to, rectified linear units (ReLUs), to introduce nonlinearity into the determination of quality control metrics 160. Additionally or alternatively, a CNN may include one or more pooling layers, each configured to reduce the dimensionality of the input data while preserving essential features within the input data. In a non-limiting example, a CNN may include one or more pooling layers configured to reduce the dimensionality of a feature map by applying downsampling, such as max pooling or average pooling, to small, non-overlapping regions of one or more features.
[0139] Still referring to FIG. 5, the CNN may further include one or more fully connected layers configured to combine features extracted by the convolutional and pooling layers described above. In some cases, the one or more fully connected layers may enable more advanced pattern recognition. In a non-limiting example, the one or more fully connected layers may connect every neuron (i.e., node) in the input to every neuron in the output and function as a traditional feedforward neural network layer. In some cases, one or more fully connected layers may be used at the end of the CNN to perform high-level inference and generate a final output, such as, but not limited to, the scanning state 172. Furthermore, each fully connected layer may be followed by one or more dropout layers configured to prevent overfitting and one or more normalization layers to stabilize the learning process described herein.
[0140] With continued reference to FIG. 5 , in one embodiment, training the CNN may include selecting an appropriate loss function to guide the training process. In a non-limiting example, a loss function that measures the difference between the predicted scanning state 172 and the ground truth may be used, such as, but not limited to, mean squared error (MSE) or a custom loss function may be designed for one or more embodiments described herein. Additionally or alternatively, an optimization algorithm such as stochastic gradient descent (SGD) may be used to adjust the parameters of the CNN to minimize such loss. In a further non-limiting embodiment, instead of directly predicting the scanning state 172, the CNN may be trained as a regression model to predict a numerical value (e.g., a quality score) associated with the slide. Furthermore, the CNN may be augmented with additional deep learning techniques, such as recurrent neural networks (RNNs) and attention mechanisms, to capture additional features and / or data relationships within the input data.
[0141] 6, an exemplary embodiment of a neural network node 600 is illustrated. The node has multiple inputs x that can receive values from, but are not limited to, inputs to the neural network that contains the node and / or from other nodes. i A node may implement one or more activation functions to generate its output given one or more inputs. Activation functions include, but are not limited to, binary step functions that compare an input to a threshold and output a logic 1 or logic 0 output or the like, linear activation functions where the output is directly proportional to the input, and / or non-linear activation functions where the output is not proportional to the input. Non-linear activation functions include, but are not limited to,
number
number
number
number
number
number
[0142] Referring now to FIG. 7, a flow diagram of an exemplary embodiment of a method 700 for digitizing slides is illustrated. Method 700 includes scanning 705 each of a plurality of slides using at least one imaging device to generate, for each slide of the plurality of slides, an initial slide image and associated scanning metadata. In some embodiments, generating, for each slide of the plurality of slides, an initial slide image and associated scanning metadata may include imaging each of the plurality of slides based on a default scanning profile using at least one imaging device. This may be performed as described above with reference to FIGS. 1-6, without limitation.
[0143] 7, method 700 can include transporting each slide of the plurality of slides between the at least one slide storage unit and the at least one imaging device using a transport component disposed between the at least one imaging device and the at least one slide storage unit. In some cases, the at least one slide storage unit can be configured to store multiple slides. This can be performed, without limitation, as described above with reference to FIGS. 1-6.
[0144] Still referring to FIG. 7 , method 700 includes step 710 of determining, by a computing device, for each slide of the plurality of slides a quality control metric as a function of scanning metadata associated with the initial slide image, where determining the quality control metric includes flagging the slide based on the determined quality control metric. In some embodiments, flagging the slide may include classifying, by the computing device, scanning metadata associated with the initial slide image into a scanning state using a slide image classifier, and flagging the slide as a function of the scanning state. In some embodiments, determining the quality control metric may include generating, by the computing device, quality control metadata as a function of the quality control metric, and modifying, by the computing device, the quality control metadata as a function of user input received from a user. This may be performed, without limitation, as described above with reference to FIGS. 1-6 .
[0145] Continuing with reference to FIG. 7 , method 700 includes step 715 of generating, by a computing device, at least one scanning profile as a function of the scanning metadata based on the quality control metric. In some embodiments, generating the at least one scanning profile may include identifying, by the computing device, at least one missing element from an initial slide image of the flagged slide as a function of the quality control metric; selecting, by the computing device, a plurality of scanning parameters configured to address the at least one missing element; and generating, by the computing device, the at least one scanning profile as a function of the plurality of scanning parameters. In some cases, selecting the plurality of scanning parameters may include training, by the computing device, a scanning parameter selection model using scanning parameter training data, where the scanning parameter training data may include a plurality of missing elements as inputs correlated to a plurality of scanning parameter sets as outputs, and selecting, by the computing device, a plurality of scanning parameters as a function of the at least one missing element using the trained scanning parameter selection model. In some cases, the plurality of scanning parameters may include a plurality of imaging device configuration parameters. This may be done, but is not limited to, as described above with reference to FIGS.
[0146] Continuing with reference to FIG. 7, method 700 includes step 720 of rescanning flagged slides within the plurality of slides at the imaging device(s) using at least one scanning profile at the at least one imaging device. In some embodiments, rescanning the flagged slides may include scanning the identified missing elements at the at least one imaging device using at least one scanning profile to generate slide image fragments for the flagged slides. This may be performed as, but is not limited to, described above with reference to FIGS. 1-6.
[0147] 7, method 700 may further include generating, by the computing device, a subsequent slide image for each flagged slide by merging the initial slide image with slide image fragments of the flagged slide using at least one scanning profile, which may be performed as, but is not limited to, described above with reference to FIGS.
[0148] It should be noted that any one or more of the aspects and embodiments described herein may be suitably implemented using one or more machines (e.g., one or more computing devices utilized as user computing devices for electronic documents, one or more server devices, such as document servers) programmed in accordance with the teachings herein, as would be apparent to those skilled in the computer arts. Appropriate software coding may be readily accomplished by skilled programmers based on the teachings of the present disclosure, as would be apparent to those skilled in the software arts. The above-described aspects and implementations employing software and / or software modules may also include appropriate hardware to assist in implementing the machine-executable instructions of the software and / or software modules.
[0149] Such software may be a computer program product employing a machine-readable storage medium. A machine-readable storage medium may be any medium capable of storing and / or encoding a sequence of instructions for execution by a machine (e.g., a computing device) and causing the machine to perform any one of the methodologies and / or embodiments described herein. Examples of machine-readable storage media include, but are not limited to, magnetic disks, optical disks (e.g., CDs, CD-Rs, DVDs, DVD-Rs, etc.), magneto-optical disks, read-only memory "ROM" devices, random-access memory "RAM" devices, magnetic cards, optical cards, solid-state memory devices, EPROMs, EEPROMs, and any combination thereof. As used herein, machine-readable medium is intended to include not only a single medium but also a collection of physically separate media, such as, for example, a collection of compact discs, one or more hard disk drives combined with computer memory, etc. As used herein, machine-readable storage medium does not include a transitory form of signal transmission.
[0150] Such software may also include information (e.g., data) carried in a data signal on a data carrier such as a carrier wave. For example, the machine-executable information may be included in a data carrier signal embodied in a data carrier, the signal encoding a sequence of instructions, or portions thereof, for execution by a machine (e.g., a computing device), and any associated information (e.g., data structures and data) that cause the machine to perform any one of the methodologies and / or embodiments described herein.
[0151] Examples of computing devices include, but are not limited to, e-book reading devices, computer workstations, terminal computers, server computers, handheld devices (e.g., tablet computers, smartphones, etc.), web appliances, network routers, network switches, network bridges, any machine capable of executing a sequence of instructions that specify actions to be performed by that machine, and any combination thereof. In one example, a computing device may include and / or be included in a kiosk.
[0152] 8 shows a diagrammatic representation of one embodiment of a computing device in the exemplary form of a computer system 800 upon which a set of instructions may be executed that causes a control system to perform any one or more of the aspects and / or methodologies of the present disclosure. It is also contemplated that multiple computing devices may be utilized to execute a set of instructions specifically configured to cause one or more of the devices to perform any one or more of the aspects and / or methodologies of the present disclosure. Computer system 800 includes a processor 804 and memory 808 that communicate with each other and with other components via a bus 812. Bus 812 may include any of several types of bus structures, including, but not limited to, a memory bus, a memory controller, a peripheral bus, a local bus, and any combination thereof, using any of a variety of bus architectures.
[0153] Processor 804 may include any suitable processor, such as, but not limited to, a processor incorporating logic circuitry for performing arithmetic and logical operations, such as an arithmetic logic unit (ALU), which may be controlled by a state machine and directed by operational input from memory and / or sensors. Processor 804 may be configured according to, by way of non-limiting example, the von Neumann architecture and / or the Harvard architecture. Processor 804 may include, incorporate, and / or be incorporated into, without limitation, a microcontroller, microprocessor, digital signal processor (DSP), field programmable gate array (FPGA), complex programmable logic device (CPLD), graphical processing unit (GPU), general-purpose GPU, tensor processing unit (TPU), analog or mixed signal processor, trusted platform module (TPM), floating point unit (FPU), system-on-module (SOM), and / or system-on-chip (SoC).
[0154] Memory 808 may include a variety of components (e.g., machine-readable media), including, but not limited to, random-access memory components, read-only components, and any combination thereof. In one example, a basic input / output system 816 (BIOS), containing the basic routines that help to transfer information between elements within computer system 800, such as during start-up, may be stored in memory 808. Memory 808 may also include (e.g., stored on one or more machine-readable media) instructions (e.g., software) 820 that embody any one or more of the aspects and / or methodologies of the present disclosure. In another example, memory 808 may further include any number of program modules, including, but not limited to, an operating system, one or more application programs, other program modules, program data, and any combination thereof.
[0155] Computer system 800 may also include a storage device 824. Examples of storage devices (e.g., storage device 824) include, but are not limited to, hard disk drives, magnetic disk drives, optical disk drives in combination with optical media, solid-state memory devices, and any combination thereof. Storage device 824 may be connected to bus 812 by an appropriate interface (not shown). Examples of interfaces include, but are not limited to, SCSI, Advanced Technology Attachment (ATA), Serial ATA, Universal Serial Bus (USB), IEEE 1394 (FIREWIRE®), and any combination thereof. In one example, storage device 824 (or one or more components thereof) may be removably interfaced with computer system 800 (e.g., via an external port connector (not shown)). In particular, storage device 824 and associated machine-readable media 828 may provide nonvolatile and / or volatile storage of machine-readable instructions, data structures, program modules, and / or other data for computer system 800. In one example, the software 820 may reside, completely or partially, within the machine-readable medium 828. In another example, the software 820 may reside, completely or partially, within the processor 804.
[0156] Computer system 800 may also include input devices 832. In one example, a user of computer system 800 can input commands and / or other information into computer system 800 via input devices 832. Examples of input devices 832 include, but are not limited to, alphanumeric input devices (e.g., keyboards), pointing devices, joysticks, gamepads, audio input devices (e.g., microphones, voice response systems, etc.), cursor control devices (e.g., mice), touchpads, optical scanners, video capture devices (e.g., still cameras, video cameras), touch screens, and any combination thereof. Input devices 832 may interface with bus 812 via any of a variety of interfaces (not shown), including, but not limited to, a serial interface, a parallel interface, a game port, a USB interface, a FIREWIRE interface, a direct interface to bus 812, and any combination thereof. Input devices 832 may include a touchscreen interface, which may be part of or separate from display 836, as described below. Input devices 832 may be utilized as a user selection device for selecting one or more graphical representations in a graphical interface, as described above.
[0157] A user may also input commands and / or other information into computer system 800 via storage device 824 (e.g., a removable disk drive, flash drive, etc.) and / or network interface device 840. A network interface device, such as network interface device 840, may be utilized to connect computer system 800 to one or more of various networks, such as network 844, and one or more remote devices 848 connected thereto. Examples of network interface devices include, but are not limited to, a network interface card (e.g., a mobile network interface card, a LAN card), a modem, and any combination thereof. Examples of networks include, but are not limited to, a wide area network (e.g., the Internet, an enterprise network), a local area network (e.g., a network associated with an office, building, campus, or other relatively small geographic space), a telephone network, a data network associated with a telephone / voice provider (e.g., a mobile communications provider's data and / or voice network), a direct connection between two computing devices, and any combination thereof. A network, such as network 844, may employ wired and / or wireless communication modes. In general, any network topology may be used. Information (eg, data, software 820 , etc.) may be communicated to and / or from computer system 800 via network interface device 840 .
[0158] Computer system 800 may further include a video display adapter 852 that communicates displayable images to a display device, such as display device 836. Examples of display devices include, but are not limited to, a liquid crystal display (LCD), a cathode ray tube (CRT), a plasma display, a light emitting diode (LED) display, and any combination thereof. Display adapter 852 and display device 836 can be utilized in combination with processor 804 to provide graphical representations of aspects of the present disclosure. In addition to a display device, computer system 800 may include one or more other peripheral output devices, including, but not limited to, audio speakers, a printer, and any combination thereof. Such peripheral output devices may be connected to bus 812 via peripheral interface 856. Examples of peripheral interfaces include, but are not limited to, a serial port, a USB connection, a FIREWIRE connection, a parallel connection, and any combination thereof.
[0159] The foregoing is a detailed description of exemplary embodiments of the present invention. Various modifications and additions may be made without departing from the spirit and scope of the present invention. Features of each of the various embodiments described above may be combined with features of other described embodiments as appropriate to provide various combinations of features in related new embodiments. Moreover, while a number of separate embodiments have been described above, what has been described herein is merely illustrative of the application of the principles of the present invention. Furthermore, while certain methods herein may be illustrated and / or described as being performed in a particular order, such orders may be varied considerably within the skill of ordinary skill in order to implement methods, systems, and software according to the present disclosure. Accordingly, the present description is intended to be illustrative only, and not to otherwise limit the scope of the present invention.
[0160] Exemplary embodiments are disclosed above and illustrated in the accompanying drawings. Those skilled in the art will appreciate that various modifications, omissions, and additions may be made to what is specifically disclosed herein without departing from the spirit and scope of the invention.
Claims
1. 1. A system for digitizing slides, said system comprising: at least one imaging device; a computing device communicatively connected to the imaging device; Including, the computing device, scanning each slide of the plurality of slides with the imaging device to generate an initial slide image and associated scanning metadata for each slide of the plurality of slides; determining a quality control metric for each slide of the plurality of slides as a function of the initial slide image and the associated scanning metadata; Determining the quality control metric comprises: flagging slides based on said determined quality control metrics; generating at least one scanning profile based on said quality control metrics as a function of said scanning metadata; Rescanning flagged slides within the plurality of slides at the imaging device using the at least one scanning profile. The system is configured as follows:
2. At least one slide storage unit that stores the plurality of slides; a transport component disposed between the at least one imaging device and the at least one slide storage section, the transport component comprising: Transporting each slide of the plurality of slides between the at least one slide storage section and the at least one imaging device. a transport component configured to The system of claim 1 further comprising:
3. generating the initial slide image and associated scanning metadata for each slide of the plurality of slides includes: The system of claim 1 , further comprising imaging each slide of the plurality of slides using the at least one imaging device based on a default scanning profile.
4. Flagging the slide may include: classifying, by the computing device, the initial slide image and the associated scanning metadata into a scanning state using a slide image classifier; flagging, by the computing device, the slides as a function of the scanning status; The system of claim 1 , comprising:
5. Determining the quality control metric comprises: generating, by the computing device, quality control metadata as a function of the quality control metric; modifying, by the computing device, the quality control metadata as a function of user input received from a user; The system of claim 1 , comprising:
6. Generating the at least one scanning profile includes: identifying, by the computing device, at least one missing element from the initial slide image of the flagged slide as a function of the quality control metric; selecting, by the computing device, a plurality of scanning parameters configured to address the at least one missing element; generating, by the computing device, at least one scanning profile as a function of a plurality of scanning parameters; The system of claim 1 , comprising:
7. The system of claim 6 , wherein the plurality of scanning parameters comprises a plurality of imaging device configuration parameters.
8. Selecting the plurality of scanning parameters includes: training, by the computing device, a scanning parameter selection model using scanning parameter training data, the scanning parameter training data including a plurality of missing elements as inputs correlated to a plurality of scanning parameter sets as outputs; selecting, by the computing device, the plurality of scanning parameters as a function of the at least one missing element using the trained scanning parameter selection model; The system of claim 6 , comprising:
9. Rescanning the flagged slides comprises: scanning the identified missing elements using the at least one scanning profile at the at least one imaging device to generate a slide image fragment for the flagged slide; generating a subsequent slide image for each flagged slide by merging the initial slide image with the slide image fragment of the flagged slide using the at least one scanning profile; The system of claim 6 , comprising:
10. Rescanning the flagged slides comprises: selecting at least one imaging device from a plurality of imaging devices based on the plurality of imaging device configuration parameters; scanning each flagged slide with the selected imaging device using the at least one scanning profile; The system of claim 7, comprising:
11. 1. A method for digitizing a slide, comprising: scanning each slide of the plurality of slides using at least one imaging device to generate, for each slide of the plurality of slides, an initial slide image and associated scanning metadata; determining, by a computing device, for each slide of the plurality of slides, a quality control metric as a function of the initial slide image and the associated scanning metadata; Determining the quality control metric comprises: flagging the slide based on the determined quality control metric; generating, by a computing device, at least one scanning profile as a function of the scanning metadata based on the quality control metrics; rescanning, at at least one imaging device, flagged slides within the plurality of slides at the imaging device using the at least one scanning profile; A method comprising:
12. 12. The method of claim 11, further comprising transporting each slide of the plurality of slides between the at least one slide storage unit and the at least one imaging device using a transport component disposed between the at least one imaging device and the at least one slide storage unit.
13. generating the initial slide image and associated scanning metadata for each slide of the plurality of slides includes: imaging each slide of the plurality of slides based on a default scanning profile using the at least one imaging device. The method of claim 11 , comprising:
14. Flagging the slide may include: classifying, by the computing device, the initial slide image and the associated scanning metadata into a scanning state using a slide image classifier; flagging, by the computing device, the slides as a function of the scanning status; The method of claim 11 , comprising:
15. Determining the quality control metric comprises: generating, by the computing device, quality control metadata as a function of the quality control metric; modifying, by the computing device, the quality control metadata as a function of user input received from a user; The method of claim 11 , comprising:
16. Generating the at least one scanning profile includes: identifying, by the computing device, at least one missing element from the initial slide image of the flagged slide as a function of the quality control metric; selecting, by the computing device, a plurality of scanning parameters configured to address the at least one missing element; generating, by the computing device, at least one scanning profile as a function of a plurality of scanning parameters; The method of claim 11 , comprising:
17. The method of claim 16 , wherein the plurality of scanning parameters comprises a plurality of imaging device configuration parameters.
18. Selecting the plurality of scanning parameters includes: training, by the computing device, a scanning parameter selection model using scanning parameter training data, the scanning parameter training data including a plurality of missing elements as inputs correlated to a plurality of scanning parameter sets as outputs; selecting, by the computing device, the plurality of scanning parameters as a function of the at least one missing element using the trained scanning parameter selection model; 17. The method of claim 16, comprising:
19. Rescanning the flagged slides comprises: scanning the identified missing elements using the at least one scanning profile at the at least one imaging device to generate a slide image fragment for the flagged slide; generating, by the computing device, a subsequent slide image for each flagged slide by merging the initial slide image with the slide image fragment of the flagged slide using the at least one scanning profile; 17. The method of claim 16, comprising:
20. Rescanning the flagged slides comprises: selecting, by the computing device, the at least one imaging device from a plurality of imaging devices based on the plurality of imaging device configuration parameters; scanning each flagged slide with the selected imaging device using the at least one scanning profile; 18. The method of claim 17, comprising:
Citation Information
Patent Citations
Method and apparatus for evaluating slide and sample preparation quality
JP2001502414A
System and method for determining the slide quality of digitized microscope slides
JP2013535048A
System equipped with a digital microscope and method for inspecting samples using the same
JP2013536471A
Image acquisition device and control method of image acquisition device
JP2016125913A
Real-time focusing in a slide scanning system
JP2022542625A