Systematic Characterization of Objects in Biological Samples

Abstract

The present invention relates to a method for classifying and counting recoverable objects from a urine sample processed onto a slide, comprising the steps of receiving at least one digitized image of the entire slide; detecting linkage components by segmentation of the image of the entire slide; classifying the detected linkage components into countable linkage components and uncountable linkage components using a classifier; in the case of countable linkage components, obtaining the number of objects of each class using an object detection model; in the case of uncountable components, obtaining the number of objects of each class using a semantic segmentation model; and summing the number of objects of each class obtained from the semantic segmentation model and the object detection model, and outputting the number of objects of each class.

Description

Technical Field

[0001] Field of the Invention The present invention relates particularly to the field of biological analysis for the detection of urological conditions. More particularly, the present invention relates to a method for providing accurate information regarding the assessment of the characteristics of at least one, preferably a plurality of objects that can be found in a biological fluid sample of a subject, particularly a urine sample.

[0002] This invention preferably relates to a method and system for analyzing relevant objects in a biological fluid sample, which is preferably a urine sample. The method and system of the present invention can be useful, for example, for detecting, identifying and / or counting objects in at least one sample containing cells or bacteria.

[0003] This invention also relates to the presentation of reliable information resulting from a systematic investigation and characterization of objects in a biological sample, particularly a urine sample.

Background Art

[0004] Background of the Invention Sample analysis is one of the most common tests for providing an overview of a subject's health status. The presence of certain characteristic objects in a sample can be clinically significant and / or indicative of a pathological condition in the subject.

[0005] Urine samples can contain various objects such as cell clumps, aggregates or sheets; blood cells, e.g., erythrocytes or red blood cells (RBC); urothelial cells, particularly atypical urothelial cells; crystals; lymphocytes, leukocytes or white blood cells (WBC); neutrophils, monocytes, eosinophils; or microorganisms such as bacteria.

[0006] A common method for performing urine sediment analysis is to deposit urine cells present in a sample onto a microscope slide using brightfield optical technology and then digitize the slide. However, despite taking care when depositing cells onto the slide, irregular layers of cells, clumps, and other cell stacks may appear on the slide.

[0007] This problem has already been raised in the prior art.

[0008] As an example of cell analysis in a prior art sample, FR2996036 is a method for identifying cells in a biological tissue, where each cell contains cytoplasm separated by a cell membrane, and the following steps: a) obtaining a raw digital image of a section of the biological tissue containing the cells to be identified, the image containing a plurality of pixels with different intensity values; b) automatically generating at least one threshold image; c) in the threshold image, automatically searching for a surface formed from a plurality of consecutive pixels having the same defined intensity value, and the cell surface thus found constituting a cell candidate. According to the present invention, the defined intensity values are the first and second intensity values corresponding to the cytoplasm.

[0009] However, this method is not accurate for samples with cell overlaps, stacks, deposits, and / or clumps.

[0010] WO2015102948 aims to improve the classification accuracy of urinary components by calculating the difference between the average pixel value of the peripheral region and the average pixel value of the central region after removing the influence of ambient light and defocus noise.

[0011] However, this technique is not useful for processing cell clumps.

[0012] WO2015168365 is a method for processing a block to be processed in a urine sediment image, comprising the following steps: - A step of approximating the color of pixels within a block to be processed to one of the kc colors in a codebook (the codebook is a set of kc colors generated from a set of urine sample blocks); - A step of obtaining a distribution histogram of the number of pixels of the color approximation results corresponding to each of the kc colors; - A step of correcting the number of pixels of the color approximation results corresponding to each color in the distribution histogram using an appearance frequency correction coefficient; - A step of normalizing the corrected number of pixels of the color approximation results corresponding to each color in the distribution histogram; and obtaining the normalized distribution histogram as a feature of the feature set of block processing, and processing the block to be processed describes a method including.

[0013] This method is particularly useful when the block is well-defined, but accurate results may not be obtained when the image is blurred or not fully focused on the block in question.

[0014] Furthermore, this method requires steps of data augmentation and preprocessing including at least one of rotation, scaling, translation, trimming, mirroring, and elastic deformation.

[0015] Consequently, in order to propose a method for urine sample screening suitable for screening large populations, especially populations at risk of developing urological pathologies related to age, smoking habits, or exposure to industrial substances, it is necessary to find a cost-effective, simple, reliable, and reproducible method and system that can handle heavy images obtained by brightfield imaging.

[0016] This is because the present invention relates to a method that uses a cost-effective, reproducible, and accurate means to count and instantaneously display the number of objects in a urine sample by calculating a digitized image, thereby satisfying the above unmet needs. The present invention also includes a process of quickly operating a large number of samples one after another.

Summary of the Invention

[0017] The present invention is a method for classifying and counting objects recoverable from a urine sample processed on a slide, comprising the following steps: - receiving at least one digitized image of the entire slide; - detecting connected components by segmenting the image of the entire slide; - classifying the detected connected components into countable and non - countable connected components using a classifier; - in the case of countable connected components: · inputting each countable connected component into an object detection model to detect objects and obtain an output including the bounding box and associated class of each detected object; · counting the bounding boxes associated with each class to obtain the number of objects in each class; - in the case of non - countable components: · inputting each non - countable connected component into a semantic segmentation model and obtaining, as an output, a segmentation mask in which all pixels are classified into one of the available classes with defined classes; · for each object class, counting the number of objects as the ratio of the total pixel area of the class, obtained as the number of pixels of the segmentation mask associated with the class, to the average area of the objects of the class; wherein the classes of the semantic segmentation model and the object detection model are the same; - summing the number of objects in each class obtained from the semantic segmentation model and the object detection model; - outputting the number of objects in each class relates to a method comprising the above.

[0018] Conveniently, the method of the present invention enables accurate counting of objects even when the object boundaries are difficult to detect due to out-of-focus images or object thickness on the slide, and in cases where some parts of the object may be out-of-focus and the image may become blurry / noisy, either because the object is actually a single object or overlapping objects have significant thickness. This method significantly reduces inaccuracies due to inaccurate segmentation.

[0019] In one embodiment, the semantic segmentation model is a U-Net.

[0020] According to one embodiment, the object detection neural network is Faster-RCNN, CenterNet, SOLO or YOLO.

[0021] According to one embodiment, the received digitized image results from a brightfield optical system.

[0022] According to one embodiment, the model is trained using a dataset of labeled digitized images.

[0023] In one embodiment, the semantic segmentation model and the object detection model are trained using a dataset of digitized images labeled by a clinician.

[0024] In one embodiment, the semantic segmentation model and the object detection model are trained using a probabilistic gradient descent training method. The probabilistic gradient descent optimization method conveniently enables saving of computation time in every optimization step.

[0025] This consists of replacing the actual gradient (computed from the entire training dataset) with an estimate (computed from a randomly selected subset of the data). This is very effective in cases of large-scale machine learning problems such as those in the present invention.

[0026] In one embodiment, each class is the following list: - White blood cells: basophils, neutrophils, macrophages, monocytes, and eosinophils; - Red blood cells; - Bacteria; - Urinary crystals; - Cylinders; - Healthy and atypical urothelial cells; - Squamous epithelial cells; - Reactive urothelial cells - Yeast and is related to at least one of the objects in.

[0027] According to one embodiment, the method further includes the step of displaying the total number of objects of each class along a digitalized image of at least one portion of the entire slide.

[0028] According to one embodiment, the method further includes the step of displaying at least one string providing information regarding the presence or absence of objects within a class in the digitalized image.

[0029] In one embodiment, the sample is stained with Papanicolaou (or Pap) stain or any other polychromatic cytological stain known to those skilled in the art.

[0030] The present invention is also a system for classifying and counting objects recoverable from a urine sample processed on a slide, - at least one input adapted to receive at least one digitalized image of the entire slide including a plurality of objects; - at least one processor configured as follows: · Detect connected components by segmenting the image of the entire slide; · Classify the detected connected components into countable and uncountable connected components using a classifier; · In the case of countable connected components: i. Input each countable connected component into an object detection model configured to detect objects and output one bounding box and a related class for each object; ii. Count the bounding boxes related to each class to obtain the number of objects in each class; · For uncountable components: i. Input each uncountable connected component into a semantic segmentation model and obtain, as an output, a segmentation mask in which all pixels are classified into one of the available classes for which definitions exist; ii. For each class, count the number of objects as the ratio of the total pixel area of the class obtained as the number of pixels of the segmentation mask related to the class to the average area of the objects of the class; · Sum the number of objects in each class obtained from the semantic segmentation model and the object detection model; wherein the classes of the semantic segmentation model and the object detection model are the same; - at least one output adapted to provide the number of objects in each class; also relates to a system comprising the same.

[0031] In an equivalent manner, the system comprises - an acquisition module configured to receive at least one digitized image of an entire slide containing a plurality of objects; - a calculation module configured as follows: · Detect connected components by segmenting the image of the entire slide; · Classify the detected connected components into countable and uncountable connected components using a classifier; · For countable connected components: ■ Input each countable connected component into an object detection model configured to detect objects and output one bounding box and a related class for each object; ■ Count the bounding boxes related to each class to obtain the number of objects in each class · In the case of non - countable combination components: ■ Input each non - countable combination component into the semantic segmentation model, and obtain as an output a segmentation mask in which all pixels are classified into one of the defined available classes; ■ For each class, count the number of objects as the ratio of the total pixel area of the class, obtained as the number of pixels of the segmentation mask related to the class, to the average area of the objects of the class; At this time, the classes of the semantic segmentation model and the object detection model are the same, - Sum the number of objects of each class obtained from the semantic segmentation model and the object detection model; - An output module configured to output the number of objects of each class, may be included.

[0032] The present invention also relates to a computer program product for classifying and counting objects recoverable from a urine sample, the program including instructions for causing a computer to execute the steps of the method according to any one of the above - described embodiments when the program is executed by the computer.

[0033] The present invention also relates to a computer - readable storage medium including instructions for causing a computer to execute the steps of the method according to any one of the above - described embodiments when the program is executed by the computer.

[0034] Definitions In the present invention, the following terms have the following meanings: · "Atypical" for cells means having at least one characteristic of cells not reported in non - pathological situations. · "Atypical urothelial cells" (AUC) are defined herein with reference to the Paris System for Reporting Urinary Cytology (TPSRUC). · A "bright field optical system" refers to an imaging technique in which an image is generated by uniformly illuminating the entire sample, resulting in the specimen appearing as a dark image against a brightly illuminated background. Bright field imaging is used as a common imaging technique for observing and examining samples. · "Classifying" refers to classifying objects in a target sample into different classes of interest, such as red blood cells. · A "connected component" refers to one object or a group of objects where all pixels share similar pixel intensity values and are connected to each other in some way. An example of a connected component is shown in Figure 2. · "Counting" refers to enumerating the number of objects in each class of interest in a target sample. · In the context of the present invention, a "countable connected component" refers to a group of components that can be identified by a trained physician as individual objects, and thus the physician can count the number of objects included in the group (i.e., white blood cells, cells, bacteria, etc.). On the other hand, an "uncountable connected component" refers to the opposite situation with respect to countable connected components, where a trained physician can identify a group of objects but cannot identify individual objects within the group. Examples of countable and uncountable components are provided in Figure 2. The objective measure of a connected component is typically defined by a panel of trained physicians, and this measure is similar to a standard and is accepted by those skilled in the art of cytological image analysis. · A "dataset" refers to a collection of data used to build a mathematical model of machine learning (ML) for data-driven prediction or decision-making. In supervised learning (i.e., inferring a function from known input-output examples in the form of labeled training data), three types of ML datasets (also designated as ML sets) are typically for three respective types of tasks, namely training, i.e., fitting parameters, validation, i.e., tuning ML hyperparameters (parameters used to control the learning process), and testing, i.e., dedicated to confirming that the latter model provides satisfactory results, independent of the training dataset used to build the mathematical model. · A "neural network or artificial neural network (ANN)" designates a category of ML that includes nodes (called neurons) and connections between neurons modeled by weights. For each neuron, the output is given by a function of the inputs or a set of inputs by an activation function. Neurons are generally organized in multiple layers, so that neurons in one layer connect only to neurons in the immediately preceding and succeeding layers. · "YOLO", i.e., "You Only Look Once", refers to a single convolutional network whose architecture is specifically configured for object detection. · "SOLO" (Segmenting Objects by Locations) refers to a type of artificial neural network (ANN) that combines self-organizing feature maps (SOFM), principal component analysis, and multivariate linear regression to generate a network architecture that gives robust, stable, and high-quality predictions. · "Faster R-CNN" refers to an object detection model belonging to the family of two-stage object detectors. The two stages of Faster R-CNN correspond to two neural networks respectively. The first one is called the Region Proposal Network (RPN) and outputs a set of bounding box candidates. The second one refines the coordinates of the bounding boxes and classifies the bounding boxes into predefined classes. · The term "processor" should not be construed as limited to hardware capable of executing software, but rather generally refers to a processing device that can include, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device (PLD). A processor can also include one or more graphics processing units (GPUs), regardless of whether they are utilized for computer graphics and image processing or other functions. Further, instructions and / or data that enable the execution of related functions and / or functions resulting therefrom can be stored on any processor-readable medium, such as, for example, an integrated circuit, a hard disk, an optical disk such as a CD (compact disc) or a DVD (digital versatile disc), a RAM (random access memory), or a ROM (read only memory). The instructions can be stored, in particular, in hardware, software, firmware, or any combination thereof. · "Semantic segmentation": Refers to an algorithm configured to individually classify each pixel of an image into predefined classes. · The terms "adapted" and "configured" are used in this disclosure to broadly encompass, likewise, the initial configuration of the device, subsequent adaptation or complementation, or any combination thereof, whether brought about via material or software means (including firmware).

[0035] Brief Description of the Drawings The features and advantages of the present invention will appear in the following description. Some modes of realization of the apparatus and method will be described with respect to the present invention.

Brief Description of the Drawings

[0036]

Figure 1

Figure 2

Best Mode for Carrying Out the Invention

[0037] Detailed Description The present invention relates to a cost-effective high-throughput method for screening biological samples, particularly urine samples processed on slides. More precisely, the method of the present invention aims to identify and count objects present in a biological sample of interest.

[0038] A urine sample is obtained from a subject. The sample may also be another body fluid such as blood, plasma, serum, lymph, ascitic fluid, cyst fluid, urine, bile, nipple exudate, synovial fluid, bronchoalveolar lavage fluid, sputum, amniotic fluid, peritoneal fluid, cerebrospinal fluid, pleural fluid, pericardial fluid, semen, saliva, sweat, feces, stool, and alveolar macrophages. The sample may be concentrated or enriched.

[0039] In one embodiment, the method of the present invention does not include obtaining a sample from a subject. In one embodiment, the sample of the subject is a sample previously obtained from the subject. The sample may be stored under appropriate conditions before being used according to the method of the present invention.

[0040] The sample can be collected from a healthy or unhealthy subject presenting tumor-like cells or at risk of developing uropathology. The method of the present invention is designed to be applied to a large number of subjects.

[0041] In one embodiment, the sample is homogenized, deposited on a filter, and then contacted with a glass slide to deposit cells therein. The material of the slide is preferably glass, but can also be other materials such as polycarbonate, for example. The material may be a single-use material.

[0042] The slide deposit is stained according to the Papanicolaou staining protocol to detect morphological changes of cells that are indicators of cancer risk. Alternatively, different staining means may be used together.

[0043] After staining, cover the slide. The slide may be covered with, for example, a cover glass or a plastic film.

[0044] According to one embodiment, an image of a slide of a urine sample is obtained from a bright field optical system such as a whole slide scanner.

[0045] For the scanning step, the attached slide may be digitized using any suitable bright field optical system, such as, for example, a Hamamatsu Nanozoomer-S60 slide scanner. Data acquisition can also be realized with a Hamamatsu Nanozoomer-S360 slide scanner or a 3DHistech P250 or P1000.

[0046] The digitized image of the slide may be rectangular. The digitized image to be analyzed may be trimmed to define a target area in which each target area is subjected to analysis. Areas with high sensitivity can be divided within the target area to improve the accuracy of the analysis.

[0047] The embodiments disclosed herein include the various operations described in this specification. The operations may be performed by hardware components and / or may be embodied in machine-executable instructions, which may be used to cause a general-purpose or special-purpose processor programmed with the instructions to perform the operations. Alternatively, the operations may be performed by a combination of hardware, software, and / or firmware.

[0048] The performance of one or more of the operations described herein may be distributed among one or more processors, and may exist not only within a single machine but also be arranged in a number of machines. In some cases, one or more processors or processor-implemented modules may be arranged at a single geographical location (e.g., within a home environment, an office environment, or a server farm). In other embodiments, one or more processors or processor-implemented modules may be distributed among a number of geographical locations.

[0049] As shown in FIG. 1, according to one embodiment, the first step 110 of method 100 consists of receiving at least one digitalized image of the entire slide or at least one part of the entire slide.

[0050] In one embodiment, the method further consists of step 120 by segmenting the image of the entire slide or at least one part of the entire slide, which is composed of connected components. The segmentation method can be a threshold-based method that enables separating the foreground from the background. The foremost connected component may be recovered from the segmentation mask. The connected component is composed of one object or a group of objects.

[0051] In one embodiment, the detected connected components are classified into countable connected components and uncountable connected components (step 130) using a classifier such as, for example, a convolutional neural network. Countable connected components are connected components in which each object contained therein can be visually identified by a human. Conversely, uncountable connected components are objects that cannot be visually identified by a human. Examples of these countable and uncountable connected components are provided in FIG. 2.

[0052] According to one embodiment, the method of the present invention includes, in the case of countable connected components, step 140 of inputting each countable connected component into an object detection model. The object detection model is configured to detect an object composed of countable connected components and obtain, as an output, one bounding box and a related class for each detected object, and a class is selected from among the defined available classes. Therefore, this step includes outputting a bounding box and one related class for each object detected from the object detection model.

[0053] According to one embodiment, the object detection model is Faster-RCNN, CenterNet, SOLO or YOLO.

[0054] In Faster R-CNN, an image is provided as input to a convolutional network that provides a convolutional feature map. Faster R-CNN consists of two modules. The first module is a deep fully convolutional network that proposes regions, and the second module is the Faster R-CNN detector that uses the proposed regions. The whole system is a single unified network for object detection. A neural network with an "attention" mechanism, the RPN module for generating region proposals. The main difference from Faster R-CNN is that the latter generates region proposals using selective search. In RPN, the time cost for generating region proposals is much smaller than selective search when the RPN shares the most computations with the object detection network. Briefly explained, RPN ranks region boxes (called anchors) and proposes the boxes with the highest probability of containing an object.

[0055] Unlike the other two object detection models (CenterNet, YOLO), Faster R-CNN is a two-stage object detector, meaning that the bounding boxes are first proposed and then refined. In a one-stage detector, the bounding boxes are not refined. Therefore, the performance of this model is usually better than that of single-stage object detectors.

[0056] CenterNet detects each object as a triplet rather than a pair of keypoints, improving both accuracy and recall. CenterNet explores the central part of the proposal, i.e., the region close to the geometric center, with one additional keypoint. The architecture of CenterNet includes a convolutional backbone network that applies cascade corner pooling and center pooling to two corner heatmaps and a center keypoint heatmap, respectively. Similar to CornerNet, potential bounding boxes are detected using embeddings similar to the detected pairs of corners. Next, the detected center keypoints are used to determine the final bounding boxes. The advantages of CenterNet over other models are usually easy implementation, fast training, and high speed during inference time.

[0057] YOLO uses fewer anchor boxes (divides the input image into an S×S grid) for regression and classification. More specifically, YOLO is a network "triggered" by GoogleNet. It has 24 convolutional layers that function as a feature extractor and two fully connected layers for prediction. The architecture of the feature extractor is called Darknet. In summary, the input image is fed into a feature extractor (Darknet) that outputs a feature map of shape S×S. Therefore, the image is divided into a grid of S×S cells. Each cell of the feature map is fed into a block of two consecutive fully connected layers that predict B bounding boxes with confidence scores and class probabilities for K classes. The confidence scores are given from the perspective of the IOU (intersection over union) metric, which basically measures how much the detected object overlaps with the ground truth object. The loss minimized by the algorithm takes into account the prediction of the bounding box positions (x, y), their sizes (h, w), the confidence scores (obj scores) of the predictions, and the predicted classes (class probabilities).

[0058] On the one hand, SOLO uses an "instance category" that assigns a category to each pixel in an instance according to the position and size of the instance, so as to convert the segmentation of the instance into a single-shot classification-solvable problem. Conveniently, SOLO provides a much simpler and more flexible instance segmentation framework with strong performance, achieving the same accuracy as Mask R-CNN and outperforming recent single-shot instance segmenters in terms of accuracy.

[0059] The simpler architectures of YOLO and SOLO can utilize only a small amount of data in the training dataset, and furthermore, they are particularly convenient for implementations in the medical field that provide faster inferences, which is important when there are thousands of cells on each single slide to be analyzed.

[0060] According to one embodiment, the method further includes step 150 of counting the bounding boxes associated with each class obtained as the output of the object detection model in order to obtain the total number of objects of each class.

[0061] According to one embodiment, the method includes step 160 of inputting each uncountable connected component into a semantic segmentation model and obtaining, as an output, a segmentation mask in which all pixels are classified into one of the defined available classes. In some cases, when there is significant overlap between objects, humans cannot distinguish the objects individually. Therefore, in such cases, the object detection model cannot detect each individual object. Thus, the segmentation model conveniently enables the calculation of the approximate number of objects.

[0062] According to one embodiment, the semantic segmentation model is a U-Net. The architecture of the U-Net looks like a "U" that justifies its name. This architecture consists of three sections: (1) a contracting section, (2) a bottleneck section, and (3) an expanding section.

[0063] The contracting section is composed of several contracting blocks. Each block receives an input, applies two 3×3 convolutional layers, and then applies 2×2 max pooling. The number of kernels or feature maps after each block doubles, so that the architecture can effectively learn a complex structure.

[0064] The bottleneck mediates between the contracting section and the expanding section. It uses two 3×3 convolutional layers followed by a 2×2 upsampling layer.

[0065] Similar to the contracting section, the expanding section consists of several expanding blocks. At the beginning of each expanding block, the output feature map of the corresponding contracting block and the output of the previous expanding block are combined. Then, this combined block is passed through two 3×3 convolutional layers and one 2×2 upsampling layer. For each expanding block, after the first 3×3 convolutional layer, the number of feature maps is divided by 2.

[0066] Finally, the resulting feature map is passed through a final 1×1 convolutional layer where the number of resulting feature maps is equal to the number of classes.

[0067] According to one embodiment, the method includes step 170 of counting the number of objects as the ratio of the total pixel area of a class, obtained as the number of pixels of the segmentation mask associated with the class, to the average area of the objects of the class, for each class of the semantic segmentation model.

[0068] According to one embodiment, the defined classes of the semantic segmentation model and the object detection model are the same.

[0069] According to one embodiment, the semantic segmentation model and the object detection model are trained using a dataset of labeled digitized images.

[0070] According to one embodiment, the semantic segmentation model and the object detection model are trained using a stochastic gradient descent training method.

[0071] According to one embodiment, each class is in the following list: - White blood cells: basophils, neutrophils, macrophages, monocytes, and eosinophils; - Red blood cells; - Bacteria; - Urinary crystals; - Cylinders; - Healthy and atypical urothelial cells; - Squamous epithelial cells; - Reactive urothelial cells, and / or - Yeast is related to at least one of the objects in.

[0072] According to one embodiment, the method further includes a step of displaying the total number of objects of each class along a digitized image of at least one part of the whole slide. Among them, the user can conveniently visualize the result of the number of objects obtained by using this method and the digitized image.

[0073] According to one embodiment, the method further includes a step of displaying at least one string that provides information about the presence or absence of objects within a class in the digitized image. The string may be displayed along the digitized image.

[0074] The present invention further relates to a system for classifying and counting objects recoverable from a urine sample processed on a slide. In the following, modules should be understood as functional entities, rather than materials or physically different components. As a result, they can be embodied as grouped into the same tangible and specific components, or distributed among several such components. Also, each of these modules is itself likely shared among at least two physical components. In addition, the modules may also be implemented in hardware, software, firmware, or a mixed form thereof. Preferably, they are embodied within at least one processor of the system.

[0075] The system of the present invention may comprise an acquisition module configured to receive at least one digitized image of the entire slide including a plurality of objects. The acquisition module may be connected to a bright-field optical system configured to acquire at least one image of the entire slide.

[0076] In one embodiment, the system - detects bound components by segmentation of an image of the entire slide; - classifies the detected bound components into countable bound components and uncountable bound components using a classifier; - in the case of countable bound components: · inputs each countable bound component into an object detection model configured to detect objects and output one bounding box and a related class for each object; · counts the bounding boxes related to each class to obtain the number of objects of each class; - in the case of uncountable components: · inputs each uncountable bound component into a semantic segmentation model and obtains as an output a segmentation mask in which all pixels are classified into one of the available classes with defined classes; · For each class, count the number of objects as the ratio of the total pixel area of the class, obtained as the number of pixels of the segmentation mask related to the class, to the average area of the objects of the class; At this time, the defined classes of the semantic segmentation model and the object detection model are the same. - Sum the number of objects of each class obtained from the semantic segmentation model and the object detection model It comprises a calculation module configured as described above.

[0077] According to one embodiment, the system comprises an output module configured to output the number of objects for each class.

[0078] The present invention further includes a computer program product for classifying and counting objects recoverable from a urine sample, the computer program product including instructions that cause a computer to perform the steps of the method according to any one of the embodiments described herein when the program is executed by the computer.

[0079] The computer program product for performing the above method can be described as a computer program, code segment, instruction, or any combination thereof for individually or collectively instructing or configuring a processor or computer to operate as a machine or special-purpose computer for performing operations executed by hardware components. In one example, the computer program product includes machine code directly executable by a processor or computer, such as machine code generated by a compiler. In another example, the computer program product includes high-level code executable by a processor or computer using an interpreter. A person skilled in the art, a programmer, can easily write instructions or software based on the block diagrams and flowcharts illustrated in the drawings and the corresponding descriptions in this specification that disclose the algorithms for performing the operations of the above method.

[0080] The present invention further includes a computer-readable storage medium including instructions that cause a computer to execute the steps of the method according to any one of the above embodiments when the program is executed by the computer.

[0081] According to one embodiment, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0082] The computer program for implementing the method of this embodiment can generally be distributed to users on a distribution computer-readable storage medium such as, but not limited to, an SD card, an external storage device, a microchip, a flash memory device, a portable hard drive, and a software website. From the distribution medium, the computer program can be copied to a hard disk or a similar intermediate storage medium. The computer program can be executed by loading computer instructions from either of those distribution media or those intermediate storage media into the execution memory of the computer, configuring the computer to act according to the method of this invention. All of these operations are well known to those skilled in the art of computer systems.

[0083] Instructions or software that control a processor or computer that implements the hardware components and executes the above methods, as well as any associated data, data files, and data structures, are recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access memory (RAM), flash memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, and any device known to those skilled in the art that stores instructions or software and any associated data, data files, and data structures in a non-transitory manner and can provide the instructions or software and any associated data, data files, and data structures to a processor or computer. In one example, the instructions or software and any associated data, data files, and data structures are distributed over a network-connected computer system so that they are stored, accessed, and executed in a distributed manner by a processor or computer.

Claims

1. A computer-implemented method (100) for classifying and counting objects recoverable from a urine sample processed on a slide, comprising: - a step (110) in which an acquisition module receives at least one digitized image of the entire slide; - a step (120) in which a calculation module detects connected components by segmenting the image of the entire slide; - a step (130) in which the calculation module classifies the detected connected components into countable connected components and non-countable connected components using a classifier; - in the case of countable connected components: · a step (140) in which the calculation module inputs each countable connected component into an object detection model to detect objects and obtain an output including a bounding box and a related class for each detected object; · a step (150) in which the calculation module counts the bounding boxes related to each class and obtains the number of objects in each class; - in the case of non-countable components: · a step (160) in which the calculation module inputs each non-countable connected component into a semantic segmentation model and obtains, as an output, a segmentation mask in which all pixels are classified into one of the available classes for which definitions are available; · a step (170) in which the calculation module counts the number of objects as the ratio of the total pixel area of the class, obtained as the number of pixels of the segmentation mask related to the class for each class, to the average area of the objects of the class; - a step in which the calculation module sums the number of objects in each class obtained from the semantic segmentation model and the object detection model; - a step in which an output module outputs the number of objects in each class wherein the classes of the semantic segmentation model and the object detection model are the same.

2. The method according to claim 1, wherein the semantic segmentation model is a U-Net.

3. The method according to any one of claims 1 or 2, wherein the object detection model is a Faster R-CNN, CenterNet, SOLO or YOLO.

4. The method according to any one of claims 1 to 3, wherein the received digitized image results from a bright-field optical system.

5. The method according to any one of claims 1 to 4, wherein the semantic segmentation model and the object detection model are trained using a dataset of labeled digitized images.

6. The method according to any one of claims 1 to 5, wherein the semantic segmentation model and the object detection model are trained using a probabilistic gradient descent training method.

7. Each class of the semantic segmentation model and the object detection model is in the following list: - White blood cells: basophils, neutrophils, macrophages, monocytes, and eosinophils; - Red blood cells; - Bacteria; - Urinary crystals; - Cylinders; - Healthy and atypical urothelial cells; - Squamous epithelial cells; - Reactive urothelial cells, and / or - Yeast The method according to any one of claims 1 to 6, which is related to at least one of the objects in.

8. The method according to any one of claims 1 to 7, further comprising the step of the output module displaying the total number of objects of each class along the digitized image of at least one part of the entire slide.

9. The method according to any one of claims 1 to 8, further comprising the step of the output module displaying at least one string providing information about the presence or absence of objects within the class in the digitized image.

10. The method according to any one of claims 1 to 9, wherein the sample is colored with a Pap stain.

11. A system for classifying and counting objects recoverable from a urine sample processed on a slide, - At least one input adapted to receive at least one digitized image of the entire slide containing a plurality of objects; - At least one processor configured as follows: · Detect connected components by segmenting the image of the entire slide; · Classify the detected connected components into countable and uncountable connected components using a classifier; · In the case of countable connected components: i. Input each countable connected component into an object detection model configured to detect objects and output one bounding box and a related class for each object; ii. Count the bounding boxes related to each class to obtain the number of objects in each class; · In the case of uncountable components: i. Input each non-countable combination component into a semantic segmentation model, and obtain, as an output, a segmentation mask in which all pixels are classified into one of the available classes that have been defined; ii. For each class, count the number of objects as the ratio of the total pixel area of the class obtained as the number of pixels of the segmentation mask related to the class to the average area of the objects of the class; - Sum the number of objects of each class obtained from the semantic segmentation model and the object detection model; At this time, the classes of the semantic segmentation model and the object detection model are the same; - At least one output adapted to provide the number of objects for each class; A system including.

12. The system according to claim 11, wherein the object detection model is Faster R-CNN, CenterNet, SOLO or YOLO.

13. The system according to any one of claims 11 or 12, wherein the semantic segmentation model and the object detection model are trained using a dataset of labeled digitized images.

14. A computer program for classifying and counting objects recoverable from a urine sample, the computer program including instructions for causing a computer to perform the method according to any one of claims 1 to 10 when the computer program is executed by the computer.

15. A computer-readable storage medium including instructions for causing a computer to perform the method according to any one of claims 1 to 10.

Images