Method and device for classifying specimens included in images by using artificial intelligence

Abstract

Provided are a method and a device for classifying specimens included in images, the method comprising the steps of: generating a plurality of specimen images from images of specimens; acquiring the type and the reliability score of a target specimen corresponding to each specimen image by using a classification model; determining, on the basis that the reliability score is included in a predetermined range, the type of the target specimen corresponding to the specimen image by using a reviewer model; and determining, on the basis of the type of the target specimen and information related to the specimen image, a medical classification by using a machine learning or rule-based system.

Description

Method and device for classifying specimens contained in an image using artificial intelligence

[0001] The present disclosure relates to a method and device for classifying specimens based on artificial intelligence, and more specifically, to a method and device for classifying the type of specimen included in an image using artificial intelligence.

[0002] Typically, to analyze a sample (e.g., cells) contained in blood or tissue, the blood or tissue is stained and then examined under a microscope to analyze the sample. Tissue can refer to a collection of cells, intermediate between cells and organs. Diagnosis using tissue can be primarily used to diagnose cancer. For example, a diagnosis can be made by observing the morphology of the samples that make up the tissue or determining the presence or absence of specific proteins. However, analyzing a sample with the naked eye requires expert expertise, making it a time-consuming and costly method. Furthermore, there are limitations in improving the accuracy of sample analysis.

[0003] With recent advancements in artificial intelligence, it is being used across industries. However, developing AI models to solve problems using AI requires significant time and resources.

[0004] Embodiments of the present disclosure aim to improve the prediction accuracy of analyzing a specimen included in an image taken of the specimen.

[0005] In one embodiment of the present disclosure, a method for classifying a specimen included in an image may be provided. The method for classifying a specimen included in an image may include the steps of generating a plurality of specimen images from images of specimens, each of the plurality of specimen images including a target specimen; obtaining a type of target specimen and a confidence score corresponding to each specimen image using a classification model, wherein the confidence score is characterized as being related to the type of target specimen; determining a type of target specimen corresponding to the specimen image using a reviewer model based on the confidence score being within a predetermined range; and determining a medical level using a machine learning or rule-based system based on the type of target specimen and information related to the specimen image. The classification model may be trained to output a confidence score related to the type of target specimen included in an input image. The reviewer model may be trained to determine the type of target specimen included in an input image.

[0006] In one embodiment of the present disclosure, the step of generating a plurality of specimen images may include a step of inputting the captured image into a third artificial intelligence model to identify information related to the positions of the plurality of target specimens and the sizes of the plurality of target specimens included in the captured image, and a step of acquiring a plurality of specimen images from the captured image based on the information related to the positions of the plurality of target specimens and the sizes of the plurality of target specimens. The third artificial intelligence model may be trained to identify information related to the positions and sizes of one or more target specimens included in the input image.

[0007] In one embodiment of the present disclosure, the images of the specimens may include a plurality of images captured at a plurality of focal lengths. The step of generating the plurality of specimen images may include the steps of determining one of the plurality of focal lengths as an optimal focal length, selecting an image having the optimal focal length from among the captured plurality of images, and generating a specimen image including the target specimen from the selected image.

[0008] In one embodiment of the present disclosure, a method for classifying a specimen included in an image may further include a step of determining the type of the target specimen using a confidence score based on whether the confidence score is not within a predetermined range.

[0009] In one embodiment of the present disclosure, the step of determining the type of target specimen may include the step of enlarging a specimen image, and the step of determining the type of target specimen corresponding to the enlarged specimen image using a reviewer model.

[0010] In one embodiment of the present disclosure, the step of determining the type of target specimen corresponding to the enlarged specimen image may include the step of dividing the enlarged specimen image into a plurality of patches, and the step of inputting the plurality of patches into a reviewer model to determine the type of target specimen.

[0011] In one embodiment of the present disclosure, the step of enlarging the specimen image may include the step of acquiring a first focus image captured with a shorter focal length than the specimen image, the step of acquiring a second focus image captured with a longer focal length than the specimen image, the step of generating a corrected specimen image based on the first focus image, the second focus image, and the specimen image, and the step of enlarging the corrected specimen image.

[0012] In one embodiment of the present disclosure, a method for classifying a specimen included in an image may include a step of selecting at least some of a plurality of specimen images using a fourth artificial intelligence model. The step of obtaining a confidence score may include a step of enlarging at least some of the selected specimen images, and a step of inputting the enlarged specimen images into a classification model to obtain a type of target specimen and a confidence score corresponding to each enlarged specimen image.

[0013] In one embodiment of the present disclosure, a method for classifying a specimen included in an image may include a step of transmitting the determined medical level to an external device or outputting the medical level through a display. The method for classifying a specimen included in an image may include a step of obtaining a user input for modifying the medical level. The method for classifying a specimen included in an image may include a step of modifying the medical level in response to the obtained user input and storing information related to the modified medical level. The method for classifying a specimen included in an image may include a step of determining a reclassification rule using the stored information.

[0014] In one embodiment of the present disclosure, the target sample may include at least one of a red blood cell (RBC), a white blood cell (WBC), a platelet, or a cervical cell.

[0015] In one embodiment of the present disclosure, a computer-readable recording medium having recorded thereon a program for performing at least one of the above-mentioned methods on a computer may be provided.

[0016] In one embodiment of the present disclosure, a device for classifying a specimen included in an image may be provided. The device for classifying a specimen included in an image may include a memory including one or more instructions, and at least one processor. By executing one or more instructions by the at least one processor, the device may generate a plurality of specimen images from images of specimens. Each of the plurality of specimen images may include a target specimen. By executing one or more instructions by the at least one processor, the device may obtain the type of target specimen and a confidence score corresponding to each specimen image using a classification model. The confidence score may be characterized as being related to the type of target specimen. By executing one or more instructions by the at least one processor, the device may determine the type of target specimen corresponding to the specimen image using a reviewer model based on whether the confidence score falls within a predetermined range. By executing one or more instructions by the at least one processor, the device may determine a medical level using a machine learning or rule-based system based on the type of target specimen and information related to the specimen image. The classification model may be trained to output a confidence score related to the type of target specimen contained in the input image. The reviewer model may be trained to determine the type of target specimen contained in the input image.

[0017] In one embodiment of the present disclosure, by having at least one processor execute one or more instructions, the device can input a captured image into a third artificial intelligence model to identify information related to the locations of a plurality of target specimens and sizes of the plurality of target specimens included in the captured image, and obtain a plurality of specimen images from the captured image based on the information related to the locations of the plurality of target specimens and sizes of the plurality of target specimens. The third artificial intelligence model may be trained to identify information related to the locations and sizes of one or more target specimens included in the input image.

[0018] In one embodiment of the present disclosure, images of specimens may include multiple images captured at multiple focal lengths. By having at least one processor execute one or more instructions, the device may determine one of the multiple focal lengths as an optimal focal length, select an image having the optimal focal length from among the captured images, and generate a specimen image including the target specimen from the selected image.

[0019] In one embodiment of the present disclosure, the device can determine the type of target specimen using a confidence score based on whether the confidence score is not within a predetermined range by having at least one processor execute one or more instructions.

[0020] In one embodiment of the present disclosure, the device can magnify a specimen image and determine a type of target specimen corresponding to the magnified specimen image using a reviewer model by having at least one processor execute one or more instructions.

[0021] In one embodiment of the present disclosure, the device can divide an enlarged specimen image into a plurality of patches by having at least one processor execute one or more instructions, and input the plurality of patches into a reviewer model to determine the type of target specimen.

[0022] In one embodiment of the present disclosure, the device can acquire a first focus image captured with a shorter focal length than a specimen image, acquire a second focus image captured with a longer focal length than the specimen image, generate a corrected specimen image based on the first focus image, the second focus image, and the specimen image, and magnify the corrected specimen image by causing at least one processor to execute one or more instructions.

[0023] In one embodiment of the present disclosure, by having at least one processor execute one or more instructions, the device can select at least some of a plurality of specimen images using a fourth artificial intelligence model, enlarge at least some of the selected specimen images, and input the enlarged specimen images into a classification model, thereby obtaining the type and reliability score of the target specimen corresponding to each enlarged specimen image.

[0024] In one embodiment of the present disclosure, at least one processor executes one or more instructions, thereby allowing the device to transmit the determined medical level to an external device or output the medical level via a display. At least one processor may obtain user input for modifying the medical level. In response to the obtained user input, at least one processor may modify the medical level and store information related to the modified medical level. At least one processor may use the stored information to determine a reclassification rule.

[0025] In one embodiment of the present disclosure, the target sample may include at least one of red blood cells, white blood cells (WBCs), platelets, or cervical cells.

[0026] The present disclosure can provide highly accurate analysis results without relying on medical technicians to provide samples. Furthermore, it can improve the time and cost required for analyzing blood or tissue samples.

[0027] The effects of the embodiments of the present disclosure are not limited to the effects described above, and effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the attached drawings.

[0028] FIG. 1 is a drawing for explaining a device for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0029] FIG. 2 is a flowchart of a method for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0030] FIG. 3 is a block diagram of a device for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0031] FIG. 4 is a block diagram of a specimen detection module according to one embodiment of the present disclosure.

[0032] FIG. 5 is a block diagram of a specimen detection artificial intelligence module according to one embodiment of the present disclosure.

[0033] FIG. 6 is a block diagram of a specimen classification module according to one embodiment of the present disclosure.

[0034] FIG. 7 is a block diagram of a specimen classification module according to one embodiment of the present disclosure.

[0035] FIG. 8A is a diagram illustrating a specimen classification module according to one embodiment of the present disclosure.

[0036] FIG. 8b is a diagram illustrating a specimen classification module according to one embodiment of the present disclosure.

[0037] FIG. 8c is a diagram illustrating a reviewer model according to one embodiment of the present disclosure.

[0038] FIG. 9a is a diagram illustrating training data according to one embodiment of the present disclosure.

[0039] FIG. 9b is a diagram illustrating training data according to one embodiment of the present disclosure.

[0040] FIG. 10 is a diagram for explaining the reliability score of a classification model according to one embodiment of the present disclosure.

[0041] Figure 11 is a diagram illustrating data that may cause errors in specimen classification.

[0042] FIG. 12 is a diagram illustrating a process for training a reviewer model according to one embodiment of the present disclosure.

[0043] FIG. 13 is a block diagram illustrating a device for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0044] FIG. 14 is a flowchart of a method for processing a sample by a device according to one embodiment of the present disclosure.

[0045] FIG. 15 is a block diagram illustrating a device for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0046] The terms used in this disclosure will be briefly explained, and one embodiment of the present disclosure will be specifically described.

[0047] The terms used in this disclosure are selected from widely used, current terms, taking into account the functions of one embodiment of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant embodiments of the disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of the disclosure.

[0048] Throughout this disclosure, when a part is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "module," and the like described herein refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.

[0049] In this disclosure, when an element or layer is referred to as being “on” or “on” another element or layer, it may include not only directly on top of the other element or layer, but also intervening layers or other components.

[0050] In the present disclosure, “at least one of a, b, or c” may include “a,” “b,” “c,” “a and b,” “b and c,” “a and c,” and “a, b, and c.” Additionally, the expression “at least one of a, b, or c” may be replaced by the term “and / or,” such as “a, b, and / or c.”

[0051] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present disclosure. However, one embodiment of the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted to clearly describe one embodiment of the present disclosure, and similar parts are designated with similar drawing reference numerals throughout the present disclosure.

[0052] FIG. 1 is a drawing for explaining a device for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0053] In one embodiment of the present disclosure, the device (2000) can classify a specimen included in an image (110). The image (110) may be a photograph of stained specimens (e.g., cells) included in blood or tissue. For example, the device (2000) can classify a specimen included in an image (110) of stained blood or tissue. In one embodiment, the image (110) may be captured using at least one of X-ray, Magnetic Resonance Imaging (MRI), Computed Tomography (CT), or Positron Emission Tomography (PET). For example, the image (110) may be a photograph of a portion of an organ using at least one of X-ray, MRI, CT, or PET. The device (2000) can classify a specimen including at least one of bacteria, fungi, mites, parasites, or viruses included in the image (110).

[0054] In one embodiment of the present disclosure, the device (2000) can analyze a specimen included in an image (110). The device (2000) can identify a specimen (e.g., a target specimen) included in the image (110). For example, the device (2000) can generate an object image (120) including the results of identifying the specimens included in the image (110). The object image (120) can include one or more specimens and a bounding box representing each specimen (e.g., a boundary area expressed to be adjacent to the specimen). For example, a square displayed in the object image (120) can represent a bounding box.

[0055] In one embodiment of the present disclosure, the device (2000) can generate a specimen image (130) including a specimen from an image (110) and / or an object image (120). In one embodiment, the device (2000) can generate a specimen image (130) including a specimen of the image (110). The specimen image (130) can include one specimen (or more than one specimen). The device (2000) can generate the specimen image (130) by segmenting the image (110) according to the location of the specimen identified using the object image (120). In one embodiment, the device (2000) can generate the specimen image (130) identical to the bounding box of the object image (120). For example, the device (2000) can determine each bounding box of the object image (120) as a specimen image (130).

[0056] In one embodiment of the present disclosure, the device (2000) can determine the type of specimen included in a specimen image using an artificial intelligence model. For example, the device (2000) can obtain the type and confidence score of a target specimen corresponding to the specimen image using a classification model. The device (2000) can determine the type of specimen included in the specimen image using a reviewer model based on whether the confidence score falls within a predetermined range. According to one embodiment of the present disclosure, a process by which the device (2000) determines the type of specimen (e.g., target specimen) using an artificial intelligence model is described in detail with reference to FIGS. 2 to 7, 8A, and 8B.

[0057] In one embodiment of the present disclosure, the device (2000) can determine a medical level based on the type of specimen using a machine learning or rule-based system. For example, the device (2000) can determine a medical level based on the results (e.g., statistical data) of analyzing multiple specimen images (130). The medical level may refer to an analysis result related to the patient's health status. For example, the medical level may include at least one of the presence or absence of malaria infection in the patient, the percentage of infected RBCs, or the type of malaria infection. For example, the medical level may include the patient's blood cell count or blood cell ratio. For example, the medical level may include the patient's cervical cancer diagnosis stage.

[0058] In one embodiment of the present disclosure, the device (2000) can acquire an image (110) by photographing blood or tissue contained in a cartridge (100). The cartridge (100) can contain a patient's blood and a blood stained sample, or a patient's tissue and a tissue stained sample. The device (2000) can photograph the patient's blood or tissue using the cartridge (100) and classify cells contained in the blood or tissue to determine a medical level. The device (2000) can automatically process the image to determine a medical level without user intervention, thereby producing highly accurate analysis results.

[0059] In one embodiment of the present disclosure, the device (2000) can process blood or tissue contained in a cartridge (100). For example, the device (2000) can perform smear processing, fixation processing, and stain processing on the cartridge. The cartridge processing process is automatically performed in the device (2000). For example, the series of processes in which a sample is smeared, fixed, and stained does not require manual intervention by a human and can be automatically performed in the device (2000). In conventional blood smear tests or tissue tests, the processes of smearing, staining, and microscopic observation of blood or tissue depend on the manual work of the examiner. Therefore, if the examiner is not skilled, there are problems such as difficulty in microscopy due to uneven smear conditions of blood or tissue, contamination of the sample due to errors in reaction conditions during the blood or tissue staining process, etc., and difficulty in accurate examination if the examiner's skill is insufficient. In addition, since the examiner must visually classify the sample, it requires a lot of time and money. However, the device (2000) according to one embodiment of the present disclosure can automatically perform the smear and staining process of blood or tissue, and can provide fast and accurate analysis results regardless of the skill of the examiner through image processing using artificial intelligence. The process of performing smear processing, fixation processing, and staining processing by the device (2000) according to one embodiment of the present disclosure will be described with reference to FIGS. 14 and 15.

[0060] In one embodiment of the present disclosure, the device (2000) can acquire an image (110) of an organ using an imaging technique such as X-ray, MRI, CT, or PET. The device (2000) can capture the image (110) using an imaging technique or acquire the image (110) captured using an external device. The device (2000) can determine a medical level by classifying a specimen contained in the organ. The device (2000) can automatically process the image to determine the medical level without user intervention, thereby deriving a highly accurate analysis result.

[0061] In one embodiment of the present disclosure, the device (2000) can provide medical information. For example, the device (2000) can provide information about the medical information to the user (140) via a display. For example, the device (2000) can transmit information about the medical information to a server (150). The device (2000) can provide information about the medical information to an external device via the server (150).

[0062] FIG. 2 is a flowchart of a method for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0063] In one embodiment of the present disclosure, a method for classifying a specimen included in an image may be performed by a device (2000). For example, the device (2000) may perform each step of the method for classifying a specimen included in an image by having a processor of the device (2000) execute at least one instruction included in a memory. The processor controlling the device (2000) may perform image processing and operations performed by artificial intelligence. In one embodiment, the processor may include an artificial intelligence processor. In one embodiment, the processor may include a plurality of processors.

[0064] In step S210, the device (2000) may generate multiple specimen images from images of specimens. Each of the multiple specimen images may include a target specimen. In the present disclosure, a target specimen may refer to a specimen required for analyzing a medical level. The target specimen may vary depending on the purpose of the analysis. For example, if the purpose of the analysis is to determine whether a patient is infected with malaria, the target specimen may be red blood cells (RBCs). For example, if the purpose of the analysis is blood cell morphology (BCM), the target specimen may include red blood cells, white blood cells (WBCs), and platelets. For example, if the purpose of the analysis is cervical analysis, the target specimen may include cervical cells. In one embodiment, the target specimen may include a stained specimen. Without being limited thereto, target specimens may include cells, organisms and viruses for determining medical levels, including bacteria, fungi, mites, parasites and viruses.

[0065] In one embodiment of the present disclosure, the device (2000) can acquire images by photographing specimens. In one embodiment, the device (2000) can photograph the specimens by magnifying them using an optical device (e.g., a microscope). The specimens can be processed using a predetermined stained sample according to the analysis purpose (e.g., analysis of malaria infection, peripheral blood smear test, analysis of cervical cancer progression, etc.). For example, the device (2000) can automatically perform staining without user intervention by treating blood or tissue located on a slide with a stained sample (e.g., a solid stained sample).

[0066] In one embodiment of the present disclosure, the device (2000) can capture an image of the entire area of ​​a slide on which stained specimens are smeared. For example, the device (2000) can capture multiple images of a portion of the slide and synthesize the captured images to obtain an image of the entire area. Alternatively, for example, the device (2000) can capture an image of the entire area of ​​the slide. In the present disclosure, an image of the entire area of ​​the slide may be referred to as a Whole Slide Image (WSI).

[0067] In one embodiment of the present disclosure, multiple specimen images may be generated by cropping images of specimens. For example, the specimen images may be images in which a boundary region containing the specimens included in the images of the specimens is cropped.

[0068] In one embodiment of the present disclosure, the device (2000) can identify information related to the positions of multiple target specimens and sizes of the multiple target specimens included in the images of the specimens by inputting images of the specimens into an artificial intelligence model. In one embodiment, the information related to the sizes may include at least one of circularity and / or aspect ratio. For example, circularity may refer to an index indicating the ratio of the boundary of an actual particle shape of the target specimen to the boundary of a circle having the same area. For example, aspect ratio may refer to an index indicating the ratio of the width and height of the specimen.

[0069] In one embodiment of the present disclosure, the device (2000) may acquire a plurality of specimen images from images of specimens based on information related to the locations of the plurality of target specimens and the sizes of the plurality of target specimens. Here, the artificial intelligence model may be trained to identify information related to the locations and sizes of one or more target specimens included in the input image.

[0070] In step S220, the device (2000) may obtain the type of target specimen and a confidence score corresponding to each specimen image using a classification model. The confidence score may be related to the type of target specimen. For example, the confidence score may mean the probability that the target specimen is included in the specimen image. For example, the confidence score may be expressed as a rational number between 0 and 1. In one embodiment, the classification model may be trained to output a confidence score related to the type of target specimen included in an input image. For example, the classification model may be an artificial intelligence model trained to perform classification or object detection. For example, the classification model may be trained using a training data set including training specimen images and the types of specimens included in the training specimen images.

[0071] In step S230, the device (2000) may determine the type of target specimen corresponding to the specimen image using the reviewer model based on whether the confidence score falls within a predetermined range. For example, if the confidence score falls between a first threshold value and a second threshold value, the device (2000) may determine the type of target specimen corresponding to the specimen image using the reviewer model. In one embodiment of the present disclosure, if the type of target specimen determined in step S220 is different from the type of target specimen determined in step S230, the device (2000) may determine the type of target specimen determined in step S230 as the final result. For example, the device (2000) may update the type of target specimen to the type of target specimen of step S230. A threshold value according to one embodiment of the present disclosure will be described in detail with reference to FIG. 10.

[0072] A device (2000) according to one embodiment of the present disclosure can improve the accuracy of classifying a specimen by additionally using a reviewer model when it is difficult to accurately determine the type of specimen using a classification model. The reviewer model may be an artificial intelligence model trained to determine the type of target specimen included in an input image. The reviewer model according to one embodiment of the present disclosure is described in more detail with reference to FIGS. 6 to 8.

[0073] In step S240, the device (2000) may determine a medical level based on the type of target specimen and information related to the specimen image. In one embodiment of the present disclosure, the device (2000) may determine a medical level based on statistical and / or cumulative analysis information of target specimens corresponding to multiple specimen images. The device (2000) may determine a medical level in WSI units.

[0074] The device (2000) can determine a medical level depending on the purpose of the analysis. For example, if the purpose of the analysis is whether the patient is infected with malaria, the device (2000) can determine whether the patient is infected, the type of malaria infected, and / or the level of infection (e.g., the percentage of infected samples). For example, if the purpose of the analysis is a peripheral blood smear test, the device (2000) can determine the type classification of white blood cells (e.g., the five main types of white blood cells) and / or basic blood test parameters. Basic blood tests can include the number of blood cells (e.g., white blood cells, red blood cells, platelets) per unit volume contained in the blood, the amount of hemoglobin per unit volume of blood, hematocrit, and / or white blood cell differential count. For example, if the purpose is cervical analysis, the device (2000) can analyze cervical cells contained in the tissue and determine the degree of progression based on the Bethesda System, a cervical cancer progression scale. However, the device (2000) can determine medical levels for various analysis purposes, not limited to the aforementioned analysis purposes.

[0075] FIG. 3 is a block diagram of a device for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0076] Referring to FIG. 3, the device (2000) may include a specimen detection module (310), a specimen classification module (320), and a medical level determination module (330). The device (2000) may determine medical level information from an input image.

[0077] The specimen detection module (310) can generate multiple specimen images from an input image. In one embodiment, the input image may be an image captured by the device (2000). For example, the input image may be an image captured of a cartridge inserted into the device (2000). According to one embodiment of the present disclosure, a process by which the device (2000) acquires an input image will be described in detail with reference to FIG. 4.

[0078] The specimen detection module (310) can identify multiple specimens included in an input image. For example, the input image may include multiple white blood cells, red blood cells, and / or platelets included in blood, and the specimen detection module (310) can identify multiple white blood cells, red blood cells, and / or platelets included in the input image. For example, the input image may include multiple tissue specimens (e.g., cervical cells) included in tissue, and the specimen detection module (310) can identify multiple tissue specimens included in the input image.

[0079] The sample detection module (310) can identify a target sample among multiple samples included in an input image. For example, the input image may include multiple white blood cells, red blood cells, and / or platelets contained in blood, but the sample detection module (310) can only identify red blood cells, which are target samples included in the input image. Red blood cells are merely an example to illustrate the target sample, and the target sample may be changed depending on the purpose of the analysis.

[0080] The specimen detection module (310) can generate a specimen image including a target specimen. The specimen detection module (310) can extract a specimen image including the target specimen from an input image. For example, the specimen image may be an image in which the input image is cropped to include the target specimen. The specimen detection module (310) can generate a plurality of specimen images corresponding to a plurality of identified target specimens. The specimen detection module (310) can transmit the specimen image to the specimen classification module (320).

[0081] The specimen classification module (320) can identify the type of target specimen included in the specimen image. In one embodiment, the type of target specimen may include the type of blood cell and / or the type of infected bacteria. For example, the device (2000) can identify whether the target specimen included in the specimen image is a red blood cell, a white blood cell, or a platelet. For example, the device (2000) can identify whether the target specimen included in the specimen image is a P. falciparum, a P. vivax, a P. ovale, a P. malariae, or a P. knowlesi. The specimen classification module (320) can transmit the type of the identified target specimen to the medical level determination module (330).

[0082] In one embodiment, the specimen classification module (320) may include an artificial intelligence model for object detection or classification. According to one embodiment of the present disclosure, the specimen classification module (320) including the artificial intelligence model is described in detail with reference to FIG. 6 .

[0083] The medical level determination module (330) can determine the medical level based on the type of identified target specimen and information related to the specimen image. In one embodiment, the information related to the specimen image may include at least one of position information, size information, focal length, color distribution information of the target specimen or specimen image, or information that can be utilized as input to the specimen identification module (530) of FIG. 5 described below.

[0084] In one embodiment, the medical level determination module (330) may generate statistical information based on the type of identified target specimen and information related to the specimen image. For example, the medical level determination module (330) may generate statistical information including the number (or weight) of target specimens per unit dose. The medical level determination module (330) may determine the medical level based on the statistical information. In one embodiment, the medical level may refer to information related to the patient's health status.

[0085] In one embodiment, the medical level determination module (330) may utilize rules or machine learning to determine the medical level by utilizing at least one of the type, number, or size information of the target specimens included in the statistical information. For example, the medical level determination module (330) may determine the medical level information as abnormal when the average size of some of the specific target specimens is enlarged or the ratio of specific target specimens determined to be abnormal is above a reference value. In addition, for example, the medical level determination module (330) may determine the medical level information including the result of comparing the area of ​​the core and electrolyte within the target specimen. In one embodiment, the determined medical level information may be displayed through the display of the device (2000).

[0086] In one embodiment, the device (2000) may further include a target specimen reclassification module. The target specimen reclassification module may change at least some of the analysis results obtained by the specimen classification module (320) and the medical level determination module (330). For example, the target specimen reclassification module may change the type of target specimen determined by the specimen classification module (320). For example, the target specimen reclassification module may change the medical level determined by the medical level determination module (330).

[0087] In one embodiment, the target specimen reclassification module may obtain user input to modify an analysis result (e.g., a medical level). For example, the target specimen reclassification module may obtain a request to change the type and / or medical level of a target specimen via a server and / or application. The target specimen reclassification module may modify at least a portion of the analysis result based on the input regarding the change request. For example, the target specimen reclassification module may modify the medical level in response to the user input. The target specimen reclassification module may store information related to the modified analysis result. For example, the target specimen reclassification module may store information related to the modified medical level in response to the user input.

[0088] In one embodiment, the target specimen reclassification module can learn reclassification rules based on data regarding changes in the analysis results. For example, the target specimen reclassification module can generate reclassification rules using data that changed the analysis results by the device (2000). For example, the target specimen reclassification module can determine the reclassification rules using stored information. The target specimen reclassification module can modify the analysis results without a user request using the generated reclassification rules. For example, the target specimen reclassification module can change at least a portion of the target specimen type determined by the specimen classification module (320) and / or the medical level determined by the medical level determination module (330) for data with a pattern similar to previously changed data without user intervention.

[0089] Although FIG. 3 is described as three modules, a specimen detection module (310), a specimen classification module (320), and a medical level determination module (330), the present invention is not limited thereto, and at least some of the specimen detection module (310), the specimen classification module (320), and the medical level determination module (330) may be merged or subdivided into one module.

[0090] FIG. 4 is a block diagram of a specimen detection module according to one embodiment of the present disclosure.

[0091] Referring to FIG. 4, the specimen detection module (310) may include a focal length analysis module (410) and a specimen detection artificial intelligence module (420).

[0092] In one embodiment of the present disclosure, the device (2000) can capture multiple images with different focal lengths or depths. The input images may be images captured at multiple focal lengths (or depths). For example, the input images may include images captured at multiple focal lengths or images captured at multiple depths.

[0093] The focal length analysis module (410) can select the image with the best focus from among images captured at multiple focal lengths (or depths). For example, the focal length analysis module (410) can determine one of the multiple focal lengths as the optimal focal length. An image with the best focus may indicate a large number of clearly captured target specimens.

[0094] In one embodiment of the present disclosure, the focal length analysis module (410) can select an image with an optimal focal length from among a plurality of captured images. The focal length analysis module (410) can select an image with a good focus based on the boundary of the target specimen. The focal length analysis module (410) can identify the boundary of the target specimen included in the captured images. For example, the focal length analysis module (410) can identify the boundary of the target specimen using a method such as Laplacian Edge Detection. However, the method is not limited to the Laplacian edge detection method, and a boundary detection method such as a Schar filter, a Sobel filter, a cross filter, or a differential filter can be used. The focal length analysis module (410) can select an image with many identified boundaries from among the captured images according to a plurality of focal lengths (or depths) as the image with the best focus. In one embodiment, the image selected as the image with the best focus may be referred to as an in-focus image or an optimal image. The focal distance analysis module (410) can transmit the selected image to the specimen detection artificial intelligence module (420).

[0095] The specimen detection artificial intelligence module (420) can output a specimen image corresponding to a specimen included in the focal image when a focal image selected by the focal distance analysis module (410) is input. The specimen detection artificial intelligence module (420) can generate a specimen image including a target specimen from an image selected with an optimal focal distance.

[0096] In one embodiment, the specimen detection artificial intelligence module (420) can identify the type and / or location of a specimen included in a focus image when a focus image is input. The specimen detection artificial intelligence module (420) can generate a specimen image including a target specimen. According to one embodiment of the present disclosure, the process of the specimen detection artificial intelligence module (420) identifying a target specimen is described with reference to FIG. 5 .

[0097] FIG. 5 is a block diagram of a specimen detection artificial intelligence module according to one embodiment of the present disclosure.

[0098] Referring to FIG. 5, the specimen detection artificial intelligence module (420) may include an encoder (510), a decoder (520), a specimen identification module (530), and a specimen image segmentation module (540).

[0099] The encoder (510) can obtain a feature vector based on an input image. The decoder (520) can generate a feature image based on the feature vector. In one embodiment of the present disclosure, the encoder (510) and the decoder (520) may be artificial intelligence models trained together to generate a feature image based on an input image.

[0100] In one embodiment of the present disclosure, the decoder (520) may include multiple decoders that generate different feature images based on a feature vector. For example, the decoder (520) may include a decoder that generates an image representing a specimen and a decoder that generates an image representing the location of the specimen. The multiple decoders have the advantage of being able to be individually trained based on the feature images they generate. A data set for training the decoder (520) is described in detail with reference to FIGS. 9A and 9B .

[0101] In one embodiment of the present disclosure, the location of a specimen may be determined based on a contour of the specimen. For example, the decoder (520) may generate an image representing the contour of the specimen for determining the location of the specimen.

[0102] A feature image may contain information regarding whether pixels in an input image have a feature. For example, a feature image may have pixels representing a sample having a first value (e.g., 1) and pixels not representing a sample having a second value (e.g., 0). Alternatively, for example, a feature image may have pixels representing the boundary of a sample having a first value and pixels not representing the boundary of a sample having a second value.

[0103] In one embodiment of the present disclosure, the decoder (520) can generate feature images for each type of specimen. For example, the decoder (520) can generate feature images for white blood cells, red blood cells, and platelets. For example, the decoder (520) can generate a feature image representing white blood cells, a feature image representing the boundaries of white blood cells, a feature image representing red blood cells, a feature image representing the boundaries of red blood cells, a feature image representing platelets, and a feature image representing the boundaries of platelets.

[0104] The specimen identification module (530) can identify a specimen based on a feature image. For example, the specimen identification module (530) can identify a specimen using the difference between an image representing the specimen boundary and an image representing the specimen itself. By identifying a specimen using the specimen boundary, the specimen identification module (530) can separate specimens that at least partially overlap each other. For example, if the specimen boundary is not utilized, two overlapping specimens may be identified as a single specimen.

[0105] The specimen image segmentation module (540) can generate a specimen image including an identified specimen. The specimen image can have a minimum rectangular size that includes the specimen. The specimen image segmentation module (540) can generate a plurality of specimen images corresponding to a plurality of specimens included in the input image. The specimen image segmentation module (540) can segment the input image such that the specimen image includes the specimens.

[0106] FIG. 6 is a block diagram of a specimen classification module according to one embodiment of the present disclosure.

[0107] Referring to FIG. 6, the specimen classification module (320) may include a classification model (610) and a reviewer model (620).

[0108] The classification model (610) can classify a specimen included in a specimen image. For example, the classification model (610) can determine the type of specimen. The type of specimen classified by the classification model (610) may vary depending on the purpose of the test. If the purpose of the test is malaria, the classification model (610) can determine the type of specimen as either normal red blood cells or malaria-infected red blood cells. If the purpose of the test is BCM, the classification model (610) can determine the type of specimen as either red blood cells, white blood cells, or platelets.

[0109] The classification model (610) can obtain the type and confidence score of the target specimen corresponding to the specimen image. Here, the confidence score may be related to the type of the target specimen. For example, the confidence score may indicate the probability that the specimen image represents the target specimen. The classification model (610) may be trained to output a confidence score related to the type of target specimen included in the input image.

[0110] In one embodiment of the present disclosure, the classification model (610) may determine the type of target specimen using a confidence score based on whether the confidence score falls within a predetermined range. For example, if the confidence score is less than a first threshold, the classification model (610) may determine that the specimen image is not a target specimen. For example, if the confidence score is greater than a second threshold, the classification model (610) may determine that the specimen image is a target specimen.

[0111] The reviewer model (620) can determine the type of target specimen corresponding to a specimen image based on whether the confidence score falls within a predetermined range. The reviewer model (620) may be trained to determine the type of target specimen included in an input image. The reviewer model (620) may only input specimen images for which the type of target specimen is not determined by the classification model (610).

[0112] In one embodiment of the present disclosure, the reviewer model (620) may have a greater number of parameters than the classification model (610). For example, the reviewer model (620) may have four times more parameters than the classification model (610).

[0113] In one embodiment of the present disclosure, the size of the input image of the reviewer model (620) may be larger than the size of the input image of the classification model (610). For example, if the size of the input image of the classification model (610) is 56 x 56, the size of the input image of the reviewer model (620) may be 224 x 224, which is four times larger. If the confidence score by the classification model (610) is within a predetermined range, upscaling may be performed on the specimen image before being input to the reviewer model (620). For example, the size of the specimen image may be increased through interpolation. The reviewer model (620) may determine the type of specimen that the classification model (610) could not accurately classify due to differences in the number of parameters and / or the sizes of the input images.

[0114] In one embodiment of the present disclosure, an input image of a reviewer model (620) may be generated based on a plurality of specimen images corresponding to a plurality of focal lengths (or depths). The specimen image may be modified based on a first focal length image captured with a focal length shorter than a focal length of a specimen image for which the type of target specimen has not been determined by the classification model (610) and a second focal length image captured with a longer focal length. For example, if the focal length at which the specimen image is captured is LB, the specimen image may be modified based on a first focal length image captured with a focal length of LB-1 that is 1 less than LB and a second focal length image captured with a focal length of LB+1 that is 1 greater than LB. Here, the first focal length image and the second focal length image may refer to images captured with the same specimen as the specimen image but with different focal lengths.

[0115] A corrected specimen image can be generated by combining the first focus image and the second focus image with the specimen image. For example, the corrected specimen image can be generated as the sum of the specimen image, the first focus image, and the second focus image. For example, the corrected specimen image can be generated as the average of the specimen image, the first focus image, and the second focus image.

[0116] FIG. 7 is a block diagram of a specimen classification module according to one embodiment of the present disclosure.

[0117] Referring to FIG. 7, the specimen classification module (320) may include a classification model (610), a reviewer model (620), and a screening model (710).

[0118] The screening model (710) can select at least some of the multiple specimen images. For example, the screening model (710) can determine which specimens are clearly target specimens (or which are clearly not target specimens) from among the multiple specimen images. For example, the screening model (710) can select the remaining specimen images, excluding those clearly target specimens (or which are clearly not target specimens) from among the multiple specimen images.

[0119] In one embodiment, the screening model (710) may be an artificial intelligence model with fewer parameters than the classification model (610) and the reviewer model (620). Therefore, the screening model (710) can be used to select a subset of multiple sample images with fewer computations. For example, when analyzing samples abundant in blood or tissue, the screening model (710) can be used to effectively process clear target samples with fewer computations.

[0120] The classification model (610) may input at least some of the specimen images selected by the screening model (710). In one embodiment, the specimen images selected by the screening model (710) may be upscaled (e.g., enlarged) before being input to the classification model (610). The classification model (610) may determine a confidence score using the upscaled specimen images.

[0121] The classification model (610) and the reviewer model (620) are described with reference to FIG. 6, and thus are omitted. In one embodiment of the present disclosure, exemplary operations of the specimen classification module using the screening model (710), the classification model (610), and the reviewer model (620) are described in detail with reference to FIGS. 8A and 8B.

[0122] FIG. 8A is a diagram illustrating a specimen classification module according to one embodiment of the present disclosure.

[0123] In one embodiment of the present disclosure, the screening model (810a), the classification model (820a), and the reviewer model (830a) may correspond to the screening model (710), the classification model (610), and the reviewer model (620) of FIGS. 6 and 7 .

[0124] In one embodiment of the present disclosure, the screening model (810a) may be implemented as an artificial intelligence model with fewer layers and parameters compared to the classification model (820a) and the reviewer model (830a). Referring to FIG. 8A, the screening model (810a) may be implemented as a 4-layer CNN. The screening model (810a) may classify a specimen image. The input image of the screening model (810a) may be a specimen image generated by the specimen detection module (310) or a downsampled version of the generated specimen image.

[0125] The screening model (810a) can select the remaining sample images, excluding those that can be determined to be target samples. That is, the screening model (810a) can provide the classification model (820a) with information on whether or not the images are target samples, even though the screening model (810a) has not yet determined whether or not they are target samples. For example, the screening model (810a) can select 10-20% of the sample images from the input images.

[0126] In one embodiment of the present disclosure, the classification model (820a) may be implemented as an artificial intelligence model with fewer layers and parameters than the reviewer model (830a). However, the classification model (820a) may be implemented as an artificial intelligence model with more layers or more parameters than the screening model (810a). Referring to FIG. 8A, the classification model (820a) may be implemented as a 34-layer Resnet. However, the types of the screening model (810a) and the classification model (820a) of FIG. 8A are merely examples and are not to be construed as being limited thereto. In addition, since the structures of the convolutional neural network (CNN) and Resnet are clearly understandable to those skilled in the art, detailed structures are omitted. The classification model (820a) may classify a sample image selected by the screening model (810a). The input image of the classification model (820a) may be upsampled compared to the image of the screening model (810a).

[0127] The classification model (820a) can determine a confidence score regarding whether an image input to the classification model (820a) represents a target specimen. The classification model (820a), like the screening model (810a), can provide the reviewer model (830a) with information regarding whether or not the image is a target specimen that was not determined by the classification model (820a). The classification model (820a) can provide the specimen image to the reviewer model (830a) when the confidence score is within a predetermined range (e.g., when the confidence score is within a range greater than or equal to a first threshold value and less than or equal to a second threshold value). For example, the classification model (810a) can provide 0.1 to 1% of the specimen images input to the classification model (810a) to the reviewer model (830a). The classification model (820a) can determine the type of the target specimen when the confidence score does not fall within a predetermined range. In this respect, the classification model (820a) differs from the screening model (810a).

[0128] In one embodiment of the present disclosure, the reviewer model (830a) may be implemented as a relatively large artificial intelligence model. The reviewer model (830a) may classify a specimen image that is not determined by the classification model (820a). The input image of the reviewer model (830a) may be upsampled compared to the image of the classification model (820a). The input image of the reviewer model (830a) may be generated by combining specimen images captured at multiple focal lengths. For example, if the images input to the screening model (810a) and the classification model (820a) are images captured at a focal length LB, the images input to the reviewer model (830a) may be generated using images captured at focal lengths LB-1, LB, and LB+1.

[0129] The reviewer model (830a) can determine a confidence score regarding whether an image input to the reviewer model (830a) represents a target specimen. The confidence score determined by the reviewer model (830a) may differ from the confidence score determined by the classification model (820a). The reviewer model (830a) can classify the image input to the reviewer model (830a) based on the confidence score determined by the reviewer model (830a).

[0130] The more parameters an AI model has, the more accurate the inference becomes. However, the computational complexity increases exponentially, potentially consuming significant time and resources. Fewer parameters in an AI model consume less time and resources, but may result in lower inference accuracy. Alternatively, the accuracy of inference may not reach a certain level. According to one embodiment of the present disclosure, a screening model (810a), a classification model (820a), and a reviewer model (830a) with different parameter sizes and purposes can be used to adjust the computational complexity while improving inference accuracy.

[0131] FIG. 8b is a diagram illustrating a specimen classification module according to one embodiment of the present disclosure.

[0132] FIG. 8b may include a reviewer model (830a) according to an embodiment different from that of FIG. 8a. The screening model (810b) and classification model (820b) according to an embodiment of the present disclosure may correspond to the screening model (810a) and classification model (820a) of FIG. 8a.

[0133] In one embodiment of the present disclosure, the reviewer model (830b) can classify a sample image that is not determined by the classification model (820b). The input image of the reviewer model (830b) can be generated based on an upsampled image compared to the image of the classification model (820b). For example, the input image of the reviewer model (830b) can include multiple patches into which the upsampled image is segmented compared to the image of the classification model (820b). Referring to FIG. 8b, the input image of the reviewer model (830b) can be segmented into nine patches.

[0134] In one embodiment of the present disclosure, the input image of the reviewer model (830b) can be generated by combining specimen images captured at multiple focal lengths. For example, as shown in FIG. 8A, if the images input to the screening model (810b) and the classification model (820b) are images captured at a focal length LB, the image input to the reviewer model (830b) can be generated using images captured at focal lengths LB-1, LB, and LB+1. For example, a combined image can be generated using images captured at multiple focal lengths, and the generated image can be segmented to obtain multiple patches.

[0135] In one embodiment of the present disclosure, the reviewer model (830b) can classify a specimen image based on a plurality of patches. The reviewer model (830b) can determine the type of target specimen based on the plurality of patches. The reviewer model (830b) can determine a confidence score regarding whether the plurality of patches represent the target specimen. The confidence score determined by the reviewer model (830b) may differ from the confidence score determined by the classification model (820b). The reviewer model (830b) can classify an image input to the reviewer model (830b) based on the confidence score determined by the reviewer model (830b).

[0136] In one embodiment of the present disclosure, the reviewer model (830b) can classify a sample image using relationships between multiple patches. For example, the reviewer model (830b) can be implemented as a transformer model (or attention model). The reviewer model (830b) can classify a sample image by identifying context through relationships between multiple patches.

[0137] According to one embodiment of the present disclosure, the accuracy of inference can be improved while adjusting the amount of computation by using a screening model (810b), a classification model (820b), and a reviewer model (830b) with different sizes and purposes of parameters.

[0138] FIG. 8c is a diagram illustrating a reviewer model according to one embodiment of the present disclosure.

[0139] In one embodiment of the present disclosure, the reviewer models (830a, 830b) of FIGS. 8A and 8B may be comprised of multiple reviewer models. The reviewer models (830a, 830b) may include multiple reviewer models (810c, 820c, 830c) that classify specimen images by inputting individual images according to multiple focal lengths without combining specimen images (e.g., cell images) captured according to focal lengths. Referring to FIG. 8C, the reviewer models (830a, 830b) may include three reviewer models (810c, 820c, 830c) that classify specimen images captured according to focal lengths. For example, the first reviewer model (810c), the second reviewer model (820c), and the third reviewer model (830c) can classify sample images by inputting images according to focal lengths LB-1, LB, and LB+1, respectively.

[0140] In one embodiment, the reviewer models (810c, 820c, 830c) can each determine a confidence score. The device (2000) can determine the type of target specimen using the confidence scores of the reviewer models (810c, 820c, 830c). In one embodiment, the device (2000) can apply various ensemble techniques to the confidence scores determined by the reviewer models (810c, 820c, 830c). In this case, the reviewer models (810c, 820c, 830c) can be implemented as transformer models (or attention models) such as the reviewer model (830b) of FIG. 8b, and can be implemented as models with smaller parameter sizes compared to the reviewer model (830b).

[0141] FIG. 9a is a diagram illustrating training data according to one embodiment of the present disclosure.

[0142] Referring to FIG. 9A, training data for training an encoder (510) and a decoder (520) according to one embodiment of the present disclosure is illustrated. The training data may include images (910) and feature images (920) for specimens.

[0143] The feature image (920) may include an image to which a label for the type of specimen is assigned. Alternatively, the feature image (920) may include an image to which a label for the location of the specimen or the boundary of the specimen is assigned. Here, the label may indicate the type of specimen.

[0144] The feature image (920) may include label information indicating the type of specimen included in the image (910) for the specimens. For example, data displayed in different colors in the feature image (920) may indicate different specimens. For example, red blood cells, white blood cells, and platelets may each be displayed in different colors in the feature image (920). Additionally, for example, white blood cells may also be displayed in different colors depending on their type. Data displayed in different colors in the feature image (920) may have different values ​​or different classes.

[0145] In one embodiment of the present disclosure, the encoder (510) and the decoder (520) can be trained using training data including images (910) and feature images (920) for specimens. The values ​​of the parameters of the encoder (510) and the decoder (520) can be determined such that when the images (910) for specimens are input to the encoder (510), the decoder (520) outputs feature images (920).

[0146] In one embodiment of the present disclosure, training data may be set differently depending on the feature image generated by the decoder (520). For example, a decoder (520) that generates a feature image representing white blood cells may be trained using training data representing white blood cells, and a decoder (520) that generates a feature image representing red blood cells may be trained using training data representing red blood cells.

[0147] FIG. 9b is a diagram illustrating training data according to one embodiment of the present disclosure.

[0148] Referring to FIG. 9B, training data for training an encoder (510) and a decoder (520) according to one embodiment of the present disclosure is illustrated. The training data may include images (930) and feature images (940) for specimens.

[0149] Unlike FIG. 9A , FIG. 9B may generate training data for only some of the samples in the image (930) of the samples. For example, although the image (930) of the samples includes multiple samples, training data may be generated only for some of the colored samples among the multiple samples. For example, the image (930) of the samples may generate training data only representing white blood cells among the multiple samples.

[0150] Generating label information for all specimens included in the image (930) of the specimens can be time-consuming and costly, and under certain conditions (e.g., when red blood cells overlap significantly), generating label information may be impossible. Therefore, the training data may include a feature image (940) containing label information for some specimens included in the image (930) of the specimens.

[0151] An encoder (510) and a decoder (520) according to one embodiment of the present disclosure can be trained using training data including label information for some samples. The training data of FIG. 9B can be used to train the encoder (510) and the decoder (520) together with the training data of FIG. 9A.

[0152] FIG. 10 is a diagram for explaining the reliability score of a classification model according to one embodiment of the present disclosure.

[0153] Referring to FIG. 10, the first graph (1010) and the second graph (1020) are graphs related to the performance of inference using an artificial intelligence model.

[0154] The first graph (1010) illustrates the relationship between precision and recall. Precision can refer to the proportion of data that are actually positive among the results predicted by the AI ​​model as positive. Precision can indicate the accuracy of the AI ​​model's predictions. Recall can refer to the proportion of actual positive data that the AI ​​model predicts as positive. Recall can indicate how well the AI ​​model identifies positive data.

[0155] As shown in the first graph (1010), increasing precision and recall tends to decrease the other. For example, if an AI model predicts all data as positive, recall may be maximized but precision may be low. Alternatively, if an AI model is very strict about predicting positives, precision may be high but recall may be low.

[0156] The second graph (1020) is a graph showing performance according to a threshold value for determining positive or negative. In one embodiment, the threshold value for determining positive or negative may mean a boundary condition for determining whether the artificial intelligence model is positive or negative. For example, if the confidence score of the artificial intelligence model is greater than (or greater than or equal to) the threshold value for determining positive or negative, it may be determined as positive. For example, if the confidence score of the artificial intelligence model is less than or equal to (or less than) the threshold value, it may be determined as negative. As the threshold value for determining positive or negative increases, precision and specificity increase, but sensitivity, which is equivalent to recall, decreases. Specificity may mean the proportion of actual negative data that the artificial intelligence model predicts as negative.

[0157] Referring to the second graph (1020), the performance of the AI ​​model changes dramatically when the criterion for determining positive or negative is within a certain range (0.99 to 0.9999). For example, in particular, sensitivity decreases slightly while specificity increases sharply. The reason for increasing the criterion for determining positive or negative despite the gradual decrease in the F1 score, which represents the balance between precision and recall, is due to the imbalance in medical data, which is mostly negative. Although the sensitivity may decrease significantly, increasing the risk of false negatives, there is still a need to reduce false positives. Therefore, when the confidence score of the AI ​​model is within a certain range, the performance of inference can be improved by using another AI model. For example, when the reliability score of the classification models (610, 820a, 820b) of FIGS. 6 to 8a and 8b is within a predetermined range (e.g., greater than or equal to the first value (here, 0.99) and less than or equal to the second value (here, 0.9999)), the performance of inference can be improved by using the reviewer models (620, 830a, 830b).

[0158] Figure 11 is a diagram illustrating data that may cause errors in specimen classification.

[0159] Referring to FIG. 11, specimen images in which errors (e.g., false negatives or false positives) may occur are illustrated.

[0160] In one embodiment of the present disclosure, errors may occur when a specimen image includes clumped specimens. Since images (1110, 1120) include clumped specimens, the specimen detection model may identify multiple clumped specimens as a single specimen. This may result in errors. For example, a specimen image may be identified as representing an infected specimen due to an area overlapping the clumped specimens.

[0161] In one embodiment of the present disclosure, errors may occur when a specimen image includes a blurry specimen. For example, image (1130) may be classified as an infected positive specimen by the AI ​​model even though it is an uninfected negative specimen because it includes a blurry area in the center of the specimen. Furthermore, for example, image (1040) may be classified as an infected positive specimen by the AI ​​model even though it is an uninfected negative specimen due to artifacts.

[0162] A device (2000) according to one embodiment of the present disclosure can improve the performance of a specimen image including a cohesive specimen by increasing the size of an input image and inputting multiple segmented patches. In addition, the device (2000) can improve the performance of a specimen image including a blurry specimen by inputting an image synthesized from images captured at multiple focal lengths into a reviewer model.

[0163] FIG. 12 is a diagram illustrating a process for training a reviewer model according to one embodiment of the present disclosure.

[0164] According to one embodiment of the present disclosure, the device (2000) can train a reviewer model using knowledge distillation. A teacher model (1210) can be trained using training data (1240). The teacher model can be an artificial intelligence model with a complex structure and high accuracy. The teacher model can be a model trained in a general manner to achieve high performance using the training data (1240).

[0165] The student model (1220) may be trained based on the teacher model (1240) and training data (1250). The training data (1250) of the student model (1220) may have a smaller amount of data than the training data (1250) of the teacher model (1240). The student model (1220) may be trained using the training data (1250) according to a loss function determined based on the teacher model (1240).

[0166] The reviewer model (1230) can be trained using the trained student model (1220) on specimen image training data (1260). While the teacher model (1210) and the student model (1220) can be trained using general images, the reviewer model (1230) can be trained using training data (1260) regarding specimen images. In one embodiment, the teacher model (1210) and / or the student model (1220) may be publicly available. The device (2000) can train the reviewer model (1230) using the pre-trained teacher model (1210) or student model with less resources and time.

[0167] In one embodiment of the present disclosure, training data (1260) may include images of photographed specimens (e.g., stained cells). In one embodiment, training data (1260) may include images in which color of the photographed images has been changed, rotated, or otherwise corrected.

[0168] FIG. 13 is a block diagram illustrating a device for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0169] As illustrated in FIG. 13, a device (2000) according to one embodiment of the present disclosure may include a processor (2100), an image capture unit (2200), and a memory (2300). Not all components of the device (2000) are essential, and each component may be added or subtracted depending on the design concept of the manufacturer.

[0170] According to one embodiment of the present disclosure, the processor (2100) can control the overall operation of the device (2000). The processor (2100) can control the image capture unit (2200) and the memory (2300) by executing programs stored in the memory (2300).

[0171] According to one embodiment of the present disclosure, the processor (2100) may include an artificial intelligence (AI) processor. The AI ​​processor may be manufactured in the form of a dedicated hardware chip for AI, or may be manufactured as part of an existing general-purpose processor (e.g., CPU or application processor) or a graphics-only processor (e.g., GPU - Graphic Processing Unit) and mounted on the device (2000). The processor (2100) may be an AI processor capable of performing a 4-layer CNN and a 34-layer ResNet according to one embodiment of the present disclosure.

[0172] According to one embodiment of the present disclosure, the processor (2100) may perform control operations according to a program that classifies specimens included in images of specimens in the device (2000) and determines their medical level. The processor (2100) may perform the specimen classification method and / or the medical level determination method described through FIGS. 1 to 12 of the present disclosure.

[0173] Information about specimen images, parameters of an artificial intelligence model, and / or medical levels of specimens according to one embodiment of the present disclosure may be stored in memory (2300).

[0174] According to one embodiment of the present disclosure, the image capture unit (2200) can capture a specimen image. The image capture unit (2200) can include a precision camera. The camera of the image capture unit (2200) may be configured as a CMOS sensor, but is not limited thereto. The image capture unit (2200) can be configured with sensors for performing an imaging technique (e.g., X-ray, MRI, CT, or PET). An image including a specimen (e.g., cell, bacteria, fungus, mite, parasite, virus, etc.) captured by the image capture unit (2200) can be subjected to image processing and / or graphic processing by the processor (2100) for image classification. The device (2000) can capture images of specimens by the processor (2100) executing one or more instructions stored in the memory (2300). The device (2000) can capture images using the image capture unit (2200).

[0175] According to one embodiment of the present disclosure, by the processor (2100) executing one or more instructions stored in the memory (2300), the device (2000) can generate a plurality of specimen images from images of specimens. Here, each of the plurality of specimen images can include a target specimen. By the processor (2100) executing one or more instructions stored in the memory (2300), the device (2000) can obtain the type and confidence score of the target specimen corresponding to each specimen image using a classification model. Here, the confidence score can be related to the type of the target specimen. The classification model can be trained to output a confidence score related to the type of the target specimen included in an input image. By the processor (2100) executing one or more instructions stored in the memory (2300), the device (2000) can determine the type of the target specimen corresponding to the specimen image using a reviewer model based on whether the confidence score falls within a predetermined range. The reviewer model may be trained to determine the type of target specimen contained in an input image. By having the processor (2100) execute one or more instructions stored in the memory (2300), the device (2000) can determine the medical level using a machine learning or rule-based system based on the type of information associated with the target specimen and the specimen image.

[0176] FIG. 14 is a flowchart of a method for processing a sample by a device according to one embodiment of the present disclosure.

[0177] In one embodiment of the present disclosure, a method for classifying a specimen included in an image may be performed by a device (2000). For example, the device (2000) may perform each step of the method for classifying a specimen included in an image by having a processor of the device (2000) execute at least one instruction included in a memory. The processor controlling the device (2000) may perform image processing and operations performed by artificial intelligence. In one embodiment, the processor may include an artificial intelligence processor. In one embodiment, the processor may include a plurality of processors.

[0178] In step S1410, the device (2000) may obtain a cartridge containing a specimen (e.g., blood or tissue). The cartridge may include a contact staining patch containing a staining sample and a specimen area where the specimen is smeared. For example, the cartridge may have the contact staining patch positioned above the specimen area where the specimen is smeared.

[0179] In one embodiment of the present disclosure, the device (2000) may include a loading area in which a cartridge is placed. The loading area may refer to a space in which a cartridge can be provided to the device (2000). For example, the device (2000) may obtain a cartridge through the loading area.

[0180] In step S1420, the device (2000) may smear a specimen contained in the cartridge on the specimen area. In one embodiment, the device (2000) may move the patch plate of the cartridge or the specimen plate of the cartridge to smear the specimen contained in the cartridge on the specimen area. For example, the device (2000) may move at least one of the patch plate or the specimen plate to create a change in relative position. The specimen may be smeared by a contact portion included in the patch plate. For example, smearing may be performed by physically moving at least a portion of the specimen according to a change in relative position between the specimen and the contact portion caused by the device (2000). The patch plate may refer to a component (e.g., a body) of the cartridge that includes a contact staining patch, and the specimen plate may refer to a component (e.g., a body) of the cartridge that includes a specimen. The patch plate and the specimen plate may include protrusions and / or grooves for mutual coupling.

[0181] In step S1430, the device (2000) can fix a specimen. In one embodiment of the present disclosure, the patch plate may include a storage portion for fixing a specimen. In one embodiment of the present disclosure, the storage portion for the fixing patch may include a fixative (e.g., methanol, ethanol). The device (2000) may discharge the fixative contained in the storage portion according to a specific operation. For example, the device (2000) may induce the discharge of the fixative by applying pressure to the storage portion.

[0182] In step S1440, the device (2000) can stain a specimen. In one embodiment of the present disclosure, the device (2000) can perform staining on a cartridge using a contact staining patch containing a staining sample. The staining sample may include a substance that stains the specimen. For example, the staining sample may include a staining reagent that directly stains the specimen, but is not limited thereto, and may include a substance that reacts with the staining target substance to enable detection of the staining target substance.

[0183] The device (2000) can perform staining by bringing a contact dye patch into contact with a specimen. For example, the specimen may undergo chemical bonding through contact with the contact dye patch. The device (2000) can sequentially bring a plurality of contact dye patches into contact with the specimen.

[0184] In one embodiment of the present disclosure, some of the dyed samples may not bind to the sample. For example, the sample and the dyed sample may not bind. Among the dyed samples in contact with the sample, the unreacted dyed samples that are not bound to the sample may be reabsorbed into the contact patch when the sample area and the contact patch are separated.

[0185] In step S1450, the device (2000) can photograph a stained specimen. The device (2000) can photograph the stained specimen using an image inspection unit (e.g., an optical lens). The device (2000) can obtain an image that is magnified (e.g., 20x, 50x, etc.) using the image inspection unit. The degree of magnification can be determined differently depending on the purpose of the examination.

[0186] The device (2000) can acquire multiple specimen images in the z-axis direction when the width of the cartridge is the x-axis and the height is the y-axis. That is, the device (2000) can acquire multiple images by changing the depth in the direction of the stacked single layer (vertical direction of the slide) in which the specimen is smeared. For example, the device (2000) can acquire multiple images at each depth within a range within the thickness of the specimen.

[0187] FIG. 15 is a block diagram illustrating a device for classifying a specimen included in an image according to one embodiment of the present disclosure.

[0188] As illustrated in FIG. 15, a device (2000) according to one embodiment of the present disclosure may include, in addition to the components of FIG. 13, a cartridge loader unit (2400), an image inspection unit (2500), a user output interface (2600), and a user input interface (2700). Not all components of the device (2000) are essential, and each component may be added or subtracted depending on the design concept of the manufacturer.

[0189] In one embodiment of the present disclosure, the processor (2100) can control the overall operation of the device (2000). The processor (2100) can control at least one of the cartridge loader unit (2400), the image inspection unit (2500), the image capture unit (2200), the user output interface (2600), the user input interface (2700), and the memory (2300) by executing programs stored in the memory (2300).

[0190] The cartridge loader unit (2400) is a unit that obtains a cartridge containing a specimen, and after the cartridge is loaded, the specimen can be smeared and mounted so that it can be inspected by the image inspection unit (2500).

[0191] The image inspection unit (2500) is an optical device, such as a microscope, that can magnify and view a specimen area. The image inspection unit (2500) may be provided together with the device (2000) or may be a separately provided optional device. The image inspection unit (2500) can be used to magnify and confirm a smeared specimen placed on the cartridge loader unit (2400).

[0192] The image capture unit (2200) can capture an image of a red blood cell specimen confirmed through the image inspection unit (2500). In one embodiment, the device (2000) can capture an image of the entire area of ​​a slide contained in a cartridge.

[0193] The user output interface (2600) is for outputting audio signals or video signals and may include a display unit (2610). Although not shown, the user output interface (2600) may optionally include an audio output unit at the manufacturer's option.

[0194] According to one embodiment of the present disclosure, the device (2000) can display information related to the device (2000) through the display unit (2610). For example, the medical level determined by the device (2000) and information related to the medical level can be displayed on the display unit (2610).

[0195] According to one embodiment of the present disclosure, the display unit (2610) and the touchpad may be configured as a touch screen by forming a layer structure. When the display unit (2610) and the touchpad are configured as a touch screen by forming a layer structure, the display unit (2610) may be used as an input device in addition to an output device. The display unit (2610) may include at least one of a liquid crystal display, a thin film transistor-liquid crystal display, a light-emitting diode (LED), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an electrophoretic display. In addition, depending on the implementation form of the device (2000), two or more display units (2610) may be included.

[0196] According to one embodiment of the present disclosure, the user output interface (2600) can output medical levels and information related to the medical levels through the display unit (2610). According to one embodiment of the present disclosure, the output interface (2500) can also display the current power level, operation mode (e.g., image inspection mode, image classification mode, sleep mode, etc.), etc.

[0197] The user input interface (2700) is for receiving input from a user. The user input interface (2700) may be at least one of a key pad, a dome switch, a touch pad (contact electrostatic capacitance type, pressure resistive film type, infrared detection type, surface ultrasonic conduction type, integral tension measurement type, piezo effect type, etc.), a jog wheel, and a jog switch, but is not limited thereto.

[0198] The user input interface (2700) may include a voice recognition module. For example, the device (2000) may receive a voice signal, which is an analog signal, through a microphone and convert the voice portion into computer-readable text using an Automatic Speech Recognition (ASR) model. The device (2000) may interpret the converted text using a Natural Language Understanding (NLU) model to obtain the user's utterance intent. Here, the ASR model or the NLU model may be an artificial intelligence model. The artificial intelligence model may be processed by an artificial intelligence processor designed with a hardware structure specialized for processing artificial intelligence models. In this case, the processor (2100) may be an artificial intelligence processor. The artificial intelligence model may be created through learning. Here, being created through learning means that a basic artificial intelligence model is learned using a plurality of training data by a learning algorithm, thereby creating a predefined operation rule or artificial intelligence model set to perform a desired characteristic (or purpose). The artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weight values.

[0199] Linguistic understanding is the technology of recognizing, applying, and processing human language / characters, including natural language processing, machine translation, dialog systems, question answering, and speech recognition / synthesis.

[0200] The memory (2300) may store a program for processing and controlling the processor (2100), and may store input / output data (e.g., diagnostic information of the device (2000), an image of a sample area, or information related to a medical level). The memory (2300) may also store an artificial intelligence model.

[0201] The memory (2300) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk. In addition, the control device (2000) may operate a web storage or cloud server that performs a storage function on the Internet.

[0202] The communication unit (2800) may include a short-range communication unit (2810) and a long-range communication unit (2820). The short-range communication unit (2810, short-range wireless communication interface) may include, but is not limited to, a Bluetooth communication unit, a BLE (Bluetooth Low Energy) communication unit, a near field communication interface, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an IrDA (infrared Data Association) communication unit, a WFD (Wi-Fi Direct) communication unit, a UWB (Ultra Wideband) communication unit, an ANT+ communication unit, etc. The long-range communication unit (2820) transmits and receives a wireless signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to a voice call signal, a video call call signal, or a text / multimedia message transmission and reception. The remote communication unit (2820) may include, but is not limited to, a 3G module, a 4G module, a 5G module, an LTE module, an NB-IoT module, an LTE-M module, etc.

[0203] According to one embodiment of the present disclosure, communication can be performed with a server or other electrical device external to the device (2000) through the communication unit (2800) and data can be transmitted and received. The communication unit (2800) may be optionally included for the purpose of sales or price competitiveness of the device (2000), or may not be included if communication is not required.

[0204] A method according to an embodiment of the present disclosure may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the present disclosure or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0205] An embodiment of the present disclosure may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically contains computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism, and includes any information delivery media. Furthermore, some embodiments of the present disclosure may also be implemented as a computer program or computer program product containing computer-executable instructions, such as a computer program that is executed by a computer.

[0206] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0207] According to one embodiment, the method according to one embodiment of the present disclosure may be installed in a memory within a device and executed by a processor of the device. According to one embodiment, the method according to one embodiment of the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Claims

In a method for classifying a specimen included in an image, A step of generating a plurality of specimen images from images of specimens, each of the plurality of specimen images including a target specimen; A step of obtaining the type and reliability score of the target specimen corresponding to each specimen image using a classification model, wherein the reliability score is characterized in that it is related to the type of the target specimen; A step of determining the type of the target specimen corresponding to the specimen image using a reviewer model based on the reliability score being within a predetermined range; A step of determining a medical level using a machine learning or rule-based system based on the type of the target specimen and information related to the specimen image is included. The above classification model is trained to output a confidence score related to the type of target sample included in the input image, A method characterized in that the above reviewer model is trained to determine the type of target specimen included in an input image. In the first paragraph, The step of generating the above multiple sample images comprises: A step of identifying information related to the locations of a plurality of target specimens included in the image and the sizes of the plurality of target specimens by inputting the image into a third artificial intelligence model; and A step of obtaining the plurality of specimen images from the image based on information related to the positions of the plurality of target specimens and the sizes of the plurality of target specimens, A method characterized in that the third artificial intelligence model is trained to identify information related to the location and size of one or more target specimens included in an input image. In the first paragraph, The images for the above specimens include multiple images taken at multiple focal distances, The step of generating the above multiple sample images comprises: A step of determining one of the above multiple focal lengths as an optimal focal length; A step of selecting an image having an optimal focal length from among the plurality of captured images; and A method comprising the step of generating a specimen image including the target specimen from the selected image. In the first paragraph, A method further comprising a step of determining the type of the target sample using the confidence score based on the confidence score not being within a predetermined range. In the first paragraph, The step of determining the type of the above target sample is: a step of enlarging the above specimen image; and A method comprising a step of determining the type of the target specimen corresponding to the enlarged specimen image using the reviewer model. In paragraph 5, The step of determining the type of the target specimen corresponding to the enlarged specimen image is: A step of dividing the enlarged sample image into multiple patches; and A method comprising the step of inputting the plurality of patches into the reviewer model to determine the type of the target specimen. In paragraph 5, The step of enlarging the above sample image is: A step of acquiring a first focus image captured with a shorter focal length than the above specimen image; A step of acquiring a second focus image captured with a longer focal length than the above specimen image; A step of generating a corrected specimen image based on the first focus image, the second focus image, and the specimen image; and A method comprising the step of enlarging the above-mentioned corrected sample image. In the first paragraph, Further comprising a step of selecting at least some of the plurality of sample images using a fourth artificial intelligence model, The steps for obtaining the above reliability score are: a step of enlarging at least some of the selected sample images; and A method comprising the step of inputting the enlarged specimen image into the classification model and obtaining the confidence score corresponding to each enlarged specimen image. In the first paragraph, A step of transmitting the determined medical level to an external device or outputting the medical level through a display; A step of obtaining user input for modifying the above medical level; In response to the user input obtained above, a step of modifying the medical level and storing information related to the modified medical level; and A method further comprising a step of determining a reclassification rule using the stored information. In the first paragraph, A method, characterized in that the target sample comprises at least one of red blood cells, white blood cells (WBC), platelets, or cervical cells. In a device for classifying a specimen included in an image, memory containing one or more instructions; and Contains at least one processor, By causing said at least one processor to execute said one or more instructions, said device Generating a plurality of specimen images from images of specimens, each of the plurality of specimen images including a target specimen, Using a classification model, the type and reliability score of the target sample corresponding to each sample image are obtained, and the reliability score is characterized in that it is related to the type of the target sample. The type of the target specimen corresponding to the specimen image is determined using a reviewer model based on the reliability score being within a predetermined range, Based on the type of the target specimen and information related to the specimen image, a medical level is determined using a machine learning or rule-based system, The above classification model is trained to output a confidence score related to the type of target sample included in the input image, A device characterized in that the above reviewer model is trained to determine the type of target specimen included in an input image. In Article 11, By causing said at least one processor to execute said one or more instructions, said device By inputting the above image into a third artificial intelligence model, information related to the locations of multiple target specimens included in the image and the sizes of the multiple target specimens is identified, Acquire the plurality of sample images from the image based on information related to the positions of the plurality of target samples and the sizes of the plurality of target samples, A device characterized in that the third artificial intelligence model is trained to identify information related to the location and size of one or more target specimens included in an input image. In Article 11, The images for the above specimens include multiple images taken at multiple focal distances, By causing said at least one processor to execute said one or more instructions, said device One of the above multiple focal lengths is determined as the optimal focal length, Select an image with the optimal focal length from among the multiple images captured above, A device for generating a specimen image including the target specimen from the selected image. In Article 11, By causing said at least one processor to execute said one or more instructions, said device A device for determining the type of the target sample using the confidence score based on the confidence score not being within a predetermined range. In Article 11, By causing said at least one processor to execute said one or more instructions, said device Enlarge the above sample image, A device for determining the type of the target specimen corresponding to the enlarged specimen image using the above reviewer model. In Article 15, By causing said at least one processor to execute said one or more instructions, said device Divide the above enlarged sample image into multiple patches, A device for determining the type of the target specimen by inputting the plurality of patches into the reviewer model. In Article 15, By causing said at least one processor to execute said one or more instructions, said device Acquire a first focus image captured with a shorter focal length than the above specimen image, Acquire a second focus image captured with a longer focal length than the above specimen image, Generating a corrected specimen image based on the first focus image, the second focus image, and the specimen image, A device for magnifying the above-mentioned corrected sample image. In Article 11, By causing said at least one processor to execute said one or more instructions, said device Using the fourth artificial intelligence model, at least some of the plurality of sample images are selected, Enlarge at least some of the sample images selected above, A device that inputs the enlarged specimen image into the classification model and obtains the reliability score corresponding to each enlarged specimen image. In Article 11, By causing said at least one processor to execute said one or more instructions, said device Transmitting the above-determined medical level to an external device or outputting the medical level through a display, Obtain user input to modify the above medical level, In response to the user input obtained above, modifying the medical level and storing information related to the modified medical level, A device that determines a reclassification rule using the stored information. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 1 to 10 on a computer.

Images