Disease diagnosis and visualization method and apparatus using symptomatic organ classification information

Abstract

Disclosed are disease diagnosis and visualization method and apparatus using symptomatic organ classification information. Provided is a computer-implemented method for training a deep learning model for disease diagnosis, according to one aspect of the present disclosure, the method comprising the steps of: training a symptomatic organ classification model by using first training data in which symptomatic organ information is tagged in a medical image; acquiring location information of a symptomatic organ predicted from the medical image by means of the trained symptomatic organ classification model, and the name of the symptomatic organ corresponding to the location information; and training a detailed disease classification model by using the location information, the name, and second training data in which detailed disease information is tagged in the medical image.

Description

Disease diagnosis and visualization method and device using symptomatic organ classification information

[0001] The present disclosure relates to a disease diagnosis and visualization method and device using symptomatic organ classification information.

[0002] The content described below merely provides background information related to the present embodiment and does not constitute prior art.

[0003] With low birth rates and an aging population, more and more people are considering their pets as family members, leading to a growing demand for pet health and disease management. In particular, interest is growing in AI-based pet disease diagnosis services that utilize deep learning models to analyze and provide advice on disease from medical or veterinary images.

[0004] Conventional pet disease diagnosis techniques typically train deep learning models to classify diseases from animal radiographic images, and then use the trained deep learning models to diagnose them. These models are trained to classify specific diseases that veterinarians require when diagnosing companion animals, such as "gastric dilatation, gastric dilatation and volvulus, small bowel dilatation, hepatomegaly, splenomegaly, renal stones, renal hypertrophy, ureteral stones, bladder stones, intra-abdominal masses (liver, spleen, etc.), and prostate enlargement." Training data is collected accordingly.

[0005] However, unlike real-life images, it's difficult to distinguish between organs requiring diagnosis in companion animal X-ray images. Training a deep learning model to diagnose specific diseases from X-ray images from the outset can lead to problems like overfitting, where the model memorizes patterns in the training data rather than extracting discriminative features. Furthermore, these training difficulties can increase the likelihood of misclassifying diseases during inference.

[0006] Furthermore, despite being different diseases, similar visual features can appear in X-ray images. For example, hepatomegaly, which is enlargement of a specific area of ​​the liver, and intra-abdominal masses (liver) are distinct diseases, but they exhibit similar visual features in X-ray images. Therefore, when training a deep learning model to diagnose a disease in detail at once, the training process can become even more challenging, as it must distinguish between multiple diseases with similar visual features.

[0007] The main purpose of the present disclosure is to provide a disease diagnosis method and device using symptomatic organ classification information.

[0008] The present disclosure aims to provide a training and inference method for an artificial intelligence model that can provide visual information on a suspected disease site along with a disease diagnosis.

[0009] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0010] According to one aspect of the present disclosure, a computer-implemented method for training a deep learning model for disease diagnosis is provided, the method including: a step of training a symptomatic organ classification model using first training data in which symptomatic organ information is tagged in a medical image; a step of obtaining location information of a symptomatic organ predicted from the medical image and a name of the symptomatic organ corresponding to the location information through the trained symptomatic organ classification model; and a step of training a detailed disease classification model using second training data in which detailed disease information is tagged in the medical image and the location information and the name.

[0011] According to another aspect of the present disclosure, a computer-implemented method for diagnosing a disease is provided, comprising: receiving a target medical image and predicting a symptomatic organ corresponding to the target medical image using a trained symptomatic organ classification model; obtaining location information of the predicted symptomatic organ and a name of the symptomatic organ corresponding to the location information; and inputting the target medical image, the location information, and the name into a trained detailed disease classification model to predict detailed disease information corresponding to the target medical image.

[0012] According to another aspect of the present disclosure, there is provided a device comprising one or more processors and a memory operably coupled to the one or more processors, wherein the memory stores instructions for causing the one or more processors to perform operations in response to execution of instructions by the one or more processors, the operations including: a step of training a symptomatic organ classification model using first training data in which symptomatic organ information is tagged in a medical image; a step of obtaining location information of a symptomatic organ predicted from the medical image and a name of the symptomatic organ corresponding to the location information through the trained symptomatic organ classification model; and a step of training a detailed disease classification model using second training data in which detailed disease information is tagged in the medical image and the location information and the name.

[0013] According to an embodiment of the present disclosure, there is an effect of improving the accuracy of disease diagnosis.

[0014] According to an embodiment of the present disclosure, the reliability of the disease diagnosis assistance service can be increased by providing the user with visual information on the disease diagnosis result and the suspected disease area together.

[0015] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0016] FIG. 1 is a diagram illustrating the configuration and data flow of a process for training multiple deep learning models for disease diagnosis according to one embodiment of the present disclosure.

[0017] FIG. 2 is an example diagram of a training dataset for symptomatic organ classification constructed from a training dataset for existing detailed disease classification according to one embodiment of the present disclosure.

[0018] FIG. 3 is a diagram illustrating a process for generating bounding boxes and name information of an inferred symptomatic organ according to one embodiment of the present disclosure.

[0019] FIG. 4 is a diagram illustrating a detailed training process of a detailed disease classification model according to one embodiment of the present disclosure.

[0020] FIG. 5 is a diagram for explaining a process of extracting region-specific features from a feature map based on bounding box information of a symptomatic organ according to one embodiment of the present disclosure.

[0021] FIG. 6 is a diagram illustrating an example of determining a location-specific learning label according to one embodiment of the present disclosure.

[0022] FIG. 7 is a diagram illustrating the configuration and data flow of a process for inferring a disease by utilizing symptomatic organ classification information according to one embodiment of the present disclosure.

[0023] FIG. 8 is a diagram for explaining a detailed inference process of a detailed disease classification model according to one embodiment of the present disclosure.

[0024] FIG. 9 shows an example of visualizing input images and diagnosis results using a disease diagnosis method based on symptomatic organ information according to one embodiment of the present disclosure.

[0025] FIG. 10 is a flowchart of a method for training a deep learning model for disease diagnosis according to one embodiment of the present disclosure.

[0026] FIG. 11 is a flowchart of a method for diagnosing a disease using symptomatic organ classification information according to one embodiment of the present disclosure.

[0027] Fig. 12 is a block diagram of a disease diagnosis device according to one embodiment of the present disclosure.

[0028] Hereinafter, some embodiments of the present disclosure will be described in detail using exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, when describing the present disclosure, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present disclosure.

[0029] In describing components of embodiments according to the present disclosure, symbols such as first, second, i), ii), a), b) may be used. These symbols are only for distinguishing the components from other components, and the nature, order, or sequence of the components are not limited by the symbols. When a part in the specification is said to "include" or "have" a component, this does not mean that other components are excluded, but rather that other components may be included, unless explicitly stated otherwise.

[0030] The detailed description set forth below, together with the accompanying drawings, is intended to explain exemplary embodiments of the present disclosure and is not intended to represent the only embodiments in which the present disclosure may be practiced.

[0031] In this specification, medical images refer to radiological images taken of a subject (e.g., an animal or a human) using radiation for the purpose of diagnosing, treating, or preventing diseases of the subject, and the term 'medical images' may be used interchangeably with the term 'radiological images' or the term 'X-ray images'.

[0032] Unlike conventional techniques that diagnose diseases in detail using a single deep learning model, the present disclosure provides a disease diagnosis method that first identifies a body part or organ (hereinafter referred to as "symptomatic organ") suspected of having a disease in a medical image and then diagnoses the specific type of disease centered on the symptomatic organ. Furthermore, the present disclosure visualizes the symptomatic organ as a bounding box, providing users with reliable information regarding the reasons for the disease diagnosis.

[0033] The present disclosure provides a method for first extracting symptomatic organs from medical images and then diagnosing specific diseases based on the extracted symptomatic organs. To this end, multiple models, i.e., a symptomatic organ classification model (200) and a specific disease classification model (400), are sequentially trained. In the process of diagnosing a disease from a target medical image using the trained multiple models, the results are output based on whether the extracted disease occurs in the extracted symptomatic organ.

[0034] Referring to FIGS. 1 to 6, the process of training multiple deep learning models for disease diagnosis according to one embodiment of the present disclosure is described in four steps.

[0035] The first step is to build a symptomatic organ dataset, in which a dataset (110, hereinafter referred to as the "symptomatic organ dataset") is built in which symptomatic organ information is tagged to medical images. This is to alleviate the difficulty of the problem by first performing supervised learning on a symptomatic organ classification model (200) as there is a high possibility that diagnostic performance and visualization results will be unstable if detailed disease data is directly used for learning.

[0036] Datasets for training disease diagnosis models are typically built to classify detailed diseases that actual doctors need when diagnosing subjects, such as "gastric dilatation, gastric foreign bodies, gastric dilatation and volvulus, small bowel dilatation, hepatomegaly, splenomegaly, renal stones, renal hypertrophy, ureteral stones, bladder stones, intra-abdominal masses (liver, spleen, etc.), prostate calcification, prostate enlargement, and uterine enlargement." In other words, the existing dataset is a dataset (100, hereinafter referred to as the "detailed disease dataset") in which detailed disease information is tagged to medical images.

[0037] Based on the knowledge of the relationship between a detailed disease and the organ in which the disease occurs, a new symptomatic organ dataset (110) is constructed for classifying symptomatic organs such as “stomach, small intestine, liver, spleen, kidney, urinary tract, prostate, uterus, etc.” without additional data collection using the existing detailed disease dataset (100). For example, if the detailed disease tagged in a specific X-ray image is “stomach dilatation,” the symptomatic organ of the image can be created as “stomach.” An additional example is illustrated in Fig. 2.

[0038] The second is a symptomatic organ classification model learning step, in which a symptomatic organ classification model (200) is trained based on the constructed symptomatic organ dataset (110). The symptomatic organ classification model (200) can use any deep learning network that receives images as input, such as a convolutional neural network (CNN), a transformer, etc., and has a feature extractor and a classifier. The symptomatic organ classification model (200) is configured to perform multi-label classification that infers all symptomatic organs that may exist when a medical image is given. Training and inference of the symptomatic organ classification model (200) can be performed as described below, but the method is not limited as long as multi-label classification can be performed.

[0039] The training process of the symptomatic organ classification model (200) is described.

[0040] X-ray images included in the symptomatic long-term dataset (110) And the symptomatic organ information given as the correct answer in the corresponding video Let's assume that there is It is made up of each and is the number of symptomatic organs diagnosed. In addition, the feature extractor of the symptomatic organ classification model (200) Features through When a classifier is given The probability values ​​for each symptomatic organ obtained through vector If expressed as X-ray image Training is performed to minimize the MBCE (Multi-label Binary Cross Entropy) loss function of Equation 1.

[0041]

[0042] Here, Is of It refers to the probability value that a disease exists in the second organ.

[0043] Also, the data given for training is If it's a dog It can be expanded to .

[0044] The symptomatic organ classification model (200) that has completed training can determine whether an organ is symptomatic based on the threshold for each organ. That is, X-ray image Enter the probability value for each symptomatic organ When it was inferred, Back side It can be determined that there is a problem in the second organ. Here, the critical value of each organ can be obtained using Youden's J Statistics, but the method is not limited.

[0045] The third step is to create a symptomatic organ bounding box, which is an input X-ray image of a symptomatic organ inferred based on a trained symptomatic organ classification model (200). My location, i.e., bounding box information, is generated. This is to extract symptomatic organ area information in the form of bounding boxes and provide it as input to the detailed disease classification model (400), so that the detailed disease classification model (400) can focus on and train the area suspected of being diseased.

[0046] The inference process is performed using the symptomatic organ classification model (200) for which training has been completed. That is, the X-ray image included in the symptomatic organ dataset (110) And the probability value for each symptomatic organ through the symptomatic organ classification model (200) When given, A set of indexes of symptomatic organs suspected of having the corresponding disease is obtained. At this time, am.

[0047] The symptomatic organs inferred through this process (i.e., ) to obtain CAM (Class Activation Map) only for X-ray images Visualize why the symptomatic organ classification model (200) determined that a problem exists in a given organ due to its location. For CAM visualization, an explainable artificial intelligence (XAI) algorithm that can visualize the classification reasons of a deep learning model can be used.

[0048] since, CAM results of the second symptomatic organ If you say so than the value All values ​​smaller than this are removed and the remaining area is based on Generate a bounding box for the area where the second organ is judged to have a problem. Here, is a hyperparameter defined by the user and is used to remove noisy areas.

[0049] Figure 3 shows an example of the steps for generating bounding boxes for symptomatic organs. Through steps ① to ④, the bounding box location information is finally generated. (yes: ) and the symptomatic long-term name corresponding to the corresponding bounding box location. (e.g. stomach, intestines, liver, etc.) are saved as the final result. For example, the classification results that are greater than the threshold are "liver" and "spleen". And, like the “liver” exemplified in Fig. 3, the “spleen” also individually performs processes ① to ④ to store BB4 location and O4 (spleen) information.

[0050] The final step is the detailed disease classification model learning step, in which the detailed disease classification model (400) is trained. After the first to third steps described above are all completed, the detailed disease classification model (400) is trained. The detailed disease classification model (400) can utilize any deep learning network that receives images as input and includes a feature extractor and classifier, such as a convolutional neural network or a transformer. The detailed disease classification model (400) is configured to perform multi-label classification, inferring all detailed diseases that may exist when a medical image is given.

[0051] X-ray images included in the detailed disease dataset (100) and the detailed disease name given as the correct answer in the corresponding video , bounding box location information of symptomatic organ area obtained through the symptomatic organ area bounding box generation step and the name of the symptomatic organ is used as input. At this time, Referring to Figure 4, an overall schematic diagram of the detailed disease classification model training process is shown.

[0052] X-ray images using feature extractor A feature map is extracted from (S410).

[0053] Symptomatic organ area A process of extracting region-specific features from the extracted feature map (hereinafter referred to as “region-specific feature extraction”) is performed (S420). This is to train a model to classify detailed diseases centered on symptomatic organ parts.

[0054] The extracted feature map is Assume that it has a size of . At this time, each The area of ​​features that overlap with the coordinates corresponding to the feature map If so, the final feature for that area is through Global Average Pooling (GAP). Obtained as a feature by area of ​​size. Feature map areas that do not correspond to If defined as, The corresponding area is without any special transformation The size characteristics are directly utilized. Figure 5 shows an example of feature extraction by region.

[0055] Obtain probability values ​​for each detailed disease by region (S430). Classifier of the detailed disease classification model (400) The probability values ​​for each disease obtained through vector is expressed as . Here, is the number of detailed diseases to be diagnosed is described in the S420 course. Or am.

[0056] Meanwhile, to train a detailed disease classification model (400) to classify detailed diseases based on symptomatic organ sites, learning labels for each location are determined (S440). Specifically, symptomatic organ names and input video Detailed disease name that is the correct answer We can use it to determine learning labels by location. Here, Silver input video is the number of detailed diseases given as the correct answer. Here, the learning labels by location are each feature by region extracted from the S420 process. Ground truth for final detailed disease names for learning about Says.

[0057] First, the detailed disease name Using each The corresponding symptomatic organ name Creates. After that, The corresponding name and the name of the symptomatic organ Compares to check if there is a matching element.

[0058] If there is no matching symptomatic organ Give a normal name to it. If there is a matching symptomatic organ An example of determining learning labels by location is shown in Fig. 6.

[0059] Finally, the correct name of the detailed disease One-hot encoding to create multiple labels consisting of 0 or 1 Convert to . Corresponding to each name My location is defined in advance by the user.

[0060] Features of areas that do not correspond to the bounding box is not an area inferred to be a symptomatic organ area. One-hot encoding is performed using the normal name. For example, one-hot encoding for the normal name is can be used.

[0061] A detailed disease classification model (400) is trained based on the probability values ​​for each detailed disease obtained by region and the learning labels determined by location (S450). Given an input image Training can be done to minimize the loss function of Equation 2.

[0062]

[0063] Here, MBCE (Multi-label Binary Cross Entropy) of Equation 3 can be used, and the method is not limited as long as it can perform multi-label classification.

[0064]

[0065] Here, It consists of and each Silver input video to The correct answer value indicates the presence or absence of the second detailed disease.

[0066] A process for diagnosing a disease using symptomatic organ classification information according to one embodiment of the present disclosure is described.

[0067] Referring to FIGS. 7 and 8, an X-ray image (120) is received, and symptomatic organ classification information corresponding to the X-ray image (120) is obtained using a symptomatic organ classification model (200) for which learning has been completed. Specifically, a feature map can be extracted from the X-ray image (120) using the symptomatic organ classification model (200), and a probability value for each symptomatic organ can be obtained.

[0068] Through the symptomatic organ bounding box generation step (300), the location of each symptomatic organ and name Specifically, the symptomatic organ is inferred by comparing the probability value for each symptomatic organ with the corresponding threshold value, and the location of each symptomatic organ is obtained by using the CAM algorithm for the inferred symptomatic organ. and name can be obtained.

[0069] X-ray images (120) and locations of symptomatic organs as inputs to the detailed disease classification model (400) and name Use it.

[0070] A feature map can be extracted from an X-ray image (120) using a feature extractor (S810).

[0071] Unlike the learning process of the detailed disease classification model (400), The feature corresponding to the feature map area of Only region-specific features are extracted (S820). That is, regions other than the bounding box are not considered during inference.

[0072] After this, the classifier Using the probability values ​​for each disease by region and obtain (S830), Back side We infer that there is a third disease. Here, the threshold for each sub-disease can be obtained using Youden's J Statistics, but the method is not limited.

[0073] Finally, the inferred detailed disease and symptomatic organ names received as input Determine the correlation between the organs (S840). Specifically, the symptomatic organ names corresponding to the inferred detailed diseases (can be converted using the symptomatic organ dataset construction step) and the symptomatic organ names received as input. Determines whether they match.

[0074] If they match, the detailed disease inference results are used as the final diagnosis results, and if they do not match, they are considered normal. In other words, the detailed disease inference results (e.g., hepatomegaly) are considered as symptomatic organs. If it does not fall under the sub-disease category (e.g., prostate), the detailed disease inference result is judged to be inaccurate.

[0075] In addition, for visualization of the suspected disease area, the final diagnosis result (detailed disease name and inferred probability value) and the location of the symptomatic organ area are provided. It is displayed by outputting it to a video (S850). At this time, it can be seen as efficient because the results of the symptomatic organ classification process can be reused instead of applying the visualization technique again using the detailed disease classification model (400).

[0076] Figure 9 illustrates an example of visualizing input images and diagnostic results using a detailed disease diagnosis method based on symptomatic organ information. Referring to Figure 9, the user is ultimately presented with visualized results in the form of bounding boxes, including the subject's detailed disease name, confidence value, and the specific organ affected by the disease.

[0077] FIG. 10 is a flowchart of a method for training a deep learning model for disease diagnosis according to one embodiment of the present disclosure.

[0078] Referring to Figure 10, first training data is constructed with symptomatic organ information tagged to the medical image (S1010). The first training data can be constructed by modifying the second training data with detailed disease information tagged to the medical image.

[0079] The symptomatic organ classification model (200) is trained using the first training data (S1020).

[0080] Specifically, a symptomatic organ classification model (200) can be trained to minimize multi-label binary cross entropy (MBCE) based on symptomatic organ information predicted from the medical image and tagged symptomatic organ information.

[0081] Using the trained symptomatic organ classification model (200), information on the bounding box and name of the symptomatic organ predicted from the medical image is generated (S1030).

[0082] Specifically, the symptomatic organ classification model (200) is used to predict the symptomatic organ corresponding to the medical image. A Class Activation Map (CAM) is calculated for each predicted symptomatic organ. The calculated CAM is used to generate a bounding box for the corresponding symptomatic organ. The generated bounding box and the name of the corresponding symptomatic organ are obtained.

[0083] A detailed disease classification model is trained using the detailed disease dataset (100) and the bounding boxes and names of symptomatic organs (S1040).

[0084] Specifically, based on the bounding box, features are extracted for each region from a feature map extracted from the medical image. Based on the extracted features for each region, probability values ​​for each detailed disease are obtained for each region. Learning labels for each location are determined using the name and detailed disease information tagged in the second training data. A detailed disease classification model (400) is trained to minimize the sum of the multi-label binary cross entropy (MBCE) for each region based on the probability values ​​for each detailed disease for each region and the learning labels for each location.

[0085] FIG. 11 is a flowchart of a method for diagnosing a disease using symptomatic organ classification information according to one embodiment of the present disclosure.

[0086] Referring to Fig. 11, a target medical image is received (S1110).

[0087] Using the trained symptomatic organ classification model (200), the symptomatic organ corresponding to the target medical image is predicted (S1120).

[0088] Obtain the bounding box and name of the predicted symptomatic organ (S1130). Specifically, a Class Activation Map (CAM) is calculated for each predicted symptomatic organ. Using the calculated CAM, a bounding box of the corresponding symptomatic organ is generated. The generated bounding box and the name of the symptomatic organ corresponding to the bounding box are obtained.

[0089] The above target medical image, the bounding box, and the name are input into a trained detailed disease classification model (400) to diagnose the detailed disease corresponding to the target medical image (S1140). Specifically, the target number of images and the bounding box are input into the detailed disease classification model (400) to obtain a detailed disease classification result. It is determined whether there is a correlation between the obtained detailed disease classification result and the name. If it is determined that there is a correlation, the obtained detailed disease classification result is predicted as a diagnosis result. If it is determined that there is no correlation, it is predicted as normal without the detailed disease.

[0090] The predicted detailed disease information and bounding boxes can be overlaid on the target medical image to visualize the diagnosis results and provide them to the user.

[0091] Fig. 12 is a block diagram of a disease diagnosis device according to one embodiment of the present disclosure.

[0092] Referring to FIG. 12, the disease diagnosis device (1200) includes all or part of an input / output interface (1210), a processor (1220), and a memory (1230). Here, the input / output interface (1210), the processor (1220), and the memory (1230) can transmit data to each other via a bus (1240).

[0093] The input / output interface (1210) can be connected to an external device to receive medical images for diagnosing a subject's disease. The input / output interface (1210) may further include hardware modules, such as a network interface card, a network interface chip, and a networking interface port, or software modules, such as a network device driver or a networking program, to establish a communication connection.

[0094] The input / output interface (1210) can transmit disease diagnosis result data generated by the processor (1220) to an external server or storage device. For example, the disease diagnosis device (1200) can transmit disease diagnosis result data to a disease diagnosis service management server via the input / output interface (1210), and the service provider can provide real-time disease diagnosis result data to the user.

[0095] The processor (1220) can perform all steps of a method for training a deep learning model for disease diagnosis according to one embodiment of the present disclosure.

[0096] The processor (1220) can perform all steps of a method for diagnosing a disease by utilizing symptomatic organ classification information according to one embodiment of the present disclosure.

[0097] The memory (1230) may include volatile memory, permanent memory, virtual memory, or other types of memory for storing information used by or output by the disease diagnosis device (1200). For example, the memory (1230) may include random access memory (RAM) or dynamic RAM (DRAM).

[0098] The memory (1230) stores basic programs, application programs, network setting information, etc. for the operation of the disease diagnosis device (1200). In addition, the memory (1230) can provide the stored information upon request from either the input / output interface (1210) or the processor (1220).

[0099] The memory (1230) can store various data for processing or controlling the disease diagnosis device (1200). For example, the memory (1230) can store a trained symptomatic organ classification model (200) and a detailed disease classification model (400).

[0100] Additionally, the memory (1230) can store various data generated by the disease diagnosis device (1200). For example, the memory (1230) can store data regarding the bounding box of the symptomatic organ generated by the processor (1220) and its corresponding name.

[0101] Each component of the device or method according to the present invention may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0102] Various implementations of the systems and techniques described herein may be implemented as digital electronic circuits, integrated circuits, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations of one or more computer programs executable on a programmable system. The programmable system includes at least one programmable processor (which may be a special purpose processor or a general purpose processor) coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device. Computer programs (also known as programs, software, software applications, or code) include instructions for the programmable processor and are stored on a "computer-readable recording medium."

[0103] A computer-readable recording medium includes any type of recording device that stores data that can be read by a computer system. Such a computer-readable recording medium may be a non-volatile or non-transitory medium such as a ROM, CD-ROM, magnetic tape, floppy disk, memory card, hard disk, magneto-optical disk, storage device, and may further include a transitory medium such as a data transmission medium. Furthermore, the computer-readable recording medium may be distributed across network-connected computer systems, so that computer-readable code can be stored and executed in a distributed manner.

[0104] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0105] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0106] (Explanation of symbols)

[0107] 100: Detailed disease dataset

[0108] 110: Symptomatic Long-Term Dataset

[0109] 200: Symptomatic Organ Classification Model

[0110] 400: Detailed disease classification model

[0111] CROSS-REFERENCE TO RELATED APPLICATION

[0112] This patent application claims priority to Korean Patent Application No. 10-2023-0126429, filed on September 21, 2023, the entire contents of which are incorporated herein by reference.

Claims

1. A computer implementation method for training a deep learning model for disease diagnosis, A process of training a symptomatic organ classification model using first training data tagged with symptomatic organ information in medical images; A process of obtaining location information of symptomatic organs predicted from the medical image and names of symptomatic organs corresponding to the location information through a trained symptomatic organ classification model; and A method comprising a process of training a detailed disease classification model using second training data tagged with detailed disease information in the medical image and the location information and the name.

2. In paragraph 1, A method wherein the first training data is generated using the second training data.

3. In paragraph 1, The process of training the above symptomatic organ classification model is as follows: A method for training to minimize multi-label binary cross entropy (MBCE) based on symptomatic organ information predicted from the above medical image and tagged symptomatic organ information.

4. In paragraph 1, The process of obtaining the above is, A process of predicting a symptomatic organ corresponding to the medical image using the above symptomatic organ classification model; The process of calculating a Class Activation Map (CAM) for each predicted symptomatic organ; A process of generating a bounding box of a corresponding symptomatic organ using the above CAM; and A method comprising a process of obtaining the bounding box and the name of the symptomatic organ corresponding to the bounding box.

5. In paragraph 4, The process of predicting the symptomatic organ corresponding to the above medical image is as follows: A process of obtaining a probability value for each symptomatic organ using the above symptomatic organ classification model; and A method including a process of predicting a symptomatic organ corresponding to the medical image when the probability value for each symptomatic organ is greater than or equal to a threshold value.

6. In paragraph 1, The process of training the above detailed disease classification model is as follows: A process of extracting features for each region from a feature map extracted from the medical image based on the above location information; A process of obtaining probability values ​​for each detailed disease by region based on the extracted region-specific features; A process of determining a learning label for each location using the above name and detailed disease information tagged in the second training data; and A method comprising a training process to minimize the sum of the multi-label binary cross entropy (MBCE) for each region based on the probability value for each detailed disease for each region and the learning label for each location.

7. A computer-implemented method for diagnosing a disease, A process of receiving a target medical image and predicting a symptomatic organ corresponding to the target medical image using a trained symptomatic organ classification model; A process of obtaining location information of a predicted symptomatic organ and a name of the symptomatic organ corresponding to the location information; and A method comprising a process of predicting detailed disease information corresponding to the target medical image by inputting the target medical image, the location information, and the name into a trained detailed disease classification model.

8. In paragraph 7, The process of obtaining the above is, The process of calculating a CAM (Class Activation Map) for each of the predicted symptomatic organs; A process of generating a bounding box of a corresponding symptomatic organ using the above CAM; and A method comprising a process of obtaining the bounding box and the name of the symptomatic organ corresponding to the bounding box.

9. In paragraph 7, The process of predicting the above detailed disease information is: A process of inputting the above target medical image and the above location information into the above detailed disease classification model to obtain a detailed disease classification result; A process for determining whether there is a correlation between the above detailed disease classification results and the above name; and A method comprising a process of predicting the detailed disease classification result as detailed disease information corresponding to the target medical image, if the above correlation is determined to exist.

10. In paragraph 7, A method further comprising a process of visualizing the predicted detailed disease information and the location information by overlaying them on the target medical image.

11. A device comprising one or more processors and a memory operably coupled to the one or more processors, The above memory stores instructions that cause one or more processors to perform operations in response to execution of instructions by one or more processors, The above actions are, A process of training a symptomatic organ classification model using first training data tagged with symptomatic organ information in medical images; A process of obtaining location information of symptomatic organs predicted from the medical image and names of symptomatic organs corresponding to the location information through a trained symptomatic organ classification model; and A device comprising a process of training a detailed disease classification model using second training data tagged with detailed disease information in the medical image and the location information and the name.

Images