Method, device, and computer program for integrated prediction of disease prognosis using image and non-image data

Abstract

A method for predicting the prognosis of a target disease on the basis of a medical image and non-image data according to an embodiment of the present invention may comprise: a step for acquiring a medical image of a patient being examined for an abnormal site; an image data analysis step for classifying the target disease in the medical image into a preset group by using a first artificial intelligence model; a first non-image data generation step for acquiring medical data on the patient and digitizing the same; a second non-image data generation step for digitizing target disease-related features in the medical image; and an integrated analysis step for predicting the prognosis of the target disease by applying a value obtained as a result of the image data analysis step and the first and second non-image data to a second artificial intelligence model.

Description

Method, device, and computer program for integrated prediction of disease prognosis using image and non-image data

[0001] The present invention relates to a method, device and computer program for predicting the prognosis of a disease based on medical images and non-image data, and more particularly, to a method, device and computer program for analyzing and classifying abnormal areas included in medical images, analyzing medical data of a subject, and integrating image and non-image data to predict the prognosis of a target disease.

[0002] The current shortage of doctors in our country is becoming increasingly serious, and there is a particular problem of a lack of specialists in urgent medical settings, hindering quick responses.

[0003]

[0004] By allowing doctors to interpret medical images acquired from various medical diagnostic devices such as radiography (X-ray), ultrasound (ultrasonography), computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET), earlier diagnosis and treatment of diseases are possible compared to the past, resulting in extended human lifespans. However, despite the increase in the number of patients using various diagnostic devices, the difficulty of timely diagnosis by specialists, inaccurate human interpretation, and the possibility of misdiagnosis due to interpretation differences between doctors or even within the same doctor are emerging as problems. Furthermore, the complex and time-consuming work processes in the medical field place an excessive burden on medical staff. Moreover, while assessing a patient's critical condition and classifying treatment priorities are crucial for timely treatment and saving lives, the comprehensive analysis of complex data is necessary, and making immediate judgments on this is realistically very difficult.

[0005]

[0006] Accordingly, various attempts are emerging in medical imaging to address these issues by introducing AI-based algorithm-based diagnostic systems. Specifically, research is actively underway to enhance diagnostic accuracy by supplementing human medical imaging interpretation with AI-based diagnostic findings (using second-opinion or double-reading). Furthermore, AI can help prioritize treatment by predicting a patient's condition, severity, and prognosis.

[0007]

[0008] However, in this field of technology, attempts are being made to automatically interpret radiographs using machine learning models. However, the method using machine learning models still has limitations in accuracy, and due to the nature of machine learning, which is called a black box, there is insufficient explanation for the automatic analysis results, so it is not being used well in the field.

[0009] The present invention is intended to solve the aforementioned problems, and has as its primary purpose the ability to rapidly and accurately predict the prognosis of a target disease based on medical image and non-image data.

[0010] In addition, another purpose of the present invention is to improve the accuracy of predicting the prognosis of a target disease by integrating image data and non-image data in medical image analysis using deep neural network artificial intelligence techniques.

[0011] The present invention is to achieve the above-described object, and a method for predicting the prognosis of a target disease based on medical image and non-image data according to an embodiment of the present invention may include a step of acquiring a medical image of a patient for which an abnormality is to be detected, an image data analysis step of classifying the target disease in the medical image into a preset group using a first artificial intelligence model, a first non-image data generation step of acquiring the patient's medical data and quantifying the same, a second non-image data generation step of quantifying target disease-related features in the medical image, and an integrated analysis step of applying the result value of the image data analysis step and the first and second non-image data to a second artificial intelligence model to predict the prognosis of the target disease.

[0012] According to the present invention as described above, the prognosis of a target disease can be predicted quickly and accurately using medical image and non-image data.

[0013] In addition, according to the present invention, in medical image analysis using deep neural network artificial intelligence techniques, the classification accuracy of medical images can be improved.

[0014] Figure 1 is a schematic diagram of an electronic device according to one embodiment of the present invention.

[0015] Figures 2 to 4 are diagrams showing the operation of an electronic device according to one embodiment of the present invention.

[0016] FIG. 5 and FIG. 6 are diagrams showing an artificial intelligence model of an electronic device according to one embodiment of the present invention.

[0017] FIG. 7 and FIG. 8 are flowcharts illustrating a method for predicting the prognosis of a target disease according to one embodiment of the present invention.

[0018] The above-described objects, features, and advantages will be described in detail below with reference to the accompanying drawings, so that those skilled in the art can easily practice the technical idea of ​​the present invention. In describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, a detailed description thereof will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components, and all combinations described in the specification and claims can be combined in any manner. In addition, it should be understood that reference to the singular may include one or more, and reference to the singular expression may also include plural expression.

[0019] The present invention is applicable to a wide range of fields. While this specification focuses on methods for predicting the prognosis of cerebral hemorrhage, degenerative arthritis, and / or growth and developmental abnormalities, the present invention is not limited to the specific disease and can be utilized in various medical image analyses to assess abnormal conditions such as cardiovascular disease and cerebrovascular disease.

[0020] The artificial intelligence model or integrated analysis model according to the present invention may be a machine learning-based deep learning model. Furthermore, these models can be trained using acquired data or statistical data.

[0021] Meanwhile, the term 'abnormal area' in this specification can be understood as a concept that includes not only an area judged to be abnormal in the binary classification of normal and abnormal, but also an area judged to be in various abnormal states (suspicion, cyst, tumor, fracture, fracture nonunion, acute fracture, bleeding, suspected bleeding, etc.) that are judged to be not normal.

[0022] In addition, the term 'abnormal condition' used in this specification can be understood as a term meaning the condition of an abnormal area existing in a medical image or area of ​​interest. 'Abnormal condition' may include normal, abnormal, and more specifically, may include a specific condition of the abnormal area such as normal, suspicious, cyst, benign tumor, negative tumor, and malignant tumor. Alternatively, if the analyzed medical image is an image suspected of cerebral hemorrhage, 'abnormal condition' may include a condition such as normal, suspected cerebral hemorrhage, microbleed, hematoma enlargement, signs of increased intracranial pressure, supratentorial hemorrhage, or supratentorial hemorrhage.

[0023] Accordingly, the classification value that classifies the target disease into a preset group may also be defined differently depending on the classification method. For example, when the electronic device (1000) binary-classifies image data or non-image data (hereinafter referred to as data) into a preset group, i.e., normal or abnormal, data classified as normal according to the binary classification will have a classification value corresponding to not being an 'abnormal state', and data classified as abnormal will have a classification value corresponding to being an 'abnormal state'. If the electronic device (1000) multi-classifies image data or non-image data (hereinafter referred to as data), data classified as normal may have a classification value corresponding to not being an 'abnormal state', data classified as being a suspicious state may have a classification value where the type of 'abnormal state' corresponds to the 'suspicious' category, data classified as being a state with a cyst may have a classification value where the type of 'abnormal state' corresponds to the 'cyst' category, and data classified as being a state with a tumor may have a classification value where the type of 'abnormal state' corresponds to the 'tumor' category. In other words, the abnormal state is a concept that includes multiple types of states, and can be understood as a concept that represents various states of abnormal areas, areas of interest, and medical images, rather than simply a concept opposite to the normal state.

[0024] Furthermore, predicting the prognosis of cerebral hemorrhage, as one embodiment of the invention, is a very important technology for providing appropriate treatment by predicting the prognosis of whether the cerebral hemorrhage is likely to expand and cause hematoma enlargement, where the hematoma volume exceeds a certain level, when the cerebral hemorrhage occurs. Hematoma enlargement, which occurs in approximately 30% of spontaneous intracerebral hemorrhages, is closely related to mortality, so it is important to predict and provide appropriate treatment first.

[0025] In addition, as an embodiment of the invention, predicting the prognosis of degenerative arthritis is very important in order to provide the right treatment method depending on the site where degenerative arthritis occurs or the severity of the prognosis.

[0026] In addition, the division of the first to third artificial intelligence models in the present invention is merely for the purpose of enhancing understanding of the invention, and in reality, there may be one or more models.

[0027]

[0028] FIG. 1 is a schematic diagram of an electronic device (1000) according to one embodiment of the present invention.

[0029] An electronic device (1000) according to one embodiment of the present invention may include a transceiver (1100), a memory (1200), and a processor (1300).

[0030] The transceiver (1100) of the electronic device (1000) can communicate with any external device (or external server) including a user terminal and / or a database. For example, the electronic device (1000) can obtain an input requesting prediction of a disease prognosis from a user terminal or an external device through the transceiver (1100). For example, the electronic device (1000) can obtain medical data necessary for predicting the prognosis of a disease, including the patient's gender, age, family history, genes, existing diseases, bone age information at the time of analysis, and / or medication information, medical history information, height, weight, body temperature, lifestyle habits, and symptom information, from the user terminal through the transceiver (1100). For example, the electronic device (1000) can obtain disease prognosis statistical data classified by age, gender, country, region, and / or race from a database through the transceiver (1100). For example, the electronic device (1000) can transmit predicted disease prognosis information to any external device, including a user terminal, through the transceiver (1100).

[0031] As needed, the electronic device (1000) can acquire medical images of the patient via the transceiver (1100). The medical images of the patient may be images of various body parts and images captured by various medical devices, depending on the disease for which the prognosis is to be predicted, and may be acquired without limitation from any body part or medical device.

[0032] The electronic device (1000) can connect to a network through a transceiver (1100) and transmit and receive various types of data. The transceiver (1100) can be broadly categorized into wired and wireless types. Since the wired and wireless types each have their own advantages and disadvantages, the electronic device (1000) may be equipped with both wired and wireless types at the same time. Here, in the case of the wireless type, a communication method of the WLAN (Wireless Local Area Network) series, such as Wi-Fi, can be mainly used. Alternatively, in the case of the wireless type, a communication method of the cellular communication series, such as LTE or 5G, can be used. However, the wireless communication protocol is not limited to the examples described above, and any appropriate wireless communication method can be used. Representative examples of the wired type include LAN (Local Area Network) and USB (Universal Serial Bus) communication, and other methods are also possible.

[0033] The memory (1200) of the electronic device (1000) can store various types of information. Various types of data can be temporarily or semi-permanently stored in the memory (1200). Examples of the memory (1200) may include a hard disk drive (HDD), a solid state drive (SSD), flash memory, read-only memory (ROM), random access memory (RAM), etc. The memory (1200) may be provided in a form that is built into the electronic device (1000) or in a detachable form. The memory (1200) may store various types of data necessary for the operation of the electronic device (1000), including an operating system (OS) for operating the electronic device (1000) and a program for operating each component of the electronic device (1000).

[0034] The processor (1300) can control the overall operation of the electronic device (1000). For example, the processor (1300) can control the overall operation of the electronic device (1000), including an operation of analyzing image data, which will be described later, an operation of generating and analyzing non-image data, and / or an operation of predicting the prognosis of a disease based on input data. Specifically, the processor (1300) can load and execute a program for the overall operation of the electronic device (1000) from the memory (1200). The processor (1300) can be implemented as an AP (Application Processor), a CPU (Central Processing Unit), an MCU (Microcontroller Unit), or a similar device according to hardware, software, or a combination thereof. In this case, the hardware may be provided in the form of an electronic circuit that processes an electrical signal to perform a control function, and the software may be provided in the form of a program or code that drives a hardware circuit.

[0035] Hereinafter, with reference to FIGS. 2 to 8, the operation of an electronic device (1000) according to one embodiment of the present invention and the method for predicting the prognosis of a disease performed by the electronic device (1000) will be described in more detail.

[0036] Figures 2 to 4 are diagrams showing the operation of a processor (1300) according to one embodiment of the present invention.

[0037] Referring to FIGS. 2 to 4, the processor (1300) of the electronic device (1000) may be configured to acquire medical images and medical data of a patient from whom an abnormal area is to be detected, and to predict the prognosis of a disease using the generated integrated analysis model.

[0038] More specifically, according to one embodiment of the present invention, the processor (1300) of the electronic device (1000) can obtain input data through the transceiver (1100). Here, the input data may include a medical image and medical data of a patient. More specifically, the medical image of the patient may be obtained through an imaging device such as a camera, an X-ray device, a computed tomography device, or may be obtained through a capture function of the electronic device, and may include data such as CT, X-ray, and MRI of a specific part.

[0039] If necessary, the processor (1300) can preprocess the acquired image.

[0040] As needed, the processor (1300) of the electronic device (1000) can extract one or more regions of interest (ROI) from the target image. Here, a ROI refers to a specific region of interest that is important for analyzing the target image. In machine learning analysis, the importance of a specific region is often based on domain knowledge of the image. In other words, analyzing centered on a ROI increases accuracy by removing noise from areas other than the primary ROI, which can potentially cause errors in the analysis.

[0041] As needed, the processor (1300) of the electronic device (1000) can manually or automatically detect a region of interest, for example, extract an area corresponding to location information set by the user as a region of interest.

[0042] If necessary, the first AI model can automatically extract one or more regions of interest from the target image. In machine learning analysis, the importance of a specific region is often based on domain knowledge of the image. In other words, focusing on the region of interest increases accuracy by removing noise from areas outside the primary region of interest that could potentially cause analysis errors.

[0043] The first AI model may include an automatic detection model, a deep neural network trained and evaluated to extract multiple regions of interest from a target image. The automatic detection model may use detection techniques such as, but not limited to, deep neural network-based Convolutional Neural Networks (CNNs), Faster-RCNN, Yolo, and Transformer models.

[0044] As needed, the processor (1300) of the electronic device (1000) can manually or automatically segment the image into a region of interest and extract a target disease-related region from the segmented image. This region of interest extraction is intended to improve accuracy. Rather than analyzing the image using the entire image as input data, as is the case with conventional general image analysis methods, machine learning modeling and application are performed for each major sub-region, enabling precise analysis for each major sub-region.

[0045] If necessary, the processor (1300) of the electronic device (1000) can train an automatic detection model using multiple cerebral hemorrhage CT images, and can extract an area corresponding to cerebral hemorrhage as a region of interest by applying an arbitrary target image (brain CT) to the learned automatic detection model. The automatic detection model can detect the region of interest based on whether an abnormal area exists within the region of interest by using a machine learning model learned with normal discrimination data and abnormal discrimination data corresponding to the region of interest as learning data, segment the image of the region of interest, and extract the region related to the target disease within the segmented image. Since the presence of an abnormal area within the region of interest ultimately means the presence of an abnormal area within the target image, it is safe to view the first artificial intelligence model as judging and classifying the normal / abnormal status of the target image. As an example, if the target image is a brain image and the first artificial intelligence model is a machine learning model trained with abnormal discrimination data indicating the presence of cerebral hemorrhage and normal discrimination data indicating the absence of cerebral hemorrhage, the device can detect a region including a cerebral hemorrhage area in the input brain image as a region of interest, segment the image of the region of interest, and extract a region related to the target disease within the segmented image, or determine whether the target disease is positive or not from the input image, and classify the abnormal state of the target image or the abnormal state of the region of interest accordingly.

[0046] If necessary, the processor (1300) of the electronic device (1000) may include a first artificial intelligence model that classifies a target disease in a medical image into a preset group. The machine learning model used in the first artificial intelligence model may be a deep neural network composed of one or more layers, and may be a convolutional neural network (CNN) that includes a plurality of convolution layers that create a feature map for features of a region of interest and a pooling layer that performs subsampling between the plurality of convolution layers. The convolutional neural network may extract features from an input image by alternately performing convolution and subsampling on the input image.

[0047] A convolutional neural network contains multiple convolution layers, multiple subsampling layers (max-pooling layers, pooling layers), and may include a global average pooling layer, a fully-connected layer, a softmax layer, etc. The convolution layer is a layer that performs convolution on the input image, and the subsampling layer is a layer that extracts the local maximum or average value on the input image and maps it to a 2D image, so that the local area can be enlarged and subsampled. The convolution layer requires information such as the kernel size, the number of kernels to use (the number of maps to be created), and the weight table to be applied during the convolution operation. The subsampling layer requires information such as the size of the kernel to subsample, and whether to select the maximum or minimum value among the values ​​within the kernel area. In one embodiment, the convolutional neural network may include a unified layer (layer 2) in which subsampling is performed between convolutional layers (layer 1, layer 3), and may include a gap-ap layer, a fully connected layer, and a softmax layer at the terminal. This is for convenience of explanation, and the convolutional neural network may be configured by combining layers with characteristics different from those applied in this example in a more diverse manner, and the present invention is not limited by the configuration and structure of a deep neural network.The value output from the first artificial neural model is a value produced by applying the region of interest to the deep neural network machine learning model, and means a classification value according to at least one of the preset groups. As described above, the classification value may be a value corresponding to normal / abnormal according to binary classification, or a value corresponding to a class according to user settings, such as normal / suspicious / cyst / tumor (negative, benign, malignant) or normal / suspicious fracture / fracture nonunion / acute fracture / fracture according to multi-classification.

[0048] If necessary, the processor (1300) of the electronic device (1000) may be a model trained to classify a target disease in a medical image into a preset group using the first artificial intelligence model. The preset group includes one or more groups from among a group including a disease type group, a disease severity grade group, a group determining whether the disease is benign or not, a disease prognosis prediction group, and a cerebral hematoma enlargement prediction group. That is, in the case of a brain image, it includes a group classified as a disease such as stroke or cerebral hemorrhage, a group classified into multiple disease severity grades, a group divided into binary classifications of whether the target disease is benign or negative, and a disease prognosis group, and the electronic device can classify a patient's medical image into one or more of these groups using the first artificial intelligence model.

[0049] According to a specific embodiment of the present invention, the electronic device can apply a brain CT image to a first artificial intelligence model to quantify and output the possibility that the volume of cerebral hemorrhage will increase by a certain amount or more.

[0050] The electronic device can train the first artificial intelligence model based on the image of the extracted target disease-related area divided into normal discrimination data and abnormal discrimination data.

[0051] If necessary, the processor (1300) of the electronic device (1000) may digitize the patient's medical data to generate first non-image data. The medical data may include one or more data from the group including the patient's age, gender, family history, genes, underlying diseases, existing diseases, medications, height, weight, body temperature, lifestyle habits, and symptoms, and may also include any information that may affect the prognosis of the disease.

[0052] Methods for quantifying medical data can be achieved through various methods, including a method where the user assigns weights to factors affecting the prognosis of a disease, a method set by the user, and the Glasgow Coma score (GCS), and are not limited to those described.

[0053] If necessary, the processor (1300) of the electronic device (1000) may generate second non-image data that quantifies target disease-related features within a medical image. The target disease-related features may include all features that may affect the prognosis of the target disease among the patient information included within the image, and the second non-image data refers to data that quantifies the location, volume, shape, etc. of the disease-related features included within the image.

[0054] If necessary, the processor (1300) of the electronic device (1000) may generate second non-image data that quantifies target disease-related features using the intermediate output value of the first artificial intelligence model. More specifically, the intermediate output value may be a value obtained by extracting a target disease-related region from an image of a region of interest detected by the first artificial intelligence model or by segmenting the image of the region of interest. By quantifying disease-related features using the intermediate output value of the first artificial intelligence model, the processor can generate data with higher accuracy.

[0055] In one embodiment, when an image related to cerebral hemorrhage is acquired, the processor (1300) of the electronic device (1000) can generate second non-image data by digitizing the thickness of a blood vessel, the location coordinates of the site of cerebral hemorrhage, the 3D coordinates of the center of cerebral hemorrhage, the relative location in the brain region, the volume and area of ​​the site of cerebral hemorrhage, the area and location of the site of the largest cerebral hemorrhage, the number and volume of each type of cerebral hemorrhage, the image pattern of the cerebral hemorrhage area, texture characteristics, the shape and condition of cerebral hemorrhage, etc.

[0056] According to another embodiment, when acquiring an image related to degenerative arthritis, the processor (1300) of the electronic device (1000) can generate second non-image data by digitizing joint spacing, sclerosis location, sclerosis degree, osteophyte location, osteophyte degree, bone number, bone spacing, bone thickness, etc. of the target area in the image.

[0057] If necessary, the processor (1300) of the electronic device (1000) according to one embodiment of the present invention can classify the first and second non-image data into preset groups by applying the third artificial intelligence model. The third artificial intelligence model can analyze the first and second non-image data using a machine learning technique such as an SVM, regression, or a fully connected neural network model, and perform the intended double classification, multi-classification, etc. In this case, the preset groups are the same as those described above, and thus a detailed description thereof is omitted.

[0058] The processor (1300) of the electronic device (1000) according to one embodiment of the present invention can generate a second artificial intelligence model for comprehensively predicting the prognosis of a disease based on medical image and non-image data. The electronic device (1000) according to one embodiment of the present invention can obtain prognostic statistical data of a target disease through the transceiver (1100). The prognostic statistical data can be grouped by location, volume, age, gender, country, region, and / or race of the target disease. The electronic device (1000) according to one embodiment of the present invention can learn a second artificial intelligence model capable of predicting the prognosis of the target disease based on input data and the prognostic statistical data of the disease.

[0059] As needed, the processor (1300) of the electronic device (1000) can use the second artificial intelligence model to predict the prognosis of a target disease from the patient's medical images and medical data. More specifically, the electronic device (1000) can predict the prognosis of a target disease using weights (or parameters) derived from the artificial intelligence model as described above.

[0060] According to one embodiment of the present invention, the second artificial intelligence model can output a prognostic prediction value of a target disease using the output value of the first artificial intelligence model and the first and second non-image data as input data.

[0061]

[0062] If necessary, the second artificial intelligence model can output a prognostic prediction value for the target disease using the output value of the first artificial intelligence model and the output value of the third artificial intelligence model as input data.

[0063]

[0064] If necessary, an AI model according to one embodiment of the present invention can use the scores or normalized probability values ​​of each class as classification values ​​for the final class classification of a deep neural network. For example, the class scores of each class from a softmax layer or the normalized probability values ​​resulting from the softmax layer can be calculated as classification values ​​for the corresponding region of interest and input into the AI ​​model.

[0065]

[0066] Accordingly, S C is the score of class C, and w c i is A i is the weight of .

[0067]

[0068]

[0069] is each feature value in the k channel is the average value of the fields, and in this case, the feature value of the region of interest h is F h = It can be composed of [S1, S2, S3,…, SM]. M is the total number of classes.

[0070]

[0071] According to one embodiment of the present invention, the processor (1300) of the electronic device (1000) may generate an integrated prediction model for predicting the prognosis of a target disease. Furthermore, the processor (1300) of the electronic device (1000) may be configured to perform an operation for predicting the prognosis of the target disease from input data including a patient's medical image and / or medical data using the generated integrated prediction model. In addition, the processor (1300) of the electronic device (1000) may be configured to perform an operation for predicting the prognosis of the target disease by applying medical images and non-image data processed from the patient's medical images and medical data to the generated integrated prediction model.

[0072]

[0073] According to a specific embodiment of the present invention, an electronic device (1000) can acquire brain CT medical images and medical data of a patient suspected of having a cerebrovascular disease. The electronic device (1000) inputs the patient's medical images into a first artificial intelligence model, classifies the type of cerebrovascular disease the patient has, and, if it determines that there is a high possibility of cerebral hemorrhage or hematoma hypertrophy, classifies the patient into the corresponding group.

[0074] In addition, the electronic device (1000) can digitize the coordinates (x, y, z) of the center of cerebral hemorrhage through the patient's medical image, digitize the volume of the cerebral hemorrhage area (e.g., mL), and store it as second non-image data. Since the risk of cerebral hemorrhage may vary depending on the coordinates of the center, volume, etc., by digitizing the corresponding data and using it for analysis, the prognosis of the target disease can be predicted more accurately. The corresponding data can be generated in the form of a score, and can be stored in the form of a vector and input into the second artificial intelligence model.

[0075] Additionally, the electronic device (1000) can generate first non-image data by digitizing medical data. In one embodiment, according to the Glasgow coma scale, the severity of the patient's condition can be determined by scoring the patient's eye response from 1 to 4, the verbal response from 1 to 5, and the motor response from 1 to 6.

[0076] In another embodiment, the patient's medical data can be scored according to the target disease by assigning a risk score based on the patient's age, gender, and, if there is an underlying disease, a risk score based on the underlying disease. That is, in the case of cerebral hemorrhage, if the patient is 60 years of age or older, the risk score can be assigned as 5, if in the 50s, 4, and if in the 40s, 3. If the patient has high blood pressure, the risk score can be assigned as 5, and if the patient already has a history of cerebral hemorrhage, the risk score can be assigned as 20. In this manner, the patient's medical data can be quantified to generate the first non-image data.

[0077] The electronic device can input the first and second non-image data into a third artificial intelligence model and classify the patient into preset groups. Based on the patient's first and second non-image data, the electronic device can determine that the patient is unlikely to develop a cerebral hemorrhage or hematoma.

[0078] The second artificial intelligence model of the electronic device can predict the prognosis of a disease using image data and first and second non-image data as input data.

[0079] If necessary, the second AI model can use the outputs of the first and third AI models as input data to predict disease prognosis. If the disease groups classified by the first and third AI models differ, the second AI model can be used to additionally determine prognosis, further enhancing the predictability of prognosis.

[0080]

[0081] Hereinafter, a method for predicting the prognosis of a target disease based on medical image and non-image data corresponding to another embodiment of the present invention is described.

[0082]

[0083] FIG. 7 and FIG. 8 are flowcharts of a method for predicting the prognosis of a target disease according to one embodiment of the present invention.

[0084] A method for predicting prognosis according to one embodiment of the present invention relates to a method for predicting the prognosis of a disease in an integrated manner by acquiring image and non-image data of a patient and analyzing them individually or in combination, and thereby to a method for predicting the prognosis of a target disease more accurately.

[0085]

[0086] A prognosis prediction method according to a first embodiment of the present invention may include a step (S1100) of acquiring a medical image of a patient from which an abnormal area is to be detected, an image data analysis step (S1200) of classifying a target disease in the medical image into a preset group using a first artificial intelligence model, a first non-image data generation step (S1300) of acquiring the patient's medical data and digitizing the same, a second non-image data generation step (S1400) of digitizing features related to the target disease in the medical image, and an integrated analysis step (S1500) of applying the result value of the image data analysis step and the first and second non-image data to a second artificial intelligence model to predict the prognosis of the target disease.

[0087]

[0088] A prognosis prediction method according to a second embodiment of the present invention may include a step (S2100) of acquiring a medical image of a patient from which an abnormal area is to be detected, an image data analysis step (S2200) of classifying a target disease in the medical image into a preset group using a first artificial intelligence model, a first non-image data generation step (S2300) of acquiring the patient's medical data and quantifying it, a second non-image data generation step (S2400) of quantifying features related to the target disease in the medical image, a non-image data analysis step (S2500) of applying the first and second non-image data to a third artificial intelligence model to classify them into a preset group, and an integrated analysis step (S2600) of applying the result values ​​of the image data analysis step and the result values ​​of the non-image data analysis step to the second artificial intelligence model to comprehensively predict the prognosis of the target disease.

[0089] If necessary, in the method for predicting the prognosis of a target disease according to the first and second embodiments of the present invention, the image data analysis step may include a step of detecting a region of interest including an abnormal area in an acquired medical image using a first artificial intelligence model, a step of segmenting the image of the region of interest (segmentation), and a step of extracting a region related to the target disease within the segmented image. If necessary, the first artificial intelligence model may extract the region related to the target disease and then classify the target disease within the extracted image into a preset group.

[0090] If necessary, the first non-image data generation step may include a step of scoring medical data. The medical data may include one or more data from the following groups: the patient's age, gender, family history, genes, underlying conditions, pre-existing conditions, medications, body temperature, height, weight, body temperature, lifestyle habits, and symptoms.

[0091]

[0092] If necessary, the second non-image data generation step may include a step of quantifying the characteristics of the target disease-related region. If necessary, the electronic device may generate second non-image data by quantifying the characteristics of the target disease-related region using data extracted from the segmented image.

[0093]

[0094] If necessary, the preset group in the method according to the first and second embodiments of the present invention may include one or more groups from among a group including a group of disease type, a group of disease severity grade, a group of whether the disease is benign or not, and a group of disease prognosis, and the target disease-related features may include one or more from among a group including a location, size, volume of a target disease-related area, and a physical feature related to the target disease.

[0095]

[0096] If necessary, the integrated analysis step in the method for predicting the prognosis of a target disease according to the first and second embodiments of the present invention may include a step for predicting the prognosis of degenerative arthritis or growth and development of the body.

[0097]

[0098] If necessary, the integrated analysis step in the method for predicting the prognosis of a target disease according to the first and second embodiments of the present invention may include a step for predicting the prognosis of cerebral hemorrhage hematoma hypertrophy.

[0099]

[0100] Another embodiment of the present invention may be a computer-readable recording medium having recorded thereon a program for executing a method for predicting the prognosis of a target disease of the present invention.

[0101]

[0102] The features, structures, effects, etc. described in the embodiments above are included in at least one embodiment of the present invention, and are not necessarily limited to just one embodiment. Furthermore, the features, structures, effects, etc. exemplified in each embodiment can be combined or modified in other embodiments by a person skilled in the art to which the embodiments pertain. Therefore, the contents related to such combinations and modifications should be interpreted as being included in the scope of the present invention. In addition, although the embodiments have been described above, these are merely examples and do not limit the present invention. Those skilled in the art to which the present invention pertains will appreciate that various modifications and applications not exemplified above are possible without departing from the essential characteristics of the embodiments. In other words, each component specifically shown in the embodiments can be modified and implemented. In addition, differences related to such modifications and applications should be interpreted as being included in the scope of the present invention defined in the appended claims.

Claims

1. A method for predicting the prognosis of a target disease based on medical image and non-image data, A step of acquiring a medical image of a patient from which an abnormal area is to be detected; An image data analysis step for classifying the target disease in the medical image into a preset group using a first artificial intelligence model; A first non-image data generation step of acquiring the patient's medical data and digitizing it; A second non-image data generation step for quantifying target disease-related features within the above medical image; An integrated analysis step for predicting the prognosis of the target disease by applying the result value of the image data analysis step and the first and second non-image data to a second artificial intelligence model; A method for predicting the prognosis of a target disease.

2. A method for predicting the prognosis of a target disease based on medical image and non-image data, A step of acquiring a medical image of a patient from which an abnormal area is to be detected; An image data analysis step for classifying the target disease in the medical image into a preset group using a first artificial intelligence model; A first non-image data generation step of acquiring the patient's medical data and digitizing it; A second non-image data generation step for quantifying target disease-related features within the above medical image; A non-image data analysis step of applying the first and second non-image data to a third artificial intelligence model to classify them into the preset groups; An integrated analysis step for comprehensively predicting the prognosis of the target disease by applying the result value of the image data analysis step and the result value of the non-image data analysis step to a second artificial intelligence model; A method for predicting the prognosis of a target disease.

3. In paragraph 1 or 2, The above video data analysis step is: A step of detecting a region of interest including the abnormal area in the medical image; A step of segmenting the image of the above region of interest (segmentation); A step of extracting the target disease-related region within the segmented image; comprising; A method for predicting the prognosis of a target disease.

4. In paragraph 3, The above second non-image data generation step is, A step of quantifying the characteristics of the target disease-related area; including; A method for predicting the prognosis of a target disease.

5. In paragraph 1 or 2, The above preset groups are: Contains one or more groups of groups including a disease type group, a disease severity grade group, a disease benignity group, and a disease prognosis group; The above target disease-related characteristics are: characterized in that it comprises at least one from a group including the location, size, volume of the target disease-related area and physical characteristics related to the target disease. A method for predicting the prognosis of a target disease.

6. In paragraph 1 or 2, The above medical data is, Contains one or more data from the group including the patient's age, sex, family history, genes, underlying disease, existing disease, medication, body temperature, height, weight, body temperature, lifestyle habits, and symptoms. The above first non-image data generation step is, A step of scoring the above medical data; including; A method for predicting the prognosis of a target disease.

7. In paragraph 1 or 2, The above integrated analysis step is, A step for predicting the prognosis of degenerative arthritis or growth and development of the body; A method for predicting the prognosis of a target disease.

8. In paragraph 1 or 2, The above integrated analysis step is, A step for predicting the prognosis of cerebral hemorrhage hematoma hypertrophy; including; A method for predicting the prognosis of a target disease.

9. A computer-readable recording medium having recorded thereon a program for executing the method according to paragraph 1 or 2 on a computer.

10. In an electronic device for predicting the prognosis of a target disease based on medical image and non-image data, A processor configured to obtain a medical image of a patient from which an abnormal area is to be detected, classify a target disease in the medical image into a preset group using a first artificial intelligence model, obtain medical data of the patient, digitize the same to generate first non-image data, digitize target disease-related features in the medical image to generate second non-image data, and apply the output value of the first artificial intelligence model and the first and second non-image data to the second artificial intelligence model to comprehensively predict the prognosis of the target disease. Electronic devices.

11. In an electronic device for predicting the prognosis of a target disease based on medical image and non-image data, A processor configured to obtain a medical image of a patient from which an abnormal area is to be detected, classify a target disease in the medical image into a preset group using a first artificial intelligence model, obtain medical data of the patient, digitize the same to generate first non-image data, digitize target disease-related features in the medical image to generate second non-image data, apply the first and second non-image data to a third artificial intelligence model to classify the target disease into a preset group, and apply output values ​​of the first and third artificial intelligence models to a second artificial intelligence model to comprehensively predict the prognosis of the target disease. Electronic devices.

Images