Integrated disease prognosis prediction method, apparatus, and computer program using video and non-video data

Abstract

A method for predicting the prognosis of a target disease based on medical images and non-image data according to one embodiment of the present invention may include the steps of: acquiring medical images of a patient in which abnormal areas are to be detected; an image data analysis step of classifying the target disease in the medical images into pre-set groups using a first artificial intelligence model; a first non-image data generation step of acquiring and quantifying the patient's medical data; a second non-image data generation step of quantifying target disease-related features in the medical images; and an integrated analysis step of applying the result values ​​from the image data analysis step, the first and second non-image data to a second artificial intelligence model to predict the prognosis of the target disease.

Description

Technical Field

[0001] The present invention relates to a method, apparatus, and computer program for predicting the prognosis of a disease based on medical images and non-image data. More specifically, it relates to an analysis method, apparatus, and computer program for analyzing and classifying abnormal sites included in medical images, analyzing the medical data of a subject, integrating image and non-image data, and predicting the prognosis of the target disease.

Background Art

[0002] Currently, the problem of doctor shortage in Korea is becoming increasingly serious. Especially in emergency medical situations, there is a shortage of specialists, resulting in the problem that prompt measures cannot be taken.

[0003] By doctors interpreting medical images obtained from various medical diagnostic devices such as radiography (Radiography or X-ray), ultrasonography, computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET), early diagnosis and treatment of diseases have become possible compared to the past, and as a result, the extension of human lifespan has been realized. However, compared to the increasing number of patients using various diagnostic devices, the difficulty of timely diagnosis by detailed specialists, the inaccurate interpretation by humans, and the possibility of misdiagnosis due to interpretation deviation between doctors or within the same doctor have also emerged as problems. In addition, due to the complex and time-consuming business processes in the medical field, excessive work is imposed on medical staff. Furthermore, the judgment of the severity of patients and the classification of treatment priorities are very important for the timely treatment and survival of patients, but comprehensive analysis of complex data is required, and immediate judgment on this is very difficult in reality.

[0004] Therefore, various attempts are being made to compensate for these problems in medical imaging by introducing AI algorithm-based diagnostic systems. Specifically, research is actively underway to improve accuracy by supplementing the interpretation of medical images by physicians with diagnostic findings using AI (utilizing it as a second opinion or double reading). In addition, AI can help classify treatment priorities by predicting the patient's condition, severity, and prognosis.

[0005] However, while attempts have been made in this field to automatically interpret radiographs using machine learning models, methods using machine learning models still have limitations in accuracy. Due to the nature of machine learning, which is often referred to as a "black box," there is a lack of explanation for the automated analysis results, and it has not been effectively utilized in the field. [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] The present invention aims to solve the aforementioned problems and, in part, to enable rapid and accurate prediction of the prognosis of a target disease based on medical imaging and non-imaging data.

[0007] Furthermore, the present invention also aims to improve the accuracy of predicting the prognosis of a target disease by integrating video data and non-video data in medical image analysis utilizing deep neural network artificial intelligence techniques. [Means for solving the problem]

[0008] The present invention aims to achieve the aforementioned objectives, and a method for predicting the prognosis of a target disease based on medical images and non-image data according to one embodiment of the present invention may include the steps of: acquiring medical images of a patient in which abnormal areas are to be detected; an image data analysis step of classifying the target disease in the medical images into pre-set groups using a first artificial intelligence model; a first non-image data generation step of acquiring and quantifying the patient's medical data; a second non-image data generation step of quantifying target disease-related features in the medical images; and an integrated analysis step of applying the result values ​​from the image data analysis step, the first and second non-image data to a second artificial intelligence model to predict the prognosis of the target disease. [Effects of the Invention]

[0009] According to the present invention as described above, it is possible to predict the prognosis of a target disease quickly and accurately using medical images and non-image data.

[0010] Furthermore, according to the present invention, it is possible to improve the classification accuracy of medical images in medical image analysis utilizing deep neural network artificial intelligence techniques. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of an electronic device according to one embodiment of the present invention.

[0012] [Figure 2] This is a diagram illustrating the operation of an electronic device according to one embodiment of the present invention. [Figure 3] This is a diagram illustrating the operation of an electronic device according to one embodiment of the present invention. [Figure 4] This is a diagram illustrating the operation of an electronic device according to one embodiment of the present invention.

[0013] [Figure 5] This is a drawing showing an artificial intelligence model of an electronic device according to one embodiment of the present invention. [Figure 6] This is a drawing showing an artificial intelligence model of an electronic device according to one embodiment of the present invention.

[0014] [Figure 7] This is a flowchart illustrating a method for predicting the prognosis of a target disease according to one embodiment of the present invention. [Figure 8] This is a flowchart illustrating a method for predicting the prognosis of a target disease according to one embodiment of the present invention. [Modes for carrying out the invention]

[0015] The aforementioned objectives, features, and advantages will be described in detail below with reference to the attached drawings, and accordingly, a person with ordinary skill in the art to which the present invention pertains will be able to easily implement the technical idea of ​​the present invention. In describing the present invention, detailed explanations of prior art related to the present invention will be omitted if it is judged that such explanations may unnecessarily obscure the gist of the present invention. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. In the drawings, the same reference numerals are used to indicate identical or similar components, and all combinations described in the specification and claims can be combined in any manner. Furthermore, unless otherwise specified, a singular reference may include one or more expressions, and a singular expression may include multiple expressions.

[0016] The present invention is applicable to a wide range of fields. While this specification will primarily describe methods for predicting the prognosis of cerebral hemorrhage, degenerative arthritis, and / or growth and developmental abnormalities, the present invention is not limited to the type of abnormal disease and can be used in various medical image analyses for determining abnormal conditions such as cardiovascular diseases and cerebrovascular diseases.

[0017] The artificial intelligence model or integrated analysis model according to the present invention may be a deep learning model based on machine learning. Furthermore, the model may be trained using acquired data or statistical data.

[0018] On the one hand, the term "abnormal site" in this specification can be understood as a concept that includes not only sites determined to be abnormal in the binary classification of normal and abnormal, but also sites determined to be in various abnormal states (suspicion, blister, tumor, fracture, nonunion fracture, acute fracture, bleeding, suspicion of bleeding, etc.) that are not normal.

[0019] Also, the term "abnormal state" used in this specification can be understood as a term meaning the state of an abnormal site existing in a medical image or the region of interest. The "abnormal state" can include normal and abnormal, and more specifically, it may include specific states of the abnormal site such as normal, suspicion, blister, benign tumor, negative tumor, malignant tumor. Or when the analysis medical image is an image suspected of cerebral hemorrhage, the "abnormal state" can include states such as normal, suspicion of cerebral hemorrhage, microbleeding, hematoma hypertrophy, signs of increased intracranial pressure, tentorial hemorrhage recognition, tentorial hemorrhage, etc.

[0020] Therefore, the classification values for classifying a target disease into a preset group can also be defined differently depending on the classification method. For example, when the electronic device 1000 binary-classifies video data or non-video data (hereinafter, data) into a preset group, that is, normal or abnormal, the data classified as a normal state by the binary classification corresponds to the classification value for those that are not in an "abnormal state", and the data classified as an abnormal state will have the classification value corresponding to the "abnormal state". If the electronic device 1000 multi-classifies video data or non-video data (hereinafter, data), the data classified as a normal state corresponds to the classification value for those that are not in an "abnormal state", the data classified as a suspicious state corresponds to the classification value for the category of "suspicion" in the "abnormal state" type, the data classified as a state with a blister corresponds to the classification value for the category of "blister" in the "abnormal state" type, and the data classified as a state with a tumor can have the classification value corresponding to the category of "tumor" in the "abnormal state" type. That is, the abnormal state is a concept that includes multiple types of states, and can be understood as a concept indicating various states of abnormal sites, regions of interest, and medical images, rather than simply a concept opposite to the normal state.

[0021] Also, as an embodiment of the relevant invention, predicting the prognosis of cerebral hemorrhage is a very important technology for providing treatment adapted thereto by predicting the prognosis of whether, when cerebral hemorrhage occurs, this cerebral hemorrhage is likely to expand and result in an increase in the hematoma volume that becomes larger than a certain level. Since the increase in hematoma, which occurs in about 30% of spontaneous intracerebral hemorrhages, is closely related to mortality, it is important to perform treatment adapted preferentially through prediction.

[0022] Also, as an embodiment of the relevant invention, predicting the prognosis of degenerative arthritis is very important for providing the correct treatment method according to the site where degenerative arthritis occurs or according to the severity of the prognosis.

[0023] Also, the reason for dividing the first to third artificial intelligence models in the present invention is just for the purpose of enhancing the understanding of the invention, and actually it can be one or a plurality of models.

[0024] FIG. 1 is a schematic diagram of an electronic device 1000 according to an embodiment of the present invention.

[0025] An electronic device 1000 according to an embodiment of the present invention may include a transceiver 1100, a memory 1200, and a processor 1300.

[0026] The transmitting / receiving unit 1100 of the electronic device 1000 can communicate with any external device (or external server) including a user terminal and / or a database. For example, the electronic device 1000 can obtain input requesting a disease prognosis prediction from a user terminal or external device through the transmitting / receiving unit 1100. For example, the electronic device 1000 can obtain medical data necessary to predict the disease prognosis from the user terminal through the transmitting / receiving unit 1100, including the patient's sex, age, family history, genes, existing diseases, bone age information at the time of analysis, and / or medication information, medical history information, height, weight, body temperature, lifestyle, and symptom information. For example, the electronic device 1000 can obtain disease prognosis statistics classified by age, sex, country, region, and / or race from a database through the transmitting / receiving unit 1100. For example, the electronic device 1000 can transmit predicted disease prognosis information to any external device including a user terminal through the transmitting / receiving unit 1100.

[0027] If necessary, the electronic device 1000 can acquire medical images of the patient through the transmitting / receiving unit 1100. The medical images of the patient may be images of various body parts and images taken from various medical devices, depending on the disease for which the prognosis is to be predicted, and can be acquired without limitation on body parts or medical devices.

[0028] The electronic device 1000 can connect to a network via the transmitting / receiving unit 1100 to send and receive various types of data. The transmitting / receiving unit 1100 can broadly include wired and wireless types. Since wired and wireless types each have their own advantages and disadvantages, the electronic device 1000 may be equipped with both wired and wireless types simultaneously depending on the circumstances. In the case of the wireless type, communication methods of the WLAN (Wireless Local Area Network) series, such as Wi-Fi, can be mainly used. Alternatively, cellular communication methods such as LTE and 5G series can be used in the case of the wireless type. However, the wireless communication protocol is not limited to the examples given above, and any appropriate wireless communication method can be used. In the case of the wired type, LAN (Local Area Network) and USB (Universal Serial Bus) communication are typical examples, but other methods are also possible.

[0029] The memory 1200 of the electronic device 1000 can store various types of information. Various types of data can be stored in the memory 1200 temporarily or semi-permanently. Examples of memory 1200 include hard disk drives (HDDs), solid state drives (SSDs), flash memory, read-only memory (ROMs), and random access memory (RAMs). The memory 1200 can be provided in a form that is built into the electronic device 1000 or in a removable form. The memory 1200 can store various types of data necessary for the operation of the electronic device 1000, including operating systems (OS) for driving the electronic device 1000 and programs for operating each component of the electronic device 1000.

[0030] The processor 1300 can control the overall operation of the electronic device 1000. For example, the processor 1300 can control the overall operation of the electronic device 1000, including operations to analyze video data, generate and analyze non-video data, and / or predict the prognosis of a disease based on input data, as described later. Specifically, the processor 1300 can load and execute a program for the overall operation of the electronic device 1000 from the memory 1200. The processor 1300 can be embodied in hardware, software, or a combination thereof as an AP (Application Processor), CPU (Central Processing Unit), MCU (Microcontroller Unit), or similar device. In this case, hardware-wise it may be provided in the form of an electronic circuit that processes electrical signals to perform control functions, and software-wise it may be provided in the form of a program or code that drives the hardware circuit.

[0031] In the following, with reference to Figures 2 to 8, we will describe in more detail the operation of the electronic device 1000 according to one embodiment of the present invention and the method for predicting the prognosis of a disease performed by the electronic device 1000.

[0032] Figures 2 to 4 are diagrams illustrating the operation of a processor 1300 according to one embodiment of the present invention.

[0033] Referring to Figures 2 to 4, the processor 1300 of the electronic device 1000 may be configured to acquire medical images and medical data of a patient in which abnormalities are to be detected, and to predict the prognosis of the disease using the generated integrated analysis model.

[0034] More specifically, in one embodiment of the present invention, the processor 1300 of the electronic device 1000 can acquire input data through the transmitting / receiving unit 1100. Here, the input data may include medical images and medical data of a patient. More specifically, the medical images of a patient may be acquired through an imaging device such as a camera, radiography device, or computed tomography device, or acquired through the capture function of the electronic device, and may include data such as CT, X-ray, and MRI of a specific area.

[0035] If necessary, the processor 1300 can preprocess the acquired video.

[0036] If necessary, the processor 1300 of the electronic device 1000 can extract one or more regions of interest from the target video. Here, a region of interest refers to a specific area that has importance in analyzing the target video. In machine learning analysis, the importance of a specific area is often based on domain knowledge to which the video belongs. In other words, the purpose of focusing the analysis on regions of interest is to improve accuracy by removing noise in areas other than the main region of interest that may cause errors in the analysis.

[0037] If necessary, the processor 1300 of the electronic device 1000 can detect regions of interest manually or automatically, for example, by extracting a region corresponding to location information set by the user as a region of interest.

[0038] If necessary, the first artificial intelligence model can automatically extract one or more regions of interest from the target video. In machine learning analysis, the importance of a particular region is often based on domain knowledge to which the video belongs. In other words, focusing the analysis on regions of interest improves accuracy by removing noise from areas other than the main region of interest, which could cause errors in the analysis.

[0039] The first artificial intelligence model may include an automated detection model, which is a deep neural network learned (trained and evaluated) to extract multiple regions of interest from a target image. This automated detection model may use, but is not limited to, detection techniques based on deep neural networks such as CNNs (Convolutional Neural Networks), Faster-RCNNs, YOLO, and Transformer-based models.

[0040] If necessary, the processor 1300 of the electronic device 1000 can manually or automatically divide the image of the region of interest and extract the disease-related region within the divided image. This extraction of the region of interest is intended to improve accuracy, and rather than analyzing the image using the entire image as input data as in existing general video analysis methods, machine learning modeling and application are performed for each major detail region, enabling precise analysis for each major detail region.

[0041] If necessary, the processor 1300 of the electronic device 1000 can train an automatic detection model using multiple cerebral hemorrhage CT images, and apply any target image (brain CT) to the trained automatic detection model to extract a region corresponding to cerebral hemorrhage as a region of interest. The automatic detection model uses a machine learning model trained with normal discrimination data and abnormal discrimination data corresponding to the region of interest as training data to detect the region of interest based on whether an abnormal area exists within it, divide the image of the region of interest, and extract the region related to the target disease within the divided image. The presence of an abnormal area within the region of interest ultimately means that an abnormal area exists in the target image, so it can be seen as the first artificial intelligence model judging and classifying the normal / abnormal state of the target image. As an example, if the target image is a brain image and the first artificial intelligence model is a machine learning model trained on abnormality detection data where a brain hemorrhage is present and normality detection data where a brain hemorrhage is absent, the device can detect the region containing the brain hemorrhage site in the input brain image as a region of interest, divide the image of the region of interest, extract the region related to the target disease within the divided image, or determine whether the target disease is benign from the input image, and accordingly classify the abnormal state of the target image or the abnormal state of the region of interest.

[0042] If necessary, the processor 1300 of the electronic device 1000 may include a first artificial intelligence model that classifies target diseases in medical images into predefined groups. The machine learning model used in the first artificial intelligence model is a deep neural network consisting of one or more layers, and may be a convolutional neural network (CNN) that includes a number of convolutional layers that create a feature map for features of the region of interest and a pooling layer that performs subsampling between the number of convolutional layers. The convolutional neural network can extract features from the input image by alternately performing convolution and subsampling on the input image.

[0043] A convolutional neural network includes numerous convolutional layers and subsampling layers (max-pooling layers, pooling layers), and can also include GAP layers (Global Average Pooling layers), fully-connected layers, and softmax layers. A convolutional layer performs convolution on the input image, while a subsampling layer extracts the maximum or average value regionally from the input image and maps it to a 2D image, allowing for further expansion of the local area for subsampling. Convolutional layers require information such as the size of the kernel, the number of kernels used (number of maps generated), and the weighting table applied during the convolution operation. Subsampling layers require information such as the size of the kernel to be subsampled and whether to select the maximum or minimum value from the values ​​within the kernel region. According to one embodiment, a convolutional neural network includes a pooling layer (layer 2) where subsampling is performed between convolutional layers (layer 1, layer 3), and may include a GAP hierarchy, a fully connected hierarchy, or a softmax hierarchy at its terminals. This is for illustrative purposes only, and a convolutional neural network can be constructed by combining a wider variety of hierarchies with different characteristics than those applied to this example; the present invention is not limited by the configuration and structure of the deep neural network. The values ​​output by the first artificial neural model are values ​​calculated by applying the region of interest to the deep neural network machine learning model, and represent classification values ​​according to at least one of the pre-set groups. As mentioned above, the classification values ​​may correspond to normal / abnormal in binary classification, and may correspond to user-defined classes such as normal / suspected / blister / tumor (negative, benign, malignant) or normal / suspected fracture / fracture failure / acute fracture / fracture in multiple classification.

[0044] If necessary, the processor 1300 of the electronic device 1000 may be a model trained to classify target diseases in medical images into pre-defined groups using a first artificial intelligence model. The pre-defined groups include one or more groups such as disease type groups, disease severity grade groups, disease benign / negative groups, disease prognosis prediction groups, and hematoma hypertrophy prediction groups for cerebral hemorrhage. In other words, in the case of brain images, the groups include those classified as diseases such as stroke and cerebral hemorrhage, those classified into multiple disease severity grades, those divided into binary classifications of whether the target disease is benign or negative, and disease prognosis groups. The electronic device can classify the patient's medical images into one or more of these groups using the first artificial intelligence model.

[0045] According to one specific embodiment of the present invention, the electronic device can apply brain CT images to a first artificial intelligence model and output a numerical value indicating the possibility that the volume of a brain hemorrhage will increase beyond a certain size.

[0046] The electronic device can separate the extracted video data related to the target disease into normal and abnormal data, allowing it to train a first artificial intelligence model.

[0047] If necessary, the processor 1300 of the electronic device 1000 can digitize the patient's medical data to generate first non-visual data. The medical data may be one or more data points in groups including the patient's age, sex, family history, genes, underlying diseases, pre-existing diseases, medications taken, height, weight, body temperature, lifestyle, and symptoms exhibited, and may also include any other information that may affect the prognosis of the disease.

[0048] The methods for quantifying medical data can be diverse and not limited to those described, including methods where users assign weighted values ​​to factors influencing disease prognosis, methods set by the user, and the GCS (Glasgow Coma score).

[0049] If necessary, the processor 1300 of the electronic device 1000 can generate second non-image data that quantifies disease-related features in the medical image. Disease-related features can include all features of patient information contained in the image that may affect the prognosis of the disease, and the second non-image data refers to data that quantifies the location, volume, shape, etc., of the disease-related features contained in the image.

[0050] If necessary, the processor 1300 of the electronic device 1000 can generate second non-video data that quantifies disease-related features using the intermediate output values ​​of the first artificial intelligence model. More specifically, the intermediate output values ​​may be the video of the region of interest detected through the first artificial intelligence model, or the values ​​obtained by extracting the disease-related region after segmenting the video of the region of interest. By using the intermediate output values ​​of the first artificial intelligence model to quantify disease-related features, the processor can generate more accurate data.

[0051] In one embodiment, when images related to cerebral hemorrhage are acquired, the processor 1300 of the electronic device 1000 can generate second non-image data by quantifying the thickness of the blood vessels, the position coordinates of the site of cerebral hemorrhage, the 3D coordinates of the center point of the cerebral hemorrhage, the relative position in the brain region, the volume and area of ​​the site of cerebral hemorrhage, the area and position of the location with the largest cerebral hemorrhage, the number and volume of each type of cerebral hemorrhage, the image pattern and texture characteristics of the cerebral hemorrhage region, and the shape and state of the cerebral hemorrhage.

[0052] Furthermore, according to other embodiments, when acquiring images related to degenerative arthritis, the processor 1300 of the electronic device 1000 can quantify the joint spacing, hardening location, degree of hardening, bone spur location, degree of bone spur, bone number, bone spacing, bone thickness, etc., of the target area in the image and generate second non-image data.

[0053] If necessary, the processor 1300 of the electronic device 1000 according to one embodiment of the present invention can apply the first and second non-video data to a third artificial intelligence model and classify them into pre-defined groups. The third artificial intelligence model can analyze the first and second non-video data using machine learning techniques such as SVM, regression, and fully connected neural network models to perform the desired dual classification, multiple classification, etc. At this time, the pre-defined groups are the same as those described above, so a detailed explanation of them will be omitted.

[0054] The processor 1300 of the electronic device 1000 according to one embodiment of the present invention can generate a second artificial intelligence model for comprehensively predicting the prognosis of a disease based on medical images and non-image data. The electronic device 1000 according to one embodiment of the present invention can acquire prognostic statistical data of a target disease through a transmitting / receiving unit 1100. The prognostic statistical data can be grouped by location, volume, age, sex, country, region, and / or race of the target disease. The electronic device 1000 according to one embodiment of the present invention can learn a second artificial intelligence model that can predict the prognosis of a target disease based on input data and prognostic statistical data of the disease.

[0055] If necessary, the processor 1300 of the electronic device 1000 can use a second artificial intelligence model to predict the prognosis of the target disease from the patient's medical images and medical data. More specifically, the electronic device 1000 can predict the prognosis of the target disease including weighted values ​​(or parameters) derived through the artificial intelligence model, as described above.

[0056] According to one embodiment of the present invention, the second artificial intelligence model can output a prognosis prediction value for a target disease using the output value of the first artificial intelligence model and the first and second non-video data as input data.

[0057] If necessary, the second artificial intelligence model can use the output values ​​of the first and third artificial intelligence models as input data to output a prognosis prediction for the target disease.

[0058] If necessary, the artificial intelligence model according to one embodiment of the present invention can use the scores or normalized probability values ​​of each class for the final classification of the deep neural network as classification values. For example, the class scores of the softmax layer or the normalized probability values ​​obtained as a result of the softmax layer can be calculated as classification values ​​for the relevant region of interest and input into the artificial intelligence model. Mathematical formula 1

[0059] TIFF2026520123000002.tif3877

[0060]

[0061] Here, S C This will be a Class C score, w c i is A i This is the weighted value. Mathematical formula 2

[0062] TIFF2026520123000003.tif2777

[0063]

[0064] TIFF2026520123000004.tif26166

[0065] A processor 1300 of an electronic device 1000 according to one embodiment of the present invention can generate an integrated predictive model for predicting the prognosis of a target disease. The processor 1300 of the electronic device 1000 may also be configured to use the generated integrated predictive model to perform calculations to predict the prognosis of the target disease from input data including a patient's medical images and / or medical data. Furthermore, the processor 1300 of the electronic device 1000 may be configured to apply processed medical images and non-image data obtained from the patient's medical images and medical data to the generated integrated predictive model to perform calculations to predict the prognosis of the target disease.

[0066] According to one specific embodiment of the present invention, the electronic device 1000 can acquire brain CT medical images and medical data of a patient suspected of having a cerebrovascular disease. The electronic device 1000 uses the patient's medical images as input data to a first artificial intelligence model to classify the type of cerebrovascular disease in the patient, and if it is determined that there is a high probability of cerebral hemorrhage, hematoma, or hypertrophy, it can classify the patient into the appropriate group.

[0067] In addition, the electronic device 1000 can identify and quantify the coordinates (x, y, z) of the center point of the cerebral hemorrhage through the patient's medical images, and quantify the volume (e.g., mL) of the cerebral hemorrhage site, saving it as second non-image data. Since the risk of cerebral hemorrhage can vary depending on the coordinates of the center point, volume, etc., quantifying this data and using it for analysis allows for a more accurate prediction of the prognosis of the target disease. This data can be generated in the form of scores, saved in the form of vectors, and input into a second artificial intelligence model.

[0068] Furthermore, the electronic device 1000 can quantify medical data and generate first non-visual data. In one embodiment, according to the Glasgow Coma Scale, the degree of the patient's eye response can be scored from 1 to 4 points, the degree of their verbal response from 1 to 5 points, and the degree of their motor response from 1 to 6 points to determine whether the patient is in serious condition.

[0069] Furthermore, according to other embodiments, the risk of the disease in question can be scored by age group of the patient, by gender, and if there is an underlying disease, the risk can be assigned based on the underlying disease, thereby scoring the patient's medical data according to the target disease. Specifically, in the case of cerebral hemorrhage, the risk can be scored as 5 if the patient is 60 years or older, 4 if they are in their 50s, and 3 if they are in their 40s. If there is hypertension, the risk can be scored as 5, and if there is a history of cerebral hemorrhage, a risk of 20 can be added. In this way, the patient's medical data can be quantified and the first non-visual data can be generated.

[0070] The electronic device can input first and second non-visual data into a third artificial intelligence model and classify them into pre-defined groups. Based on the patient's first and second non-visual data, the electronic device can determine that the patient is less likely to develop cerebral hemorrhage or hematoma enlargement.

[0071] The second artificial intelligence model of the electronic device can predict disease prognosis using video data and first and second non-video data as input.

[0072] If necessary, the second artificial intelligence model can use the output values ​​of the first and third artificial intelligence models as input data to predict the prognosis of a disease. If the disease groups classified by the first and third artificial intelligence models are different, the second artificial intelligence model can be used to make additional prognosis determinations, thereby increasing the accuracy of prognosis prediction.

[0073] The following describes a method for predicting the prognosis of a target disease based on medical video and non-video data, which corresponds to yet another embodiment of the present invention.

[0074] Figures 7 and 8 are flowcharts illustrating a method for predicting the prognosis of a target disease according to one embodiment of the present invention.

[0075] The prognosis prediction method according to one embodiment of the present invention is a method for predicting the prognosis of a disease by acquiring video and non-video data of a patient and using the results of analyzing these individually or in combination to predict the prognosis of the target disease more accurately.

[0076] A prognosis prediction method according to a first embodiment of the present invention may include the steps of: acquiring medical images of a patient in which abnormal areas are to be detected (S1100); video data analysis (S1200) which classifies target diseases in the medical images into pre-set groups using a first artificial intelligence model; a first non-video data generation (S1300) which acquires and quantifies the patient's medical data; a second non-video data generation (S1400) which quantifies target disease-related features in the medical images; and an integrated analysis (S1500) which applies the result values ​​from the video data analysis step, the first and second non-video data to a second artificial intelligence model to predict the prognosis of the target disease.

[0077] A prognosis prediction method according to a second embodiment of the present invention may include the steps of: acquiring medical images of a patient in which abnormal areas are to be detected (S2100); video data analysis (S2200) which classifies target diseases in the medical images into pre-set groups using a first artificial intelligence model; a first non-video data generation (S2300) which acquires and quantifies the patient's medical data; a second non-video data generation (S2400) which quantifies target disease-related features in the medical images; a non-video data analysis (S2500) which applies the first and second non-video data to a third artificial intelligence model and classifies them into pre-set groups; and an integrated analysis (S2600) which applies the result values ​​from the video data analysis step and the result values ​​from the non-video data analysis step to the second artificial intelligence model and comprehensively predicts the prognosis of the target disease.

[0078] If necessary, in the method for predicting the prognosis of a target disease according to the first and second embodiments of the present invention, the video data analysis step may include the step of detecting a region of interest containing an abnormal area from medical video acquired using a first artificial intelligence model, the step of segmenting the video of the region of interest, and the step of extracting a region related to the target disease within the segmented video. If necessary, after extracting the region related to the target disease, the first artificial intelligence model can classify the target disease in the extracted video into a predetermined group.

[0079] If necessary, the first non-visual data generation stage may include a stage for scoring medical data. Medical data may include one or more data points in groups, including the patient's age, sex, family history, genes, underlying diseases, pre-existing diseases, medications taken, body temperature, height, weight, lifestyle, and symptoms exhibited.

[0080] If necessary, the second non-video data generation stage may include a stage for quantifying the characteristics of disease-related regions. If necessary, the electronic device can generate the second non-video data by using data extracted from disease-related regions within the divided video to quantify the characteristics of those regions.

[0081] If necessary, the groups pre-configured by the methods of the first and second embodiments of the present invention may include one or more groups, such as disease type groups, disease severity grade groups, disease benignity / non-benignity groups, and disease prognosis groups, and the target disease-related features may include one or more groups, such as the location, size, volume of the target disease-related region and physical characteristics related to the target disease.

[0082] If necessary, the integrated analysis step of the method for predicting the prognosis of a target disease according to the first and second embodiments of the present invention may include a step of predicting the prognosis of degenerative arthritis or physical growth and development.

[0083] If necessary, the integrated analysis step in the method for predicting the prognosis of target diseases according to the first and second embodiments of the present invention may include a step of predicting the prognosis of cerebral hemorrhage hematoma hypertrophy.

[0084] A further embodiment of the present invention may be a computer-readable recording medium that stores a program for performing a prognosis prediction method for a target disease of the present invention.

[0085] The features, structures, and effects described in the embodiments above are included in at least one embodiment of the present invention and are not necessarily limited to just one embodiment. Furthermore, the features, structures, and effects exemplified in each embodiment can be combined or modified and implemented in other embodiments by a person with ordinary skill in the art to which the embodiment belongs. Therefore, such combinations and modifications should be interpreted as being within the scope of the present invention. Also, although the above description has focused on embodiments, these are merely illustrative and do not limit the present invention. A person with ordinary skill in the art to which the present invention belongs will understand that various modifications and applications not exemplified above are possible without departing from the essential characteristics of these embodiments. In other words, each component specifically shown in the embodiments can be modified and implemented. And any differences related to such modifications and applications should be interpreted as being within the scope of the present invention as defined in the attached claims.

Claims

1. In a method for predicting the prognosis of a target disease based on medical imaging and non-imaging data, The stage of acquiring medical images of the patient in order to detect abnormal areas; A video data analysis step in which the target disease in the medical video is classified into a predetermined group using a first artificial intelligence model; A first non-video data generation step involves acquiring and quantifying the patient's medical data; A second non-video data generation step involves quantifying the disease-related characteristics within the aforementioned medical video; A method for predicting the prognosis of a target disease, comprising: an integrated analysis step in which the result values ​​of the video data analysis step and the first and second non-video data are applied to a second artificial intelligence model to predict the prognosis of the target disease;

2. In a method for predicting the prognosis of a target disease based on medical imaging and non-imaging data, The stage of acquiring medical images of the patient in order to detect abnormal areas; A video data analysis step in which the target disease in the medical video is classified into a predetermined group using a first artificial intelligence model; A first non-video data generation step involves acquiring and quantifying the patient's medical data; A second non-video data generation step involves quantifying the disease-related characteristics within the aforementioned medical video; A non-video data analysis step in which the first and second non-video data are applied to a third artificial intelligence model and classified into the pre-set groups; A method for predicting the prognosis of a target disease, comprising: an integrated analysis step of applying the result values ​​of the video data analysis step and the result values ​​of the non-video data analysis step to a second artificial intelligence model to comprehensively predict the prognosis of the target disease;

3. The aforementioned video data analysis stage is A step of detecting a region of interest, including the abnormal area, in the aforementioned medical image; The step of segmenting the image of the region of interest; A method for predicting the prognosis of a target disease according to claim 1 or claim 2, comprising the step of extracting the target disease-related region from the divided video.

4. The second non-video data generation step is: A method for predicting the prognosis of a target disease according to claim 3, comprising the step of quantifying the characteristics of the area related to the target disease.

5. The aforementioned pre-configured group is It includes one or more groups, including disease type groups, disease severity grade groups, benignity / non-benignity groups, and disease prognosis groups. The aforementioned disease-related characteristics are, A method for predicting the prognosis of a target disease according to claim 1 or 2, characterized in that it includes one or more groups comprising the location, size, volume, and physical characteristics related to the target disease in the area related to the target disease.

6. The aforementioned medical data The group includes one or more data points, including the patient's age, sex, family history, genes, underlying disease, pre-existing disease, medications taken, body temperature, height, weight, lifestyle, and symptoms exhibited. The first non-video data generation step is: A method for predicting the prognosis of a target disease according to claim 1 or claim 2, comprising the step of scoring the aforementioned medical data.

7. The aforementioned integrated analysis step is A method for predicting the prognosis of a target disease according to claim 1 or 2, comprising the step of predicting the prognosis of degenerative arthritis or physical growth and development.

8. The aforementioned integrated analysis step is A method for predicting the prognosis of a target disease according to claim 1 or claim 2, comprising the step of predicting the prognosis of hematoma enlargement due to cerebral hemorrhage.

9. A computer-readable recording medium having recorded a program for causing a computer to perform the method described in claim 1 or claim 2.

10. In an electronic device that predicts the prognosis of a target disease based on medical imaging and non-imaging data, An electronic device comprising a processor configured to acquire medical images of a patient in which abnormal areas are to be detected, classify target diseases in the medical images into pre-defined groups using a first artificial intelligence model, acquire the patient's medical data, quantify it to generate first non-image data, quantify target disease-related features in the medical images to generate second non-image data, and apply the output values ​​of the first artificial intelligence model and the first and second non-image data to the second artificial intelligence model to integrally predict the prognosis of the target disease.

11. In an electronic device that predicts the prognosis of a target disease based on medical imaging and non-imaging data, An electronic device comprising a processor configured to acquire medical images of a patient in which abnormal areas are to be detected, classify target diseases in the medical images into pre-defined groups using a first artificial intelligence model, acquire the patient's medical data, quantify it to generate first non-image data, quantify target disease-related features in the medical images to generate second non-image data, apply the first and second non-image data to a third artificial intelligence model to classify the target diseases into pre-defined groups, and apply the output values ​​of the first and third artificial intelligence models to the second artificial intelligence model to comprehensively predict the prognosis of the target diseases.

Images