Apparatus for evaluating performance of determination criteria set to analyze biometric data and control method thereof

Abstract

The present disclosure provides an apparatus for evaluating the performance of determination criteria set to analyze biometric data, and a control method thereof. A control method according to an embodiment of the present disclosure comprises the steps of: setting, on the basis of biometric data, a disease to be predicted, and identifying a preset determination criteria for the biometric data, in correspondence to the disease; adjusting at least one of a plurality of elements included in the determination criteria and setting a new determination criteria corresponding to the disease; applying the new determination criteria to a first rule-based model and obtaining prediction results for the disease, corresponding to respective pieces of a plurality of biometric data; and evaluating on the basis of the prediction results the new determination criteria and displaying the evaluation results, wherein the plurality of elements respectively correspond to a plurality of pieces of feature information identified from the biometric data by the first rule-based model to predict the disease.

Description

Device for evaluating the performance of judgment criteria set for analyzing biometric data and its control method

[0001] The present disclosure relates to artificial intelligence technology in the medical field, and more specifically, to a device for evaluating the performance of judgment criteria set for analyzing biometric data and a control method thereof.

[0002] Recent advancements in information and communication technology have led to the utilization of artificial intelligence (AI) in various fields. In particular, various attempts are being made to integrate AI technology into the medical field, which has traditionally relied on specialized and limited resources such as doctors and researchers to diagnose patients' illnesses, to improve efficiency. For example, this involves inputting biometric data acquired in real time from patients into an AI-based model, analyzing the biometric data to diagnose the patient's condition or predict disease.

[0003] Meanwhile, most methods for analyzing biometric data using AI technology begin with extracting identifiable features from the biometric data. These features are then compared against pre-determined conditions or criteria to predict a patient's condition or diagnose a disease. However, even for the same biometric data, different analysis results can be obtained depending on the applied conditions or criteria. Therefore, to improve the accuracy of biometric data analysis, it is necessary to apply appropriate conditions or criteria to the extracted features, as well as accurately extract them.

[0004] However, conventional AI technologies in the medical field have focused solely on analyzing biometric data, failing to adequately consider the conditions or criteria applied to analyze biometric data. This lack of evaluation inevitably results in reduced accuracy and reliability of biometric data interpretation.

[0005] The present disclosure has been made in response to the aforementioned background technology, and aims to provide a device for evaluating the performance of judgment criteria set for analyzing biometric data and a control method thereof.

[0006] However, the problems to be solved in this disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood based on the description below.

[0007] According to an embodiment of the present disclosure for realizing the task as described above, a method for evaluating the performance of a judgment criterion set for analyzing bio-data, which is performed by a computing device including at least one processor, includes the steps of: setting a disease to be predicted based on bio-data; identifying a judgment criterion for the bio-data set in correspondence with the disease; setting a new judgment criterion corresponding to the disease by adjusting at least one of a plurality of elements included in the judgment criterion; applying the new judgment criterion to an artificial intelligence-based first model to obtain a prediction result regarding the disease corresponding to each of a plurality of bio-data; and evaluating the new judgment criterion based on the prediction result and displaying the evaluation result.

[0008] Alternatively, the plurality of elements each correspond to a plurality of feature information that the artificial intelligence-based first model identifies from the biometric data for prediction of the disease.

[0009] Alternatively, the method may further include the step of displaying a list of a plurality of diseases that can be predicted based on the biometric data, a step of displaying a plurality of elements included in a judgment criterion for the biometric data set corresponding to the selected disease when any one of the lists of the plurality of diseases is selected, and a step of displaying a UI (User Interface) for modifying the plurality of elements.

[0010] Alternatively, the method may include a step of displaying a plurality of characteristic information identifiable from the biometric data, and a step of obtaining at least one characteristic information selected from the plurality of biometric data when at least one characteristic information among the displayed plurality of characteristic information is selected, and the step of displaying the evaluation result may include a step of displaying the obtained prediction result and the obtained at least one characteristic information for each of the plurality of biometric data.

[0011] Alternatively, the method may include the step of obtaining the at least one selected feature information, when at least one feature information among the displayed plurality of feature information is selected, inputting the plurality of biometric data into a second model based on pre-learned machine learning, thereby obtaining the at least one feature information corresponding to each of the plurality of biometric data.

[0012] Alternatively, the step of obtaining the prediction result may include a step of inputting the plurality of biometric data into a third model based on pre-learned machine learning to obtain the plurality of characteristic information corresponding to the plurality of elements for the plurality of biometric data, and a step of inputting the obtained plurality of characteristic information into a first model based on a rule to which the new judgment criterion is applied to obtain the prediction result regarding the disease corresponding to each of the plurality of biometric data.

[0013] Alternatively, the displaying step may include a step of obtaining a plurality of label data corresponding to the plurality of biometric data to which labels relating to the disease have been assigned, a step of evaluating the performance of a rule-based first model to which the new judgment criterion is applied based on the obtained plurality of label data, and a step of displaying an evaluation result for the new judgment criterion.

[0014] Alternatively, the step of displaying the evaluation results for the new judgment criteria may include a step of evaluating the performance of a rule-based first model to which preset judgment criteria are applied in response to the disease based on the plurality of label data, and a step of comparing the evaluation results for the new judgment criteria with the evaluation results for the preset judgment criteria, and displaying the comparison results.

[0015] Alternatively, the method may further include a step of updating the new judgment criterion and the evaluation results for the new judgment criterion in the history information regarding the plurality of judgment criteria previously applied to the disease.

[0016] Alternatively, the method may further include a step of selecting an optimal element for each of a plurality of elements included in a judgment criterion corresponding to the disease based on the updated history information, and a step of generating an optimal judgment criterion corresponding to the disease by combining the selected optimal elements.

[0017] Alternatively, the preset judgment criterion may further include a step of determining whether to change the preset evaluation criterion in response to the disease based on the evaluation result, wherein the preset judgment criterion is a judgment criterion having the highest evaluation result among the plurality of judgment criteria applied to the artificial intelligence-based first model to predict the disease in the past.

[0018] According to one embodiment of the present disclosure, a computer program stored in a computer-readable storage medium, when the computer program is executed on one or more processors, performs an operation of evaluating the performance of a judgment criterion set for analyzing biometric data, and includes an operation of setting a disease to be predicted based on the biometric data and identifying a judgment criterion for the biometric data set in correspondence with the disease, an operation of adjusting at least one of a plurality of elements included in the judgment criterion to set a new judgment criterion corresponding to the disease, an operation of applying the new judgment criterion to an artificial intelligence-based first model to obtain a prediction result for the disease corresponding to each of a plurality of biometric data, and an operation of evaluating the new judgment criterion based on the prediction result and displaying the evaluation result.

[0019] A computing device for evaluating the performance of a judgment criterion set for analyzing biometric data according to one embodiment of the present disclosure includes a processor including at least one core, a memory including program codes executable by the processor, and a display, wherein the processor sets a disease to be predicted based on biometric data, identifies a judgment criterion for the biometric data set in correspondence with the disease, adjusts at least one of a plurality of elements included in the judgment criterion to set a new judgment criterion corresponding to the disease, applies the new judgment criterion to an artificial intelligence-based first model, obtains a prediction result for the disease corresponding to each of a plurality of biometric data, evaluates the new judgment criterion based on the prediction result, and displays the evaluation result.

[0020] According to an apparatus for evaluating the performance of a judgment criterion set for analyzing biometric data of an embodiment of the present disclosure and a control method thereof, by providing an evaluation result for a judgment criterion applied to an artificial intelligence-based model for analyzing biometric data, a user can set an appropriate judgment criterion for analyzing biometric data, and improve the reliability of the user for the artificial intelligence-based model.

[0021] FIG. 1 is an exemplary diagram schematically illustrating a control method of a computing device for evaluating the performance of judgment criteria set for analyzing biometric data according to one embodiment of the present disclosure.

[0022] FIG. 2 is a schematic block diagram of a computing device according to one embodiment of the present disclosure.

[0023] FIG. 3 is a schematic flowchart of a control method of a computing device for evaluating the performance of judgment criteria set for analyzing biometric data according to an embodiment of the present disclosure.

[0024] FIG. 4 is a drawing for explaining a plurality of elements included in a judgment criterion according to one embodiment of the present disclosure.

[0025] FIG. 5 is a diagram illustrating a method for setting a new judgment criterion by adjusting a plurality of elements included in the judgment criterion according to one embodiment of the present disclosure.

[0026] FIG. 6 is an exemplary diagram showing prediction results for a set disease obtained based on the new judgment criteria set in FIG. 5 according to one embodiment of the present disclosure.

[0027] FIG. 7 is a block diagram of a computing device according to another embodiment of the present disclosure.

[0028] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. The embodiments presented in this disclosure are provided to enable those skilled in the art to utilize or implement the contents of the present disclosure. Accordingly, various modifications to the embodiments of the present disclosure will be apparent to those skilled in the art. That is, the present disclosure may be implemented in various different forms and is not limited to the embodiments described below.

[0029] Throughout the specification of this disclosure, identical or similar drawing numbers refer to identical or similar components. Furthermore, for the purpose of clearly describing the disclosure, drawing numbers for parts in the drawings that are not relevant to the description of the disclosure may be omitted.

[0030] The term "or" as used herein is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified herein or the context makes the meaning unclear, "X employs A or B" should be understood to mean either of its natural inclusive permutations. For example, unless otherwise specified herein or the context makes the meaning unclear, "X employs A or B" can be interpreted to mean either X employs A, X employs B, or X employs both A and B.

[0031] The term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the related concepts listed.

[0032] The terms "comprises" and / or "comprising" as used herein should be understood to imply the presence of certain features and / or components. However, it should be understood that the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other features, other components, and / or combinations thereof.

[0033] Unless otherwise specified in this disclosure or unless the context makes it clear that the singular form is being referred to, the singular should generally be construed to include “one or more.”

[0034] The term "Nth (N is a natural number)" used in this disclosure can be understood as an expression used to mutually distinguish components of this disclosure based on a predetermined standard, such as a functional perspective, a structural perspective, or convenience of explanation. For example, components performing different functional roles in this disclosure can be distinguished as a first component or a second component. However, components that are substantially the same within the technical spirit of this disclosure but must be distinguished for convenience of explanation may also be distinguished as a first component or a second component.

[0035] The term "acquisition" as used in this disclosure may be understood to mean not only receiving data through a wired or wireless communication network with an external device or system, but also generating data in an on-device form.

[0036] Meanwhile, the term "module" or "unit" used in the present disclosure can be understood as a term referring to an independent functional unit that processes computing resources, such as a computer-related entity, firmware, software or a part thereof, hardware or a part thereof, or a combination of software and hardware. At this time, the "module" or "unit" may be a unit composed of a single element, or a unit expressed as a combination or set of multiple elements. For example, as a narrow concept, a "module" or "unit" may refer to a hardware element of a computing device or a set thereof, an application program that performs a specific function of software, a processing process implemented through software execution, or a set of instructions for program execution, etc. In addition, as a broad concept, a "module" or "unit" may refer to the computing device itself that constitutes the system, or an application running on the computing device, etc. However, since the above-described concept is only an example, the concept of “module” or “part” may be defined in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0037] The term "model" as used herein can be understood as a system implemented using mathematical concepts and language to solve a specific problem, a set of software units for solving a specific problem, or an abstract model of a processing process for solving a specific problem. For example, a neural network "model" can refer to an entire system implemented as a neural network that has problem-solving capabilities through learning. In this case, the neural network can have problem-solving capabilities by optimizing the parameters connecting nodes or neurons through learning. A neural network "model" may include a single neural network or a set of neural networks composed of multiple neural networks.

[0038] The term "data" used in this disclosure may include "images," signals, and the like. The term "image" used in this disclosure may refer to multidimensional data composed of discrete image elements. In other words, "image" may be understood as a term referring to a digital representation of an object visible to the human eye. For example, "image" may refer to multidimensional data composed of elements corresponding to pixels in a two-dimensional image. "Image" may refer to multidimensional data composed of elements corresponding to voxels in a three-dimensional image.

[0039] The explanation of the above terms is intended to aid understanding of the present disclosure. Therefore, unless explicitly stated as limiting the contents of the present disclosure, it should be noted that the above terms are not intended to limit the technical ideas of the contents of the present disclosure.

[0040] FIG. 1 is an exemplary diagram schematically illustrating a control method of a computing device for evaluating the performance of judgment criteria set for analyzing biometric data according to one embodiment of the present disclosure.

[0041] A computing device (100) according to an embodiment of the present disclosure may be a hardware device or a part of a hardware device that performs comprehensive processing and calculation of data, or may be a software-based computing environment connected to a communication network. For example, the computing device (100) may be a server that performs intensive data processing functions and shares resources, or may be a client that shares resources through interaction with a server. In addition, the computing device (100) may be a cloud system in which multiple servers and clients interact to comprehensively process data. Since the above description is only one example related to the type of computing device (100), the type of computing device (100) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0042] For example, a computing device (100) according to one embodiment of the present disclosure may be implemented as a biosignal measuring device that measures biometric data of a user (e.g., a patient, etc.). However, the present invention is not limited thereto, and the computing device (100) may be implemented as various electronic devices such as a desktop, laptop, smartphone, server device, smart watch, smart band, smart ring, etc.

[0043] Meanwhile, the biometric data acquired by the computing device (100) may be diverse, such as electrocardiogram (ECG), electroencephalogram (EEG), and photoplethysmogram (PPG). However, for the convenience of explaining the present disclosure, the biometric data will be limited to electrocardiogram data and an example of the present disclosure will be explained below.

[0044] A computing device (100) according to an embodiment of the present disclosure can obtain biometric data from a subject (e.g., a patient, etc.) and analyze the obtained biometric data to predict or diagnose a disease of the subject. To this end, the computing device (100) can use an artificial intelligence-based model. Specifically, referring to FIG. 1, the computing device (100) can input the obtained biometric data into an artificial intelligence-based model to analyze the biometric data and obtain an analysis result of the biometric data. As a result of the analysis, the computing device (100) can obtain information such as the possibility of a disease of the subject (e.g., a disease probability value) from the artificial intelligence-based model. Meanwhile, predicting or diagnosing a disease may also include analyzing the obtained biometric data to determine the condition of the subject, particularly the condition of the body organ from which the biometric data is obtained. For example, if the biometric data is electrocardiogram data, diagnosing a disease may include not only determining a heart disease but also determining the condition of the heart (heart rate, electrical activity pattern, etc.).

[0045] At this time, the AI-based model can diagnose (or predict) the disease of the subject of the biometric data by applying preset judgment criteria (or conditions) to the input biometric data. Specifically, the AI-based model extracts feature information (i.e., feature values) from the input biometric data and applies preset judgment criteria to the extracted feature information to diagnose the disease of the subject of the biometric data.

[0046] An artificial intelligence-based model according to one embodiment of the present disclosure may be a rule-based learning model. In this case, the preset judgment criteria may be a rule or rule set applied to the rule-based learning model. However, this is not limited thereto, and the artificial intelligence-based model may also be a pre-trained neural network model (e.g., a multi-layer perceptron (MLP), a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network, etc.).

[0047] Meanwhile, the preset judgment criteria may vary depending on the type of biometric data and the type of disease to be diagnosed through the biometric data. For example, if the biometric data is electrocardiogram (ECG) data or electroencephalogram (EEG) data, the preset judgment criteria applied to each biometric data may differ. Furthermore, even for the same ECG data, the criteria may differ depending on whether the disease to be diagnosed through the ECG data is atrial fibrillation (AF) or right ventricular hypertrophy (RVH). Similarly to the preset judgment criteria, the feature information extracted from the biometric data to which the preset judgment criteria are applied may also vary depending on the type of biometric data and the type of disease.

[0048] A computing device (100) according to an embodiment of the present disclosure can evaluate the performance of judgment criteria set in an artificial intelligence-based model. Specifically, the computing device (100) can evaluate whether the preset judgment criteria are appropriate based on output data (i.e., disease diagnosis results) of the artificial intelligence-based model obtained by applying preset judgment criteria to biometric data. In particular, when the judgment criteria set in the artificial intelligence-based model are changed by a user, the computing device (100) can also evaluate the performance of the new judgment criteria that have been changed. Through this, the computing device (100) can identify which of the previously set judgment criteria and the new judgment criteria is appropriate.

[0049] In particular, the computing device (100) can evaluate the performance of preset judgment criteria by judging the accuracy of output data (i.e., the diagnosis result of a disease). Through this, the user can change the preset judgment criteria for the biometric data (and for the disease to be diagnosed through the biometric data) and receive the evaluation result for the changed criteria from the computing device (100), thereby finding an appropriate criterion to be applied to the biometric data. In addition, the computing device (100) can also suggest an optimized criterion to the user based on the history of the preset judgment criteria set in the artificial intelligence model and the evaluation result for each criterion.

[0050] Hereinafter, embodiments of the present disclosure related thereto will be described in detail with reference to FIGS. 2 to 6.

[0051] FIG. 2 is a schematic block diagram of a computing device (100) according to one embodiment of the present disclosure.

[0052] Referring to FIG. 2, a computing device (100) according to one embodiment of the present disclosure may include a processor (110), a memory (120), and a display unit (130). However, FIG. 2 is merely an example, and the computing device (100) may include other components for implementing a computing environment. In addition, only some of the disclosed components may be included in the computing device (100).

[0053] The processor (110) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for performing computing operations. For example, the processor (110) may read a computer program to perform data processing for machine learning. The processor (110) may process computational processes such as processing input data for machine learning, feature extraction for machine learning, and error calculation based on backpropagation. The processor (110) for performing such data processing may include a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The type of the processor (110) described above is only one example, and thus, the type of the processor (110) may be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0054] The processor (110) is connected to other components of the computing device (100) (i.e., memory (120) and display (130)) and controls the overall operation of the computing device (100).

[0055] The memory (120) according to one embodiment of the present disclosure may be understood as a configuration unit including hardware and / or software for storing and managing data processed in the computing device (100). That is, the memory (120) may store any type of data generated or determined by the processor (110) and any type of data received by a communication interface (not shown). For example, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory (120) may also include a database system that controls and manages data in a predetermined system. The type of memory (120) described above is only one example, and thus the type of memory (120) can be configured in various ways within a range understandable to those skilled in the art based on the contents of the present disclosure.

[0056] The memory (120) can structure and organize and manage data, combinations of data, and program codes executable by the processor (110) required for the processor (110) to perform operations. For example, the memory (120) can store an artificial intelligence-based model to be described later, a plurality of biometric data (10) (e.g., a plurality of electrocardiogram data, etc.). In addition, the memory (120) can store a program code for operating an artificial intelligence-based model or evaluating preset conditions set in an artificial intelligence-based model, a program code for operating an artificial intelligence model to perform inference in accordance with the intended use of the computing device (100), and processed data generated as the program code is executed.

[0057] The display (130) can display various images. Here, the images include both still images and moving images. In particular, the display (130) can display biometric data acquired by the computing device (100), and can also display the evaluation results of preset conditions applied to the biometric data. To this end, the display (130) can be implemented as various types of displays, such as an LCD (Liquid Crystal Display Panel), an OLED (Organic Light Emitting Diodes), an LCoS (Liquid Crystal on Silicon), a DLP (Digital Light Processing), etc. In addition, the display (130) can also include a driving circuit, a backlight unit, etc., which can be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc.

[0058] Meanwhile, the display (130) may be implemented as a touch screen by being combined with a touch panel, and in this case, the display (130) may perform the function of not only an output interface that outputs an image through the touch screen, but also an input interface that receives a user's touch input.

[0059] FIG. 3 is a schematic flowchart of a control method of a computing device for evaluating the performance of judgment criteria set for analyzing biometric data according to an embodiment of the present disclosure.

[0060] Referring to FIG. 3, the processor (110) can set a disease to be predicted based on biometric data and identify a judgment criterion for biometric data set in response to the disease (S310).

[0061] Specifically, the processor (110) can analyze biometric data to set a disease to be predicted (or diagnosed). In particular, prior to setting a disease, the processor (110) can set the type of biometric data. For each type of biometric data, multiple disease types that can be predicted through the corresponding biometric data can be set. Once the type of biometric data is set, the processor (110) can set any one of the multiple disease types that can be predicted using the set type of biometric data. In other words, the processor (110) can set specific biometric data and a specific disease type corresponding to the specific biometric data.

[0062] Meanwhile, the processor (110) can set multiple diseases to be predicted based on the same biometric data. For example, the processor (110) can simultaneously set ventricular tachycardia and atrial tachycardia as diseases to be predicted based on electrocardiogram data. In this case, the processor (110) can identify judgment criteria for the preset biometric data corresponding to each of the multiple diseases (ventricular tachycardia and atrial tachycardia).

[0063] Additionally, the processor (110) may simultaneously set diseases to be predicted based on different biometric data. For example, the processor (110) may set myocardial infarction and stroke as diseases to be predicted based on electrocardiogram data and electroencephalogram data, respectively.

[0064] Meanwhile, the processor (110) can identify a judgment criterion for the biometric data set in response to the disease after setting the disease. Here, the judgment criterion may be a reference value or reference range applied to feature information extracted from the biometric data to predict the disease. Furthermore, the judgment criterion may be the type, shape, trend, etc. of the feature information extracted from the biometric data.

[0065] Criteria for judgment may vary depending on the type of biometric data and the type of disease. That is, even for identical electrocardiogram data, the criteria for judgment for ventricular tachycardia may differ from those for atrial tachycardia.

[0066] FIG. 4 is a drawing for explaining a plurality of elements included in a judgment criterion according to one embodiment of the present disclosure.

[0067] According to one embodiment of the present disclosure, the judgment criteria may include multiple elements corresponding to a plurality of characteristic pieces of information identified from biometric data through a rule-based model to predict a set disease. The elements may be reference values, reference ranges, types, shapes, trends, etc., applied to the characteristic pieces identified from the biometric data. Multiple elements may be set for the same characteristic piece of information.

[0068] Meanwhile, elements may also correspond to the location where the feature information is identified, i.e., the sensing location. For example, if the biometric data is electrocardiogram data, elements may correspond to the feature information identified in the electrocardiogram data and the lead location where the identified feature information was acquired.

[0069] Referring to Fig. 4, judgment criteria may be set for each of a plurality of diseases that can be predicted through electrocardiogram data, and each judgment criterion may include a plurality of elements corresponding to feature information extracted from electrocardiogram data to predict the corresponding disease. 1) The judgment criteria for lateral myocardial infarction (Lateral STEMI) may include two elements: ① detection of ST segment elevation identified based on the voltage at the J point of the ST segment (in Lead I, Lead AvL, Lead V5, and Lead V6), ② detection of ST segment depression identified based on the voltage at the J point of the ST segment (in Lead III and Lead aVF). 2) The judgment criteria for atrial fibrillation may include four elements: ① detection of irregularity of the RR interval identified based on the RR interval, ② non-observation of the P wave, ③ detection of change in the ventricular rate, and ④ duration of the QRS complex less than 120 ms. 3) The criteria for judging right ventricular hypertrophy may include the following four elements: ① right axis deviation of 110° or more, ② (lead V1) R wave height of 7 mm or more, ③ (lead V5 or lead V6) S wave depth of 7 mm or more, and ④ detection of QRS complex duration of less than 120 ms.

[0070] Meanwhile, according to one embodiment of the present disclosure, the processor (110) can display a list of multiple diseases that can be predicted based on biometric data.

[0071] The processor (110) can display a disease list including information on multiple diseases through the display (130). When the processor (110) receives a user input for selecting one of the output disease lists, it can set the selected disease as a target of the analysis result. At this time, the disease list can be displayed separately according to biometric data required to predict each disease. Specifically, the processor (110) can display multiple biometric data (10) and display disease types that can be predicted using each biometric data as a disease list. Alternatively, the processor (110) can display multiple biometric data (10) and, when any one of the multiple biometric data (10) is selected, display multiple disease types that can be predicted using the selected biometric data as a disease list. For example, disease types that can be predicted using electrocardiogram data and disease types that can be predicted using electroencephalogram data can be displayed separately. Alternatively, when a user selects biometric data to be used, a list of multiple diseases that can be predicted using the selected biometric data can be displayed on the display (130).

[0072] Meanwhile, the processor (110) may display a list of diseases by dividing them into an upper disease type and a plurality of lower disease types corresponding to the upper disease type. For example, the processor (110) may set ① myocardial infarction and ② electrical activity pattern of the heart as upper disease types, and display i) anterior wall acute myocardial infarction, ii) lateral wall acute myocardial infarction, iii) ventricular tachycardia, iv) atrial tachycardia, etc. according to the location of occurrence as lower disease types of ① myocardial infarction. Through this, the processor (110) may usefully convey disease information that can be predicted through biometric data to the user.

[0073] And, when any one of a plurality of disease lists is selected, the processor (110) can display a plurality of elements included in the judgment criteria for biometric data set corresponding to the selected disease. To explain again with the above-described example, the processor (110) can display a list of a plurality of diseases (such as lateral myocardial infarction, atrial fibrillation, and right ventricular hypertrophy) that can be predicted through electrocardiogram data, and when atrial fibrillation is selected by the user among the plurality of diseases, the processor (110) can display four elements included in the judgment criteria for atrial fibrillation (in the case of ① detection of irregularity of RR interval identified based on the RR interval, ② non-observation of P wave, ③ detection of change in ventricular beat rate, and ④ duration of QRS complex less than 120 ms).

[0074] At this time, the processor (110) may display a UI (User Interface) for modifying multiple elements. Then, the user may modify at least one of the multiple elements included in the judgment criteria for the selected disease through the UI. For example, the processor (110) may display a field, a pop-up window, etc., for modifying each element. Then, the user may adjust the judgment criteria by changing the value of at least one of the multiple elements, changing the size of the range, or further specifying the judgment method related to the element through the displayed UI (field or pop-up window). More specifically, when the selected disease is atrial fibrillation, the user may ① newly add a time limit for the irregularity of the RR interval identified based on the RR interval, or ② newly add a time limit for the non-observation time of the P wave. Alternatively, the user may ④ change the judgment criterion for the length of the QRS complex duration from 120 ms to 121 ms. In this way, the user may freely change the judgment criteria set for each disease.

[0075] For example, the processor (110) may display, on the display (130), a code corresponding to a judgment criterion applied to a rule-based model related to a disease. At this time, the judgment criterion code may display codes corresponding to multiple elements included in the judgment criterion. The processor (110) may provide a UI that allows the user to modify the code corresponding to each element, thereby enabling the user to change the judgment criterion and the multiple elements included in the judgment criterion.

[0076] Referring again to FIG. 3, according to one embodiment of the present disclosure, the processor (110) may set a new judgment criterion corresponding to a disease by adjusting at least one of a plurality of elements included in the judgment criterion (S320).

[0077] FIG. 5 is a diagram illustrating a method for setting a new judgment criterion by adjusting a plurality of elements included in the judgment criterion according to one embodiment of the present disclosure.

[0078] Specifically, the processor (110) can determine whether a preset judgment criterion has changed in response to a set disease, based on a user input. In particular, if at least one element among a plurality of elements included in the judgment criterion is changed through a UI displayed by the processor (110) via the display (130), the processor (110) can determine that the preset judgment criterion has changed in response to the set disease. In addition, the processor (110) can set the changed judgment criterion as a new judgment criterion corresponding to the set disease.

[0079] Referring to FIG. 5, if a disease to be predicted by electrocardiogram data is set as right ventricular hypertrophy, and among the multiple factors included in the judgment criteria for right ventricular hypertrophy according to the user's input, ② the judgment criteria for the height of the R wave of electrocardiogram data detected in lead V1 is changed from 7 mm to 7.5 mm, and ④ the judgment criteria for the length of the duration of the QRS complex is changed from 120 ms to 119 ms, the processor (110) can determine that the judgment criteria for right ventricular hypertrophy have changed. In addition, the processor (110) can set the changed judgment criteria as new judgment criteria for right ventricular hypertrophy.

[0080] Referring again to FIG. 3, according to one embodiment of the present disclosure, the processor (110) can obtain prediction results regarding diseases corresponding to each of a plurality of biometric data (10) by applying new judgment criteria to a rule-based model (S330).

[0081] Specifically, the processor (110) can apply newly set new judgment criteria to a rule-based model and input a plurality of biometric data (10) into the rule-based model. The plurality of biometric data (10) can be stored in the memory (120) or can be acquired through a sensing unit and a communication interface. Meanwhile, the memory (120) can store a plurality of rule-based models corresponding to each biometric data and a disease predicted by each biometric data. Accordingly, when the biometric data and the disease predicted using the biometric data are set, the processor (110) can select a rule-based model corresponding to the biometric data and the set disease. The processor (110) can apply a new judgment criterion to the selected rule-based model and input the plurality of biometric data (10) into the rule-based model to which the new judgment criterion is applied. Then, the processor (110) can obtain a prediction result of the set disease for the plurality of biometric data (10). In particular, the processor (110) can obtain a score corresponding to the probability of the set disease from the rule-based model. For example, the score can be divided into multiple scores (-2, -1, 0, 1) according to the degree of disease likelihood, and a rule-based model can apply a new judgment criterion among the multiple scores to the biometric data to produce a score corresponding to the degree of disease likelihood identified.

[0082] According to one embodiment of the present disclosure, a processor (110) may input a plurality of biometric data (10) into a pre-trained machine learning-based model (hereinafter, a first machine learning-based model) to obtain a plurality of characteristic information corresponding to a plurality of elements of the plurality of biometric data (10). In addition, the processor (110) may input the obtained plurality of characteristic information into a rule-based model to which a new judgment criterion is applied to obtain a prediction result regarding a disease corresponding to each of the plurality of biometric data (10).

[0083] Specifically, in order to reduce the time required to predict a disease for a plurality of biometric data (10), the processor (110) may acquire feature information from each of the plurality of biometric data (10) and then input the acquired feature information into a rule-based model corresponding to each biometric data. In particular, the processor (110) may input the plurality of biometric data (10) into a first model based on machine learning that is trained to extract feature information for the biometric data, thereby acquiring feature information for each biometric data.

[0084] To this end, a plurality of machine learning-based first models trained to extract feature information of each biometric data corresponding to each biometric data may be stored in the memory (120). For example, the plurality of machine learning-based first models may be models trained using a plurality of biometric data (10) of different types and a plurality of biometric data (10) labeled with feature information as a learning data set. The machine learning-based first model may be implemented as a linear regression model, a logistic regression model, a support vector machine (SVM) model, a multi-layer perceptron (MLP), a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network, etc.

[0085] Meanwhile, according to one embodiment of the present disclosure, the processor (110) may select a machine learning-based first model corresponding to the types of a plurality of biometric data (10), and input the plurality of biometric data (10) into the selected machine learning-based first model, thereby obtaining feature information corresponding to the plurality of biometric data (10). For example, when the biometric data is electrocardiogram data, the feature information may be heart rate, RR interval, QRS interval, etc. Then, the processor (110) may input the obtained feature information into a rule-based model corresponding to each biometric data. Then, the processor (110) may obtain a prediction result regarding a disease corresponding to each of the plurality of biometric data (10) through the rule-based model.

[0086] Again, referring to FIG. 3, the processor (110) can evaluate a new judgment criterion based on the acquired prediction result and display the evaluation result (S340).

[0087] Specifically, the processor (110) may evaluate the new judgment criterion based on a plurality of prediction results corresponding to a plurality of biometric data (10) obtained from a rule-based model to which the new judgment criterion is applied. For example, the processor (110) may evaluate the new judgment criterion by combining the accuracy of the new judgment criterion and the time required to obtain the prediction results for the plurality of biometric data (10) by reviewing the prediction results for each biometric data. In particular, the processor (110) may also calculate an evaluation score for the new judgment criterion based on the evaluation results.

[0088] FIG. 6 is an exemplary diagram showing prediction results for a set disease obtained based on the new judgment criteria set in FIG. 5 according to one embodiment of the present disclosure.

[0089] In addition, the processor (110) can display the evaluation results on the display (130). At this time, the processor (110) can also display the prediction results on the display (130). Referring again to FIG. 6, the processor (110) can display the prediction results of right ventricular hypertrophy for each electrocardiogram data acquired by applying a new judgment criterion, along with the identification information of each electrocardiogram data.

[0090] According to one embodiment of the present disclosure, the processor (110) may acquire a plurality of label data corresponding to a plurality of biometric data (10) labeled with a set disease. Furthermore, the processor (110) may evaluate the performance of a rule-based model to which a new judgment criterion is applied based on the acquired plurality of label data. Furthermore, the processor (110) may display the evaluation results for the new judgment criterion.

[0091] That is, the processor (110) can obtain a plurality of label data in which the presence or absence of a disease (or the degree of the disease, the possibility of the disease) set for each biometric data is labeled. In addition, the processor (110) can evaluate the new judgment criterion by comparing the prediction results for the plurality of biometric data (10) input into the rule-based model to which the new judgment criterion is applied with the labels assigned to the plurality of label data. In particular, the processor (110) can evaluate the accuracy of the rule-based model to which the new judgment criterion is applied by comparing the difference between the prediction results and the labels. In addition, the processor (110) can display the prediction results and labels obtained from the rule-based model for the same biometric data as the evaluation results, thereby allowing the user to determine whether the new judgment criterion is appropriate and to evaluate the accuracy of the rule-based model to which the new judgment criterion is applied.

[0092] In particular, the processor (110) can evaluate the rule-based model to which the new judgment criteria are applied by calculating the accuracy, sensitivity, specificity, AUROC (Area Under the Receiver Operating Characteristic Curve), PPV (Positive Predictive Value), NPV (Negative Predictive Value), etc. for the rule-based model to which the new judgment criteria are applied using a plurality of label data. In particular, the processor (110) can calculate the evaluation score for the rule-based model to which the new judgment criteria are applied by combining the accuracy, sensitivity, specificity, AUROC, PPV, etc. for the rule-based model to which the new judgment criteria are applied. Referring to FIG. 6, the processor (110) can display the evaluation result for the new judgment criteria based on the prediction result. Meanwhile, since the method for evaluating the rule-based model based on the above-described evaluation items is equally applicable to a known technology, a detailed description thereof will be omitted.

[0093] In this regard, the processor (110) may evaluate the performance of a rule-based model to which preset judgment criteria are applied in response to a disease based on a plurality of label data, compare the evaluation results for new judgment criteria with the evaluation results for preset judgment criteria, and display the comparison results.

[0094] Specifically, the processor (110) can evaluate the judgment criteria (i.e., preset judgment criteria) applied to the rule-based model before new judgment criteria are applied based on a plurality of label data. The processor (110) can apply the preset judgment criteria to the rule-based model and input the plurality of biometric data (10) into the rule-based model, thereby obtaining prediction results regarding diseases for the plurality of biometric data (10). In addition, the processor (110) can use the plurality of label data to calculate accuracy, sensitivity, specificity, AUROC, PPV, NPV, etc. for the rule-based model to which the preset judgment criteria are applied, thereby evaluating the rule-based model to which the preset judgment criteria are applied. In addition, the processor (110) can compare the evaluation result of the rule-based model to which the new judgment criteria are applied (hereinafter, referred to as a first evaluation result) with the judgment result of the rule-based model to which the preset judgment criteria are applied (hereinafter, referred to as a second evaluation result), and display the comparison result on the display (130). In particular, the processor (110) can compare the first and second evaluation results for each evaluation item (accuracy, sensitivity, specificity, AUROC, PPV, etc.) to evaluate which model (i.e., a rule-based model with new judgment criteria applied and a rule-based model with preset judgment criteria applied) is superior for each item, and display the evaluation results. In addition, the processor (110) can also calculate and display first and second evaluation scores corresponding to the first and second evaluation results, respectively.

[0095] Meanwhile, according to one embodiment of the present disclosure, the processor (110) may display a plurality of characteristic information items identifiable from the biometric data. Specifically, when biometric data (or a disease to be predicted through the biometric data) is selected, the processor (110) may display a plurality of characteristic information items extractable from the biometric data. The processor (110) may display a plurality of diseases predictable through the biometric data (or a judgment criterion corresponding to the selected disease) together with characteristic information items extractable from the biometric data. In addition, when at least one characteristic information among the displayed plurality of characteristic information items is selected, the processor (110) may acquire at least one characteristic information selected from the plurality of biometric data (10).

[0096] That is, when at least one feature information item is selected from among the displayed plurality of feature information items according to a user input, the processor (110) can extract the selected feature information from each biometric data. In particular, in order to obtain the selected at least one feature information, when at least one feature information item is selected from among the displayed plurality of feature information items, the processor (110) inputs the plurality of biometric data (10) into a pre-learned machine learning-based model (hereinafter, a second machine learning-based model), thereby obtaining at least one feature information corresponding to each of the plurality of biometric data (10).

[0097] That is, the processor (110) can input a plurality of biometric data (10) into a second model based on machine learning, and can obtain at least one selected feature information corresponding to the plurality of biometric data (10). In addition, the processor (110) can display the obtained prediction results and at least one obtained feature information for each of the plurality of biometric data (10). Through this, the user can monitor not only the evaluation results for the rule-based model to which the new judgment criteria are applied, but also the feature information of each biometric data, and can also set another new, more appropriate judgment criterion with reference to the feature information. Meanwhile, the first model based on machine learning and the second model based on machine learning may be the same model.

[0098] According to one embodiment of the present disclosure, the processor (110) may update history information regarding a plurality of judgment criteria previously applied to a disease with new judgment criteria and evaluation results for the new judgment criteria. That is, the memory (120) may store history information including a plurality of judgment criteria applied to a rule-based model in response to a previously established disease and evaluation results for the plurality of judgment criteria. In this case, the processor (110) may update the history information by adding a new judgment criterion and an evaluation result (e.g., an evaluation score) for the new judgment criterion to the history information.

[0099] Meanwhile, according to one embodiment of the present disclosure, the processor (110) may select an optimal element for each of a plurality of elements included in the judgment criteria corresponding to a disease based on updated history information, and may generate an optimal judgment criterion corresponding to a disease by combining the selected optimal elements. That is, the processor (110) may identify an optimal judgment criterion for predicting a set disease based on updated history information. Specifically, the processor (110) may analyze the elements and evaluation results included in the plurality of judgment criteria applied to the rule-based model to predict the set disease, and may identify a correlation between the changed elements and the evaluation results. In addition, the processor (110) may identify an optimal judgment criterion by calculating an optimal value (or range, criterion, etc.) for a plurality of elements included in the judgment criteria.

[0100] Meanwhile, to determine correlations, the processor (110) may utilize a deep learning-based model trained based on multiple elements and evaluation results. Here, the deep learning-based model may be a model trained to determine the optimal element combination by determining correlations between each element and the evaluation results based on multiple element combinations and the evaluation results for each element combination.

[0101] Meanwhile, according to one embodiment of the present disclosure, the preset judgment criterion may be the judgment criterion with the highest evaluation result among the plurality of judgment criteria applied to the rule-based first model to predict a past disease. That is, when multiple judgment criteria have been applied to a preset disease in the past, the judgment criterion set for the preset disease may be the judgment criterion with the highest evaluation result (e.g., evaluation score) among the plurality of judgment criteria. At this time, the processor (110) may determine whether to change the preset evaluation criterion in response to the disease based on the evaluation result of the new judgment criterion. Specifically, when the evaluation score of the new judgment criterion is higher than the evaluation scores of the plurality of judgment criteria applied to the rule-based model for the past preset disease, the processor (110) may set the new judgment criterion as the judgment criterion applied to the rule-based model for the preset disease. Accordingly, when the user subsequently selects the preset disease, the processor (110) may display the new judgment criterion as the preset judgment criterion in response to the preset disease.

[0102] FIG. 7 is a block diagram of a computing device (700) according to another embodiment of the present disclosure.

[0103] Referring to FIG. 7, a computing device (700) according to an embodiment of the present disclosure includes a processor (710), a memory (720), a display (730), a sensing unit (740), a communication interface (750), a user interface (760), a camera (770), and a speaker (780). Among the configurations illustrated in FIG. 7, the processor (710), the memory (720), and the display (730) correspond to the configurations of the processor (110), the memory (120), and the display (130) of the computing device (100) illustrated in FIG. 2, and thus a detailed description thereof will be omitted.

[0104] The sensing unit (740) can obtain biometric information about the user. For example, the sensing unit (740) may include a plurality of electrodes. In this case, the processor (710) can obtain electrocardiogram data of the user as biometric information through at least one electrode. In addition, the sensing unit (740) may include an image sensor or a light sensor. In this case, the processor (710) can obtain a photo-blood flow signal of the user as biometric information through the image sensor (or light sensor). In addition, the sensing unit (740) may further include a blood pressure monitor, a thermometer, etc.

[0105] A communication interface (750) according to an embodiment of the present disclosure may be understood as a component that transmits and receives data via any known wired or wireless communication system. For example, the communication interface (750) may perform data transmission and reception using a wired or wireless communication system such as a local area network (LAN), wideband code division multiple access (WCDMA), long term evolution (LTE), wireless broadband internet (WiBro), fifth generation mobile communication (5G), ultrawide-band, ZigBee, radio frequency (RF) communication, wireless LAN, wireless fidelity, near field communication (NFC), or Bluetooth. Since the above-described communication systems are only examples, the wired and wireless communication system for data transmission and reception of the communication interface (750) may be applied in various ways other than the above-described examples.

[0106] The communication interface (750) can receive data necessary for the processor (710) to perform calculations through wired or wireless communication with any system or any client, etc. In addition, the communication interface (750) can transmit data generated through calculations of the processor (710) through wired or wireless communication with any system or any client, etc. For example, the communication interface (750) can receive medical data through communication with a cloud server that performs tasks such as databases in a hospital environment, standardization of medical data, or a computing device, etc. The communication interface (750) can transmit output data of a neural network model, intermediate data derived from the calculation process of the processor (710), processed data, etc. through communication with the aforementioned database, server, or computing device, etc. The processor (710) can obtain a plurality of biometric data (10) through the communication interface (750).

[0107] The user interface (760) is a component used by the computing device (100) to perform interaction with the user, and may include at least one of a touch sensor, a motion sensor, a button, a jog dial, and a switch, but is not limited thereto. The processor (110) may receive user input for selecting biometric data and diseases through the user interface (760), or may receive user input for changing a plurality of elements included in the judgment criteria through the user interface (760).

[0108] The camera (770) captures an image of the user and obtains an image of the user. Specifically, the camera (770) can capture biometric data of the user by capturing the user. To this end, the camera (770) may be implemented with an imaging device such as a CMOS image sensor (CIS) having a CMOS structure, a charge coupled device (CCD) having a CCD structure, etc. However, the present invention is not limited thereto, and the camera (770) may be implemented with a camera module of various resolutions capable of capturing an object. Meanwhile, the camera (770) may be implemented with a depth camera (e.g., an IR depth camera), a stereo camera, an RGB camera, etc.

[0109] The speaker (780) is a component that outputs various audio data that has undergone various processing operations, such as decoding, amplification, and noise filtering, by an audio processing unit (not shown). The speaker (780) can output various notification sounds or voice messages. According to one embodiment of the present disclosure, the processor (710) can convert an electrical signal received from an external device into a user's voice and output it through the speaker (780).

[0110] The various embodiments of the present disclosure described above can be combined with additional embodiments and modified within the scope understood by those skilled in the art in light of the detailed description above. It should be understood that the embodiments of the present disclosure are illustrative in all respects and not restrictive. For example, each component described as a single component may be implemented in a distributed manner, and likewise, components described as distributed may be implemented in a combined manner. Accordingly, all changes or modifications derived from the meaning, scope, and equivalent concepts of the claims of the present disclosure should be construed as being included within the scope of the present disclosure.

Claims

1. A method for evaluating the performance of a judgment criterion set for analyzing biometric data, performed by a computing device including at least one processor, A step of setting a disease to be predicted based on biometric data and identifying a judgment criterion for the biometric data set in response to the disease; A step of setting a new judgment criterion corresponding to the disease by adjusting at least one of the plurality of factors included in the judgment criteria; A step of applying the above new judgment criteria to an artificial intelligence-based first model to obtain prediction results regarding the disease corresponding to each of a plurality of bio-data; and A step of evaluating the new judgment criteria based on the above prediction results and displaying the evaluation results; method.

2. In paragraph 1, the plurality of elements are: The first model based on the artificial intelligence corresponds to a plurality of feature information identified from the biometric data for prediction of the disease, method.

3. In paragraph 1, A step of displaying a list of multiple diseases that can be predicted based on the above biometric data; When any one of the above multiple disease lists is selected, a step of displaying a plurality of elements included in the judgment criteria for the biometric data set corresponding to the selected disease; and A step of displaying a UI (User Interface) for modifying the above plurality of elements; including more, method.

4. In paragraph 2, A step of displaying a plurality of identifiable characteristic information from the above biometric data; and When at least one feature information among the plurality of feature information displayed above is selected, a step of obtaining the selected at least one feature information from the plurality of biometric data is included; The step of displaying the above evaluation results is: A step of displaying the prediction results obtained for each of the plurality of biometric data and at least one feature information obtained; method.

5. In paragraph 4, The step of acquiring at least one selected feature information comprises: When at least one feature information among the plurality of feature information displayed above is selected, a step of inputting the plurality of biometric data into a second model based on pre-learned machine learning to obtain the at least one feature information corresponding to each of the plurality of biometric data is included. method.

6. In paragraph 2, The step of obtaining the above prediction result is: A step of inputting the plurality of biometric data into a third model based on pre-learned machine learning, thereby obtaining the plurality of feature information corresponding to the plurality of elements for the plurality of biometric data; and A step of inputting the acquired plurality of characteristic information into a rule-based first model to which the new judgment criteria are applied, thereby obtaining a prediction result regarding the disease corresponding to each of the plurality of bio-data; including; method.

7. In paragraph 1, The steps indicated above are: A step of obtaining a plurality of label data corresponding to the plurality of biometric data to which labels relating to the disease are assigned; A step of evaluating the performance of a rule-based first model to which the new judgment criterion is applied based on the acquired plurality of label data; and a step of displaying the evaluation results for the above new judgment criteria; including; method.

8. In paragraph 7, The step of displaying the evaluation results for the above new judgment criteria is: A step of evaluating the performance of a rule-based first model to which preset judgment criteria are applied in response to the disease based on the plurality of label data; and A step of comparing the evaluation results for the new judgment criteria with the evaluation results for the preset judgment criteria and displaying the comparison results; including; method.

9. In paragraph 7, A step of updating the history information on the multiple judgment criteria previously applied to the disease with the new judgment criterion and the evaluation result for the new judgment criterion; further comprising; method.

10. In paragraph 9, A step of selecting the optimal element for each of the multiple elements included in the judgment criteria corresponding to the disease based on the updated history information; and A step of generating an optimal judgment criterion corresponding to the disease by combining the above-mentioned optimal elements; further comprising; method.

11. In paragraph 1, The above-mentioned criteria for judgment are: Among the multiple judgment criteria applied to the first artificial intelligence-based model to predict the aforementioned disease in the past, the judgment criterion with the highest evaluation result is A step of determining whether to change the preset evaluation criteria in response to the disease based on the evaluation result; further comprising; method.

12. A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs an operation of evaluating the performance of a judgment criterion set for analyzing biometric data. An operation of setting a disease to be predicted based on biometric data and identifying a judgment criterion for the biometric data set in response to the disease; An action of setting a new judgment criterion corresponding to the disease by adjusting at least one of the plurality of elements included in the judgment criterion; An operation of applying the above new judgment criteria to an artificial intelligence-based first model to obtain prediction results regarding the disease corresponding to each of a plurality of biometric data; and An operation of evaluating the new judgment criterion based on the above prediction result and displaying the evaluation result; including; Computer program.

13. A computing device that evaluates the performance of judgment criteria set for analyzing biometric data. A processor comprising at least one core; A memory containing program codes executable by the processor; and including a display; The above processor, A method for predicting a disease based on biometric data, comprising: setting a disease to be predicted based on biometric data; identifying a judgment criterion for the biometric data set in correspondence with the disease; adjusting at least one of a plurality of elements included in the judgment criterion to set a new judgment criterion corresponding to the disease; applying the new judgment criterion to an artificial intelligence-based first model to obtain a prediction result for the disease corresponding to each of a plurality of biometric data; evaluating the new judgment criterion based on the prediction result; and displaying the evaluation result. Computing device.

Images