Method for monitoring the status of thyroid eye disease and system for carrying out the same

Abstract

A method for monitoring the entire treatment of thyroid eye disease is proposed, and the method includes: a step of requesting a user device to take a facial image and input a questionnaire survey content according to a treatment monitoring cycle during a treatment period in which a therapeutic drug proven to be effective in the treatment of thyroid eye disease through clinical trials is prescribed and administered to the user; a step of acquiring the facial image and the questionnaire survey content from the user device; a step of acquiring personalized estimates of the user corresponding to exophthalmos information, CAS (Clinical Activity Score) information, and diplopia information among the indicators proven in the approval stage of the therapeutic drug using the facial image and the questionnaire survey content acquired from the user device; and a step of storing the personalized estimates of the user in the user device in chronological order from the time the user started administering the therapeutic drug. the displaying step including visualizing on the user device by comparing the user's personalized estimate corresponding to the exophthalmos information with the exophthalmos information at the end of the administration of the therapeutic agent; and the displaying step including visualizing on the user device by comparing the user's personalized estimate corresponding to the exophthalmos information with the exophthalmos information at the end of the administration of the therapeutic agent; and the displaying step including visualizing on the user device a message suggesting a visit to a hospital based on a difference between the user's personalized estimate corresponding to the exophthalmos information and the exophthalmos information at the end of the administration of the therapeutic agent.

Description

[Technical Field]

[0001] The present disclosure relates to a method for monitoring the status of thyroid eye disease through the administration of medication to alleviate exophthalmos, and a system for carrying out the same. [Background technology]

[0002] Although typical thyroid eye disease can be caused by hyperthyroidism, thyroid eye disease can also be caused by other factors.

[0003] Thyroid eye disease can cause exophthalmos, which can result in cosmetic or physical discomfort for the patient.

[0004] A common method for treating exophthalmos is physical surgery, which is performed by extracting the eyeball, removing some of the bone within the eye, and using a method to reposition the eyeball.

[0005] Although patients with thyroid eye disease can alleviate their exophthalmos through physical surgery, patients usually have a reluctance to undergo physical surgery on their eyes. Therefore, there is a need to develop a therapeutic drug to alleviate thyroid eye disease by using drugs instead of physical surgery.

[0006] Therefore, clinical trials for therapeutic agents aimed at treating thyroid eye disease are being conducted, and recently the U.S. Food and Drug Administration (FDA) granted the first regulatory approval for a therapeutic agent aimed at treating thyroid eye disease, specifically, Horizon Therapeutics' TEPEZZA.

[0007] However, in the case of therapeutic drugs for the treatment of thyroid eye disease, these drugs are expensive and require multiple periodic administrations over a predetermined period rather than a single administration, and patients receiving treatment are therefore interested in how their condition, such as exophthalmos, changes as a result of the administration of the drug.

[0008] However, the patient's own exophthalmos condition can only be measured by medical staff when the patient visits the hospital for treatment, so there is a limitation in that the patient can only confirm the limited condition information provided.

[0009] Therefore, there is a need for a method for monitoring the efficacy process of administering therapeutic drugs for the treatment of thyroid eye disease without a clinic visit. Summary of the Invention [Problem to be solved by the invention]

[0010] In the problem solved by the contents disclosed in the present application, the purpose of the present disclosure to solve the problem is to provide a method for monitoring thyroid eye disease treatment that obtains, sequentially arranges, and displays personalized estimates for a patient corresponding to exophthalmos information, CAS information, and diplopia information, as evidenced through the effects of a therapeutic drug while the patient is administered the therapeutic drug intended to treat thyroid eye disease.

[0011] Another object of the present disclosure to solve the problem solved by the contents disclosed in the present application is to provide a method for monitoring thyroid eye disease treatment, which suggests a hospital visit to a patient after the administration period of a therapeutic drug intended to treat the patient's thyroid eye disease has ended by comparing the patient's personalized estimate corresponding to the degree of exophthalmos with the degree of exophthalmos at the time the administration period of the therapeutic drug has ended.

[0012] In solving the problem solved by the contents disclosed in the present application, yet another object of the present disclosure is to provide a method for monitoring the effectiveness of a therapeutic drug, which involves acquiring facial images at two different time points while a patient is being administered a therapeutic drug for treating thyroid eye disease, and determining the trend of change in exophthalmos between the two time points based on a comparison of the facial images, thereby determining whether the therapeutic drug is effective.

[0013] The problems solved in the present application are not limited to those mentioned above, and problems not mentioned will be clearly understood by those skilled in the art to which the technology disclosed in the present application pertains from the accompanying drawings. [Means for solving the problem]

[0014] According to an exemplary embodiment of the present disclosure, a method for monitoring the entire treatment of thyroid eye disease includes: a step of requesting a user device to take a facial image and input a questionnaire content according to a treatment monitoring cycle during a treatment period in which a user is prescribed and administered a therapeutic drug proven to be effective in the treatment of thyroid eye disease through clinical trials; a step of acquiring the facial image and the questionnaire content from the user device; a step of acquiring personalized estimates of the user corresponding to exophthalmos information, CAS (Clinical Activity Score) information, and diplopia information among the indicators proven in the approval stage of the therapeutic drug using the facial image and the questionnaire content acquired from the user device; and a step of displaying the personalized estimates of the user in chronological order from the time when the user started administration of the therapeutic drug. the displaying step including visualizing the user's personalized estimate corresponding to the exophthalmos information on the user device by comparing the user's personalized estimate corresponding to the exophthalmos information with the exophthalmos information at the end of the administration of the therapeutic drug; and the displaying step including visualizing the user's personalized estimate corresponding to the exophthalmos information on the user device by comparing the user's personalized estimate corresponding to the exophthalmos information with the exophthalmos information at the end of the administration of the therapeutic drug.

[0015] According to an exemplary embodiment of the present disclosure, the post-treatment monitoring cycle is different from the treatment monitoring cycle.

[0016] According to an exemplary embodiment of the present disclosure, the treatment monitoring cycle and the post-treatment monitoring cycle are determined by the therapeutic agent.

[0017] According to an exemplary embodiment of the present disclosure, the treatment monitoring cycle is determined by considering the administration cycle of the therapeutic drug, and the post-treatment monitoring cycle is determined by considering the time of recurrence of symptoms after administration of the therapeutic drug has ceased.

[0018] According to an exemplary embodiment of the present disclosure, the survey content includes questions regarding side effects based on the results of clinical trials, and the occurrence of side effects is determined by using the survey content obtained from the user device during the treatment period.

[0019] According to an exemplary embodiment of the present disclosure, if the user experiences side effects during treatment, a message is displayed on the user device suggesting at least one of calling a hospital and visiting the hospital's internet site.

[0020] According to an exemplary embodiment of the present disclosure, during the treatment period, comparative data obtained based on the results of clinical trials is displayed on the user device in correspondence with the date of acquisition of the user's personalized estimate.

[0021] According to an exemplary embodiment of the present disclosure, the comparative data is obtained by adjusting the normative data contained in the clinical trial results according to the personalized estimates.

[0022] According to an exemplary embodiment of the present disclosure, the user's personalized estimate corresponding to the exophthalmos information includes a numerical value for exophthalmos, the user's personalized estimate corresponding to the CAS information includes a numerical value for CAS, and the user's personalized estimate corresponding to the diplopia information includes a grade for diplopia, and the step of visualizing and displaying the user's personalized estimate in chronological order from the time the user started administering the therapeutic agent includes displaying the numerical value for exophthalmos, the numerical value for CAS, and the grade for diplopia on the user device in at least one of a chronological data table and a chronological data graph.

[0023] According to an exemplary embodiment of the present disclosure, the step of displaying a message suggesting a visit to a hospital based on the difference between the user's personalized estimate corresponding to the exophthalmos information and the exophthalmos information at the end of administration of the therapeutic drug includes the steps of determining whether the difference between the user's personalized estimate corresponding to the exophthalmos information and the exophthalmos numerical value included in the exophthalmos information at the end of administration of the therapeutic drug is greater than 2 mm; and if the difference in the exophthalmos numerical value is equal to or greater than 2 mm, displaying a message suggesting a visit to a hospital on the user device.

[0024] According to an exemplary embodiment of the present disclosure, after the treatment period is completed, according to a post-treatment monitoring cycle, request and acquire input of questionnaire content into the user device; acquire a personalized estimate of the user corresponding to the CAS information by using the facial image and questionnaire content acquired from the user device; determine whether the personalized estimate of the user corresponding to the CAS information is equal to or greater than 3; and if the personalized estimate of the user corresponding to the CAS information is equal to or greater than 3, display a message on the user device suggesting a hospital visit.

[0025] According to an exemplary embodiment of the present disclosure, the step of obtaining a personalized estimate of the user corresponding to the exophthalmos information, CAS (Clinical Activity Score) information, and diplopia information using facial images and questionnaire content obtained from the user device includes: obtaining a personalized estimate of the user corresponding to the exophthalmos information by using the facial images; obtaining a personalized estimate of the user corresponding to the CAS information by using the facial images; and obtaining a personalized estimate of the user corresponding to the diplopia information by using the facial images.

[0026] According to an exemplary embodiment of the present disclosure, an actual numerical measurement of a user's exophthalmos and a facial image corresponding to the actual numerical measurement of the exophthalmos are obtained, wherein a personalized estimate of the user corresponding to the exophthalmos information is obtained by using the facial image obtained from the user device, the facial image corresponding to the actual numerical measurement of the exophthalmos, and the actual measurement of the exophthalmos.

[0027] According to an exemplary embodiment of the present disclosure, the step of acquiring facial images and questionnaire survey contents from the user device during the treatment period includes: providing a shooting guide to the user device; capturing a facial image when the shooting guide is met; displaying the questionnaire survey contents on the user device; and acquiring the results of the questionnaire; wherein the step of acquiring facial images from the user device after the treatment period has ended includes: providing a shooting guide to the user device; and capturing a facial image when the shooting guide is met.

[0028] According to an exemplary embodiment of the present disclosure, the shooting guide is a guide for guiding at least one of the left-right angle of the face, the up-down angle of the face, facial expression, and eye position in the image.

[0029] According to an exemplary embodiment of the present disclosure, the shooting guide includes indicators of whether the left-right angle of the face, the up-down angle of the face, the facial expression, and the eye position in the image are each satisfied, and the step of capturing a facial image when the shooting guide is satisfied includes a step of determining whether the shooting guide is satisfied according to whether the left-right angle of the face, the up-down angle of the face, and the eye position in the image satisfy criteria; when the shooting guide is satisfied, a step of changing the display status of the indicators of whether the left-right angle of the face, the up-down angle of the face, the facial expression, and the eye position in the image are satisfied; and a step of capturing a facial image when the shooting guide is satisfied.

[0030] According to an exemplary embodiment of the present disclosure, a user visits a hospital to receive a therapeutic medication, and a method for monitoring the overall treatment of thyroid eye disease treatment includes obtaining a thyroid dysfunction management history for the user; and providing the thyroid dysfunction management history to a medical staff device when the user visits the hospital.

[0031] A non-transitory computer-readable recording medium storing a computer program for performing a method for monitoring overall thyroid eye disease treatment, according to an exemplary embodiment of the present disclosure.

[0032] According to an exemplary embodiment of the present disclosure, a method for determining the therapeutic effectiveness of a therapeutic drug includes: acquiring a first image representing a user's eye captured according to an imaging guide at a first time while the user is prescribed and administered a therapeutic drug proven effective in treating thyroid eye disease through clinical trials; acquiring a first exophthalmos-related variable based on the first image; acquiring a second image representing the user's eye captured according to an imaging guide at a second time after the first time; acquiring a second exophthalmos-related variable based on the second image; determining a trend regarding the user's exophthalmos by comparing the first exophthalmos-related variable and the second exophthalmos-related variable, wherein the trend is classified as an increase, a decrease, or no change; and determining whether the therapeutic drug is effective based on the trend regarding the exophthalmos.

[0033] According to an exemplary embodiment of the present disclosure, the exophthalmos-related variables include at least one of the Radial MPLD value, the horizontal eye length, and the 3D facial landmark coordinate value.

[0034] According to an exemplary embodiment of the present disclosure, the exophthalmos-related variable is not a numerical value of exophthalmos.

[0035] According to an exemplary embodiment of the present disclosure, determining whether the therapeutic agent is effective further includes displaying a message on the user device suggesting a visit if there is an increasing trend in exophthalmos.

[0036] Solutions to the problems of the present disclosure are not limited to the solutions described above, and solutions not mentioned can be clearly understood from this specification and the accompanying drawings by a person skilled in the art to which the present disclosure pertains. [Effects of the Invention]

[0037] According to exemplary embodiments disclosed herein, a method for monitoring the effectiveness of a therapeutic drug is provided, the method comprising the steps of: acquiring and displaying a patient's personalized estimate corresponding to exophthalmos information, CAS information, and diplopia information as time series data while the patient is being administered a therapeutic drug for the treatment of thyroid eye disease; and suggesting a hospital visit to the patient after the administration of the therapeutic drug is completed by comparing the patient's personalized estimate corresponding to the exophthalmos with the exophthalmos at the end of the administration of the therapeutic drug.

[0038] In the problem solved by the contents disclosed in the present application, a method for determining whether a therapeutic drug is effective may be provided, the method comprising: acquiring facial images at two different time points while a patient is being administered a therapeutic drug for treating thyroid eye disease; and determining a trend in the degree of exophthalmos between the two time points based on a comparison of the acquired facial images, thereby determining whether the therapeutic drug is effective according to whether the trend is increasing, decreasing, or unchanged, without determining the exact numerical value of the degree of exophthalmos.

[0039] The effects of the present disclosure are not limited to the effects described above, and effects not mentioned in this specification can be clearly understood by those skilled in the art to which the present disclosure pertains from this specification and the accompanying drawings. [Brief explanation of the drawings]

[0040] [Figure 1] FIG. 1 illustrates a method for monitoring overall thyroid eye disease treatment, according to an exemplary embodiment.

[0041] [Figure 2] FIG. 1 illustrates a method for determining a clinical activity score for thyroid eye disease activity.

[0042] [Figure 3] FIG. 1 illustrates a method for determining the severity of thyroid eye disease based on the European Group on Graves Orbitopathy (EUGOGO) criteria.

[0043] [Figure 4] FIG. 1 is a diagram illustrating a general treatment process by administering a therapeutic agent.

[0044] [Figure 5] FIG. 1 illustrates a method for monitoring a therapeutic process based on therapeutic drug administration, according to an exemplary embodiment.

[0045] [Figure 6] FIG. 1 illustrates an overall monitoring treatment process based on therapeutic drug administration, according to an exemplary embodiment.

[0046] [Figure 7] FIG. 1 illustrates a user interface (UI) for displaying a personalized estimate for a patient, according to an exemplary embodiment.

[0047] [Figure 8] FIG. 10 illustrates a UI for displaying a personalized estimate for a patient, according to an exemplary embodiment.

[0048] [Figure 9] 10A and 10B are diagrams illustrating a method for estimating exophthalmos from landmarks detected from a front face image.

[0049] [Figure 10] 10A and 10B are diagrams illustrating an eyeball region, a pupil region, and an iris region detected from a front face image.

[0050] [Figure 11] FIG. 2 is a diagram illustrating feature amounts that can be acquired from a face image.

[0051] [Figure 12] FIG. 1 illustrates experimental results from a trained exophthalmos prediction model.

[0052] [Figure 13] FIG. 1 illustrates experimental results for a trained exophthalmos prediction model.

[0053] [Figure 14] 1A-1C illustrate a method for estimating exophthalmos by using profile facial images.

[0054] [Figure 15] 10A and 10B are diagrams illustrating a method for determining a trend in exophthalmos based on a face image.

[0055] [Figure 16] 1 is a graph illustrating the relationship between Radial Multiple Radial Mid-Pupil Lid Distance (Radial MPLD) values and exophthalmos. [Figure 17] 1 is a graph illustrating the relationship between Radial Multiple Radial Mid-Pupil Lid Distance (Radial MPLD) values and exophthalmos.

[0056] [Figure 18] 1 is a graph illustrating the relationship between horizontal eye length and exophthalmos.

[0057] [Figure 19]10A and 10B are diagrams illustrating a method for determining eyelid retraction from a front face image.

[0058] [Figure 20] 1 is a flowchart illustrating a process for capturing and transmitting a facial image, according to an example embodiment.

[0059] [Figure 21] FIG. 10 illustrates adjustment of a patient's facial image according to an exemplary embodiment.

[0060] [Figure 22] FIG. 1 illustrates a method for displaying a facial image of a patient, according to an exemplary embodiment.

[0061] [Figure 23] FIG. 1 illustrates a method for monitoring the effectiveness of a therapeutic drug in a clinical trial, according to an exemplary embodiment.

[0062] [Figure 24] FIG. 1 illustrates an overall method for monitoring the effectiveness of therapeutic drugs in clinical trials, according to an exemplary embodiment.

[0063] [Figure 25] FIG. 1 illustrates a method for post-treatment monitoring, according to an exemplary embodiment.

[0064] [Figure 26] FIG. 10 illustrates a UI for displaying a personalized estimate for a patient, according to an exemplary embodiment.

[0065] [Figure 27] FIG. 1 illustrates a method for monitoring overall therapy, according to an exemplary embodiment.

[0066] [Figure 28]FIG. 1 illustrates a system for monitoring the effectiveness of a therapeutic agent, according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0067] The exemplary embodiments described herein are intended to clearly describe the concept of the present disclosure to those skilled in the art. Therefore, the present disclosure is not limited by the exemplary embodiments, and the scope of the present disclosure should be interpreted as including modifications and variations without departing from the concept of the present disclosure.

[0068] The terms used in this specification are selected from currently widely used general terms in consideration of their function in this disclosure, and may have varying meanings depending on the intentions of those skilled in the art, customs in the art, the emergence of new technologies, or the like. However, in contrast, when a specific term is defined and used in any sense, the meaning of the term will be separately described. Therefore, the terms used in this specification should be interpreted not only based on the name of the term, but also based on the actual meaning and the context of the entire specification.

[0069] The numbers (eg, first, second, etc.) used in the processes described herein are merely identifiers to distinguish one element and / or component from another element and / or component.

[0070] In the exemplary embodiments below, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise.

[0071] In the following exemplary embodiments, terms such as "comprises," "includes," or "has" mean the presence of features or components described in the specification, and do not exclude the possibility that one or more other features or components may be added.

[0072] The accompanying drawings of this specification are intended to easily describe the present disclosure, and shapes shown in the drawings may be exaggerated as necessary to facilitate understanding of the present disclosure. Therefore, the present disclosure is not limited by the drawings.

[0073] When an example embodiment can be implemented differently, the order of specific processes may also be different from that described. For example, two processes described as successive may be performed substantially simultaneously or may be performed in the reverse order from that described.

[0074] When it is determined that a detailed description of well-known components or functions related to the present disclosure may obscure the subject matter of the present disclosure, such detailed description may be omitted in this specification as necessary.

[0075] According to an exemplary embodiment of the present disclosure, a method for monitoring the entire treatment of thyroid eye disease includes: a step of requesting a user device to take a facial image and input a questionnaire content according to a treatment monitoring cycle during a treatment period in which a user is prescribed and administered a therapeutic drug proven to be effective in the treatment of thyroid eye disease through clinical trials; a step of acquiring the facial image and the questionnaire content from the user device; a step of acquiring personalized estimates of the user corresponding to exophthalmos information, CAS (Clinical Activity Score) information, and diplopia information among the indicators proven in the approval stage of the therapeutic drug using the facial image and the questionnaire content acquired from the user device; and a step of displaying the personalized estimates of the user in chronological order from the time when the user started administration of the therapeutic drug. the displaying step including visualizing the user's personalized estimate corresponding to the exophthalmos information on the user device by comparing the user's personalized estimate corresponding to the exophthalmos information with the exophthalmos information at the end of the administration of the therapeutic drug; and the displaying step including visualizing the user's personalized estimate corresponding to the exophthalmos information on the user device by comparing the user's personalized estimate corresponding to the exophthalmos information with the exophthalmos information at the end of the administration of the therapeutic drug.

[0076] According to an exemplary embodiment of the present disclosure, the post-treatment monitoring cycle is different from the treatment monitoring cycle.

[0077] According to an exemplary embodiment of the present disclosure, the treatment monitoring cycle and the post-treatment monitoring cycle are determined by the therapeutic agent.

[0078] According to an exemplary embodiment of the present disclosure, the treatment monitoring cycle is determined by considering the administration cycle of the therapeutic drug, and the post-treatment monitoring cycle is determined by considering the time of recurrence of symptoms after administration of the therapeutic drug has ceased.

[0079] According to an exemplary embodiment of the present disclosure, the survey content includes questions regarding side effects based on the results of clinical trials, and the occurrence of side effects is determined by using the survey content obtained from the user device during the treatment period.

[0080] According to an exemplary embodiment of the present disclosure, if the user experiences side effects during treatment, a message is displayed on the user device suggesting at least one of calling a hospital and visiting the hospital's internet site.

[0081] According to an exemplary embodiment of the present disclosure, during the treatment period, comparative data obtained based on the results of clinical trials is displayed on the user device in correspondence with the date of acquisition of the user's personalized estimate.

[0082] According to an exemplary embodiment of the present disclosure, the comparative data is obtained by adjusting the normative data contained in the clinical trial results according to the personalized estimates.

[0083] According to an exemplary embodiment of the present disclosure, the user's personalized estimate corresponding to the exophthalmos information includes a numerical value for exophthalmos, the user's personalized estimate corresponding to the CAS information includes a numerical value for CAS, and the user's personalized estimate corresponding to the diplopia information includes a grade for diplopia, and the step of visualizing and displaying the user's personalized estimate in chronological order from the time the user started administering the therapeutic agent includes displaying the numerical value for exophthalmos, the numerical value for CAS, and the grade for diplopia on the user device in at least one of a chronological data table and a chronological data graph.

[0084] According to an exemplary embodiment of the present disclosure, the step of displaying a message suggesting a visit to a hospital based on the difference between the user's personalized estimate corresponding to the exophthalmos information and the exophthalmos information at the end of administration of the therapeutic drug includes the steps of determining whether the difference between the user's personalized estimate corresponding to the exophthalmos information and the exophthalmos numerical value included in the exophthalmos information at the end of administration of the therapeutic drug is greater than 2 mm; and if the difference in the exophthalmos numerical value is equal to or greater than 2 mm, displaying a message suggesting a visit to a hospital on the user device.

[0085] According to an exemplary embodiment of the present disclosure, after the treatment period is completed, according to a post-treatment monitoring cycle, request and acquire input of questionnaire content into the user device; acquire a personalized estimate of the user corresponding to the CAS information by using the facial image and questionnaire content acquired from the user device; determine whether the personalized estimate of the user corresponding to the CAS information is equal to or greater than 3; and if the personalized estimate of the user corresponding to the CAS information is equal to or greater than 3, display a message on the user device suggesting a hospital visit.

[0086] According to an exemplary embodiment of the present disclosure, the step of obtaining a personalized estimate of the user corresponding to the exophthalmos information, CAS (Clinical Activity Score) information, and diplopia information using facial images and questionnaire content obtained from the user device includes: obtaining a personalized estimate of the user corresponding to the exophthalmos information by using the facial images; obtaining a personalized estimate of the user corresponding to the CAS information by using the facial images; and obtaining a personalized estimate of the user corresponding to the diplopia information by using the facial images.

[0087] According to an exemplary embodiment of the present disclosure, an actual numerical measurement of a user's exophthalmos and a facial image corresponding to the actual numerical measurement of the exophthalmos are obtained, wherein a personalized estimate of the user corresponding to the exophthalmos information is obtained by using the facial image obtained from the user device, the facial image corresponding to the actual numerical measurement of the exophthalmos, and the actual measurement of the exophthalmos.

[0088] According to an exemplary embodiment of the present disclosure, the step of acquiring facial images and questionnaire survey contents from the user device during the treatment period includes: providing a shooting guide to the user device; capturing a facial image when the shooting guide is met; displaying the questionnaire survey contents on the user device; and acquiring the results of the questionnaire; wherein the step of acquiring facial images from the user device after the treatment period has ended includes: providing a shooting guide to the user device; and capturing a facial image when the shooting guide is met.

[0089] According to an exemplary embodiment of the present disclosure, the shooting guide is a guide for guiding at least one of the left-right angle of the face, the up-down angle of the face, facial expression, and eye position in the image.

[0090] According to an exemplary embodiment of the present disclosure, the shooting guide includes indicators of whether the left-right angle of the face, the up-down angle of the face, the facial expression, and the eye position in the image are each satisfied, and the step of capturing a facial image when the shooting guide is satisfied includes a step of determining whether the shooting guide is satisfied according to whether the left-right angle of the face, the up-down angle of the face, and the eye position in the image satisfy criteria; when the shooting guide is satisfied, a step of changing the display status of the indicators of whether the left-right angle of the face, the up-down angle of the face, the facial expression, and the eye position in the image are satisfied; and a step of capturing a facial image when the shooting guide is satisfied.

[0091] According to an exemplary embodiment of the present disclosure, a user visits a hospital to receive a therapeutic medication, and a method for monitoring the overall treatment of thyroid eye disease treatment includes obtaining a thyroid dysfunction management history for the user; and providing the thyroid dysfunction management history to a medical staff device when the user visits the hospital.

[0092] A non-transitory computer-readable recording medium storing a computer program for performing a method for monitoring overall thyroid eye disease treatment, according to an exemplary embodiment of the present disclosure.

[0093] According to an exemplary embodiment of the present disclosure, a method for determining the therapeutic effectiveness of a therapeutic drug includes: acquiring a first image representing a user's eye captured according to an imaging guide at a first time while the user is prescribed and administered a therapeutic drug proven effective in treating thyroid eye disease through clinical trials; acquiring a first exophthalmos-related variable based on the first image; acquiring a second image representing the user's eye captured according to an imaging guide at a second time after the first time; acquiring a second exophthalmos-related variable based on the second image; determining a trend regarding the user's exophthalmos by comparing the first exophthalmos-related variable and the second exophthalmos-related variable, wherein the trend is classified as an increase, a decrease, or no change; and determining whether the therapeutic drug is effective based on the trend regarding the exophthalmos.

[0094] According to an exemplary embodiment of the present disclosure, the exophthalmos-related variables include at least one of the Radial MPLD value, the horizontal eye length, and the 3D facial landmark coordinate value.

[0095] According to an exemplary embodiment of the present disclosure, the exophthalmos-related variable is not a numerical value of exophthalmos.

[0096] According to an exemplary embodiment of the present disclosure, determining whether the therapeutic agent is effective further includes displaying a message on the user device suggesting a visit if there is an increasing trend in exophthalmos.

[0097] Below, a monitoring method and a monitoring system according to exemplary embodiments are described.

[0098] 1. Monitoring the efficacy of therapeutic drugs

[0099] FIG. 1 is a diagram illustrating a method for monitoring overall thyroid eye disease treatment, according to an exemplary embodiment.

[0100] Referring to FIG. 1, based on a method for monitoring the overall course of thyroid eye disease treatment according to an exemplary embodiment, a patient and / or medical staff may monitor the patient's condition after administration of a therapeutic agent intended to treat thyroid eye disease.

[0101] A therapeutic drug for the treatment of thyroid eye disease may be a therapeutic drug that has been proven effective through clinical trials in reducing exophthalmos, reducing Clinical Activity Score (CAS) scores, and improving diplopia in patients receiving the therapeutic drug. That is, a therapeutic drug may mean a therapeutic drug that has been marketed and prescribed after undergoing clinical trials.

[0102] The patient's condition may include, but is not limited to, the degree of exophthalmos, the CAS score, and whether the patient has diplopia. The patient's condition may include the patient's condition regarding the improved efficacy of the therapeutic agent submitted at the time of approval for the therapeutic agent.

[0103] Referring to FIG. 1, according to an exemplary embodiment, a method for monitoring therapeutic drug effectiveness allows a patient's condition to be monitored during a treatment period 101 during which the therapeutic drug is being administered, and during a post-treatment period 102 after the administration of the therapeutic drug has ceased.

[0104] With reference to FIG. 1, the treatment is illustrated as being administered a total of eight times at three-week intervals, although the dosing schedule is not so limited.

[0105] Referring to Figure 1, at the time of first administration of the therapeutic agent 111, the patient may be in a condition with symptoms of thyroid eye disease 112. In this case, the patient may have high exophthalmos, high CAS scores, and diplopia.

[0106] 1, at the end of administration of the therapeutic agent 121, the patient may be in a state where the symptoms of thyroid eye disease are completely cured 122. In this case, the patient may have reduced exophthalmos, improved CAS scores, and improved diplopia.

[0107] Patients will visit the clinic every three weeks to receive treatment, and at each visit, medical staff will measure exophthalmos and assess CAS scores.

[0108] That is, in the case of general patient monitoring, only data regarding the patient's condition measured by medical staff when the patient visits the hospital can be obtained, and data regarding the patient's condition during periods when the patient does not visit the hospital cannot be obtained.

[0109] However, the patient's condition needs to be checked during clinic visits and after treatment has ended.

[0110] More specifically, because thyroid eye disease is a symptom that can be seen externally, patients are sensitive to changes in their own condition and need to confirm in real time whether a therapeutic drug is effective and the extent of the therapeutic drug's effectiveness.

[0111] Additionally, because the intervals between clinic visits are long, medical staff have a need to see how a patient's condition has changed between their clinic visits.

[0112] According to an exemplary embodiment, a method for monitoring therapeutic drug efficacy can monitor a patient's condition and obtain information about therapeutic drug efficacy (such as exophthalmos, CAS score, and whether the patient has diplopia) even when the patient is not visiting a hospital. Specific details regarding the method for monitoring a patient's condition are described below.

[0113] 1 , patient condition data may be obtained even when the patient does not visit a clinic, thereby obtaining continuous data regarding the patient's condition even when clinic visits are spaced as long as three weeks apart. In this case, the patient condition data may include exophthalmos information, CAS information, and / or diplopia information about the patient. Without being limited thereto, the patient condition data may include information regarding eyelid retraction, information regarding the severity of thyroid eye disease, and / or information regarding side effects, each of which is described in more detail below.

[0114] The patient and / or medical staff can view the patient condition data in real time and, depending on the patient's condition, necessary measures and the like can be taken.

[0115] Additionally, by referring to graph 130 in FIG. 1, it can be seen how the patient's condition changes even when the patient does not visit the clinic during period 102 after the therapeutic drug administration has ended.

[0116] Therefore, as shown in graph 130, even if the patient's condition worsens again during the period 102 after the administration of the therapeutic drug has ended (e.g., exophthalmos increases, CAS values increase, or diplopia occurs), the patient's condition can be monitored so that necessary measures and the like can be taken according to the patient's condition even during the period when the patient is not visiting the hospital regularly due to the end of the patient's treatment.

[0117] For example, depending on the patient's condition, measures may be taken such as instructing the patient to visit a hospital and changing medical staff depending on the patient's condition.

[0118]

[0119] The following describes the diagnostic criteria for thyroid eye disease.

[0120] For thyroid eye disease, there can be two indicators: thyroid eye disease activity and thyroid eye disease severity.

[0121] 2.Activity for thyroid eye disease

[0122] Thyroid eye disease has no obvious warning symptoms, making early diagnosis difficult.

[0123] Therefore, the medical community has been working to enable early diagnosis of thyroid eye disease through a method for assessing the Clinical Activity Score (CAS), which has been proposed since 1989.

[0124] The activity of thyroid eye disease can be determined by a clinical activity score, which can be calculated by considering seven items.

[0125] FIG. 2 illustrates a method for determining a clinical activity score for thyroid eye disease activity.

[0126] Referring to FIG. 2, a total of seven items may be considered for the clinical activity score 250. The total seven items considered include: (1) eyelid redness 221, (2) conjunctival redness 222, (3) eyelid swelling 223, (4) conjunctival swelling 224, (5) caruncle swelling 225, (6) spontaneous retrobulbar pain 231, and (7) pain when trying to look up or down 232.

[0127] Of these items, five symptoms, namely, eyelid redness 221, conjunctival redness 222, eyelid swelling 223, conjunctival swelling 224, and caruncle swelling 225, can be determined from the eye of patient 210. Specifically, the five symptoms, eyelid redness 221, conjunctival redness 222, eyelid swelling 223, conjunctival swelling 224, and caruncle swelling 225, can be determined from the actual eye of patient 210 or an image of the eye of patient 210.

[0128] The two symptoms, spontaneous retrobulbar pain 231 and pain when trying to look up or down 232, can be determined by administering a questionnaire to the patient 210.

[0129] A score for each symptom is calculated according to the presence or absence of each of the seven symptoms, with 1 point if the symptom is present and 0 point if the symptom is absent, and the calculated scores are then summed to calculate a total score 240. The maximum score for the clinical activity score 250 is 7 points, and if the total score 240 is 3 points or more, it can be determined that thyroid eye disease activity is present.

[0130] The presence or absence of the five conditions 221, 222, 223, 224, and 225 that can be determined from the eyes of the patient 210 can be determined by medical staff examining the patient's eyes and assessing the conditions. Alternatively, the presence or absence of the five conditions 221, 222, 223, 224, and 225 that can be determined from the eyes of the patient 210 can be determined by assessing the presence or absence of the conditions based on captured images of the face or eyes of the patient 210.

[0131] Meanwhile, five symptoms 221, 222, 223, 224 and 225 that can be determined from the eye of the patient 210 can also be determined by using the trained prediction model.

[0132] Specifically, the presence or absence of each of the five symptoms can be predicted from a patient's facial image by using a model for predicting eyelid redness from facial images, a model for predicting conjunctival redness from facial images, a model for predicting eyelid swelling from facial images, a model for predicting conjunctival swelling from facial images, and a model for predicting caruncle swelling from facial images.

[0133] Without being limited thereto, the presence or absence of each of the five symptoms can also be predicted from the patient's facial image by using a single predictive model that predicts all of eyelid redness, conjunctival redness, eyelid swelling, conjunctival swelling, and eyelid swelling.

[0134]

[0135] 3. Severity of thyroid eye disease

[0136] The severity of thyroid eye disease is an index used to classify and indicate the level of thyroid eye disease in a patient, and various classification criteria can be used.

[0137] For example, the severity of thyroid eye disease can be classified based on the European Group on Graves Orbitopathy (EUGOGO).

[0138] FIG. 3 is a diagram illustrating a method for determining the severity of thyroid eye disease based on the EUGOGO criteria.

[0139] Referring to FIG. 3, the severity of thyroid eye disease 340 may be determined based on assessments including whether exophthalmos 321 is 3 mm or more higher than normal for that race and gender; whether eyelid retraction 322 has increased 2 mm or more; whether the grade for soft tissue involvement 323 has increased; whether diplopia 331 is present; and / or scores from a quality of life questionnaire 332 (i.e., Graves' Orbitopathy Quality of Life (Go-QoL) Questionnaire).

[0140] In this case, the severity of thyroid eye disease 340 can be classified as none, mild (moderate), or severe (severe).

[0141] The degree of exophthalmos 321, eyelid retraction 322 and soft tissue involvement 323 can be determined based on the eyes of the patient 310, and whether diplopia 331 is present and a quality of life questionnaire 332 can be determined by administering a questionnaire to the patient 310.

[0142] Specifically, the exophthalmos 321 can be measured by using an exophthalmosmeter, whereas the exophthalmos can be determined or estimated by using facial images, as will be described in more detail below.

[0143] The eyelid retraction 322 can be determined or estimated by using a facial image, the specific details of which will be described later in 7. Eyelid retraction determination method based on facial image.

[0144] Soft tissue involvement 323 may be determined by considering whether one of four symptoms has an increased rating, which may include eyelid swelling, eyelid redness, conjunctival redness, and conjunctival swelling.

[0145] The grades for the four symptoms, eyelid swelling, eyelid redness, conjunctival redness, and conjunctival swelling, may be determined from the eye of the patient 310. Specifically, the four symptoms may be determined from the actual eye of the patient 310 or an image of the eye of the patient 310.

[0146] The predictive model trained to predict the grade for each of the four symptoms can be used to determine the grade for each of the four symptoms from an image of the patient's 310 eye.

[0147] The presence of diplopia 331 can be determined by using the Gorman criteria, which can grade diplopia as one of no diplopia, intermittent diplopia, diplopia at extreme gaze, and persistent diplopia.

[0148] On the other hand, the severity of thyroid eye disease can also be classified based on NOSPECS and / or VISA.

[0149] Below, therapeutic process monitoring that can be performed during the therapeutic drug administration process is described.

[0150]

[0151] 4.Treatment process monitoring

[0152] (1) General monitoring methods

[0153] FIG. 4 is a diagram illustrating a general treatment process by administering a therapeutic drug.

[0154] Referring to FIG. 4, a patient 402 may be administered a medication when visiting a hospital 401 .

[0155] The therapeutic agent may be a therapeutic agent intended to treat thyroid eye disease and may require multiple administrations during the treatment period. Figure 4 illustrates a therapeutic agent requiring a total of eight administrations at three-week intervals, but the administration schedule is not limited thereto.

[0156] For example, the therapeutic agent may be administered at weekly intervals for the first five doses, then at monthly intervals thereafter. In another example, the therapeutic agent may require two doses at three-week intervals. In yet another example, the therapeutic agent may require daily oral administration, including but not limited to the examples described above.

[0157] Hospital 401 may refer to a hospital where medical staff are assigned and / or work. Hereinafter, actions performed by a hospital may be understood to be performed by medical staff assigned to the hospital, a hospital server, a medical staff server, and / or a medical staff device, and redundant descriptions will be omitted below.

[0158] Patient 402 may be a patient with thyroid eye disease and may be a patient who is prescribed and administered a therapeutic drug for the treatment of thyroid eye disease.

[0159] Referring to FIG. 4, a patient 402 may receive a first therapeutic medication 411 during a first visit 410 to a hospital 401 , where the hospital 401 may obtain measured patient data 412 from the patient 402 .

[0160] Thereafter, the patient 402 may receive a second therapeutic medication 421 during a second visit 420 to the hospital 401 , in which case the hospital 401 may obtain actual patient data 422 from the patient 402 .

[0161] In the same manner, similar procedures may be performed for patient 402 during the third, fourth, fifth, sixth, and seventh visits to hospital 401 .

[0162] The patient 402 may receive medication at the hospital 401 until an eighth visit 480 to the hospital, and at each visit the hospital 401 may obtain measured patient data from the patient 402 .

[0163] After the eighth administration of the therapeutic agent, the patient 402 may make a final visit 490 to the hospital 401 , at which time the hospital 401 may also obtain measured patient data 492 from the patient 402 .

[0164] Finally, according to the existing method, hospital 401 can obtain a total of nine patient data over a 24-week period, but because the intervals between the time points at which the patient data is obtained are long, the continuity of the data is weak, and hospital 401 and patient 402 can only obtain a limited number of patient data.

[0165]

[0166] (2) Monitoring method according to the present disclosure

[0167] FIG. 5 is a diagram illustrating a method for monitoring a therapeutic process based on therapeutic drug administration, according to an exemplary embodiment.

[0168] 5, a patient 502 may be administered a therapeutic drug when visiting a hospital 501. Specifically, when the patient 502 visits the hospital 501, medical staff assigned to the hospital 501 may administer the therapeutic drug to the patient 502. Hereinafter, the process of administering a therapeutic drug to a patient when visiting a hospital may be understood to be performed by medical staff assigned to the hospital.

[0169] The therapeutic agent may be one intended for the treatment of thyroid eye disease and may be one that requires multiple administrations during the treatment period.

[0170] Hospital 501 may refer to a hospital where medical staff are assigned, while actions performed by hospital 501 may be understood to be performed by medical staff assigned to the hospital, a hospital server, a medical staff server, and / or a medical staff device, and redundant descriptions will be omitted below.

[0171] The patient 502 may be a patient with thyroid eye disease, and may be a patient who is prescribed and administered a therapeutic drug for the treatment of thyroid eye disease. Meanwhile, hereinafter, actions performed by the patient 502 may be understood to be performed by the patient and / or the patient's user device, and redundant descriptions will be omitted below.

[0172] 5 , a patient 502 may be provided with a first therapeutic drug administration 511 during a first visit 510 to a hospital 501, and the hospital 501 may acquire actual patient data 512 from the patient 502. Specifically, when the patient 502 makes the first visit 510 to the hospital 501, medical staff assigned to the hospital 501 may administer the first therapeutic drug to the patient 502, and the medical staff assigned to the hospital 501 may perform acquiring 512 actual patient data from the patient 502. Below, the process of the hospital acquiring patient data when the patient visits the hospital may be understood to be performed by the medical staff assigned to the hospital.

[0173] In addition, the hospital 501 may execute sending 513 of the acquired patient data to the analysis server 503. Specifically, the hospital server, a medical staff server and / or a medical staff device installed in the hospital 501 may execute sending 513 of the patient data to the analysis server 503. Hereinafter, the process of the hospital sending data to the analysis server may be understood to be executed by the hospital server, a medical staff server and / or a medical staff device installed in the hospital.

[0174] The analysis server 503 may be a device that transmits and receives data to and from the hospital 501 and the patient 502, obtains patient data about the patient 502, determines and / or estimates the patient's condition, and provides the determined and / or estimated patient data to the hospital 501 and / or the patient 502. Specifically, the analysis server 503 may transmit and receive data to and from a hospital server, a medical staff server, and / or a medical staff device located in the hospital 501, and to and from a user device of the patient 502. In addition, the analysis server 503 may transmit the determined and / or estimated patient data to the hospital server, a medical staff server, and / or a medical staff device located in the hospital 501, and transmit the determined and / or estimated patient data to the user device of the patient 502.

[0175] Additionally, the analysis server 503 may be a device that stores patient data and / or information related to thyroid eye disease obtained from the hospital 501 and / or the patient 502, and provides the stored information to the hospital 501 and / or the patient 502. Specifically, the analysis server 503 may transmit the stored information to a hospital server, a medical staff server, and / or a medical staff device located in the hospital 501, and may transmit the stored information to a user device of the patient 502.

[0176] On the other hand, the time point at which the administration of a therapeutic agent begins may be understood as the time point at which treatment begins. Without being limited thereto, the administration of a therapeutic agent may begin after the time point at which treatment begins.

[0177] On the other hand, in the following, the time when the administration of the therapeutic agent has ended may be understood as the time when the treatment has ended. Without being limited thereto, the treatment may also end after the time when the administration of the therapeutic agent has ended.

[0178] Meanwhile, in the following, the time of administration of a therapeutic drug may be understood as the time when a patient visits a hospital and receives the therapeutic drug, but may also mean, without being limited thereto, the time when a patient self-administers or orally takes the therapeutic drug without visiting a hospital.

[0179]

[0180] The patient data transmitted by the hospital 501 to the analysis server 503 may include exophthalmos measurements obtained from the patient 502, an image of the patient's face at the time the exophthalmos was measured, a thyroid dysfunction management history, thyroid eye disease treatment information, patient physical information, and / or patient health information.

[0181] The exophthalmos measurement may be, but is not limited to, an actual exophthalmos value obtained directly from the patient by medical staff, or it may be an exophthalmos value estimated from a patient's facial image captured when the patient visits the hospital.

[0182] The facial image of the patient when measuring exophthalmos may refer to the facial image corresponding to the exophthalmos measurement value. For example, if the exophthalmos measurement value is an actual exophthalmos value, the facial image of the patient when measuring exophthalmos may refer to the facial image corresponding to the actual exophthalmos value.

[0183] The patient's thyroid dysfunction management history may include, but is not limited to: the diagnosis of the thyroid dysfunction, the time of the thyroid dysfunction diagnosis, blood test results, information about surgeries and / or procedures resulting from the thyroid dysfunction, and the type, dosage, and / or duration of medication used to treat the thyroid dysfunction.

[0184] Thyroid dysfunction may refer to, but is not limited to, hyperthyroidism. Thyroid dysfunction may be understood to include symptoms related to thyroid dysfunction. Specifically, symptoms related to thyroid dysfunction may include, but are not limited to, hypothyroidism, thyroiditis, and / or thyroid nodules, examples of which are described herein.

[0185] Blood test results may include hormone and antibody levels.

[0186] The hormone levels can be values for hormones related to hyperthyroidism, for example, hormones related to hyperthyroidism can include, but are not limited to, free T4 (thyroxine), thyroid-stimulating hormone (TSH), free T3 (triiodothyronine), and / or total T3 (triiodothyronine).

[0187] The antibody level can be a value for antibodies related to hyperthyroidism. For example, antibodies related to hyperthyroidism can include, but are not limited to, anti-TSH receptor Ab, anti-TPO Ab, and / or anti-Tg Ab.

[0188] In addition, blood test results may also include thyroglobulin (TG) and thyroxine-binding globulin (TBG) levels.

[0189] The thyroid eye disease treatment information may include, but is not limited to, the type, dosage, and duration of treatment administered to treat the thyroid eye disease, the date of steroid prescription; the amount of steroid prescription, the date of radiation therapy, the date of thyroid eye disease surgery, and / or the date of triamcinolone administration.

[0190] Patient physical and health information may include, but is not limited to, information based on the patient's physical characteristics, such as the patient's age, sex, race, and weight.

[0191] 5 shows that the hospital 501 transmits 513 the patient data to the analysis server 503 after administering 511 the therapeutic drug to the patient 502, but is not limited thereto, and the hospital 501 may also transmit the patient data to the analysis server 503 before administering the therapeutic drug to the patient 502. For example, the hospital may have patient data for a patient with a thyroid dysfunction and may transmit the patient data to the analysis server before and / or after the patient is prescribed a therapeutic drug for treating thyroid eye disease.

[0192]

[0193] Referring to FIG. 5, patient monitoring may be performed at each first monitoring time point 520 .

[0194] According to an exemplary embodiment, the first monitoring time point 520 may be a time point during a period when the patient 502 visits the hospital 501 to receive a therapeutic medication. Specifically, the first monitoring time point 520 may be any time point during a treatment period when the patient 502 visits the hospital 501 to receive a therapeutic medication that requires multiple doses.

[0195] On the other hand, the treatment period may also include the period during which the patient 502 visits the hospital 501 to monitor his / her condition and the like, even after receiving all the therapeutic drugs that require multiple doses.

[0196] According to an exemplary embodiment, the first monitoring time point 520 may be determined based on an established treatment monitoring cycle.

[0197] In this case, the treatment monitoring cycle may be set based on factors such as the characteristics of the therapeutic drug, the patient's condition, and / or the design of the treatment process.

[0198] For example, a treatment monitoring cycle may be set so that monitoring is performed at specific intervals.

[0199] As a specific example, the therapeutic monitoring cycle may be set so that monitoring is performed every two days. Alternatively, the therapeutic monitoring cycle may be set so that monitoring is performed once every one to three weeks. In this case, the therapeutic monitoring cycle may be set so that monitoring is performed once every one to one and a half weeks. Preferably, the therapeutic monitoring cycle may be set so that monitoring is performed once a week, but is not limited thereto, and the specific period may be freely determined.

[0200] In another example, since the administration cycle may differ for each therapeutic agent, the treatment monitoring cycle may be set by taking into consideration the administration cycle of each therapeutic agent.

[0201] Specifically, the therapeutic monitoring cycle can be set to perform monitoring a set number of times between therapeutic drug administration times. For example, if the therapeutic drug is administered every three weeks, the therapeutic monitoring cycle can be set to perform monitoring every one week, but is not limited thereto.

[0202] As another example, the time at which a therapeutic agent takes effect may differ for each therapeutic agent, so a therapeutic monitoring cycle may be established by taking into consideration the time at which a therapeutic agent takes effect.

[0203] Specifically, the therapeutic monitoring cycle may be set to perform more frequent monitoring starting from the period when the therapeutic effect begins to appear according to the clinical trial results. Alternatively, the therapeutic monitoring cycle may be set to perform more frequent monitoring during the period when the therapeutic effect is rapidly appearing according to the clinical trial results. For example, if the therapeutic effect is not significant in the early stage of treatment but becomes significant after the sixth week, the therapeutic monitoring cycle may be set to perform more frequent monitoring after the sixth week, but is not limited thereto.

[0204] On the other hand, the treatment monitoring cycle can also be divided into a monitoring cycle set by the hospital and a monitoring cycle desired by the patient.

[0205] Specifically, as described above, the monitoring cycle set by the hospital may be set according to the characteristics of the therapeutic drug, the patient's condition, and / or the design of the treatment process. The monitoring cycle desired by the patient is not limited to the monitoring cycle set by the hospital, but may be freely set according to the patient's needs.

[0206] For example, the monitoring cycle set by the hospital may be set to at least once between therapeutic drug administration times, and the patient-desired monitoring cycle may be set to, but is not limited to, once every two days.

[0207] Alternatively, as described above, the monitoring cycle set by the hospital may be set according to the characteristics of the therapeutic drug, the patient's condition, and / or the design of the treatment process, but monitoring may be performed arbitrarily at any time desired by the patient without separately setting a patient-desired monitoring cycle.

[0208] For example, if the monitoring cycle set by the hospital is set to at least once between the time of therapeutic drug administration, monitoring may be performed at least once between the time of therapeutic drug administration according to the monitoring cycle set by the hospital, but additional monitoring may be performed at any time desired by the patient, including but not limited to.

[0209] However, according to an exemplary embodiment, the first monitoring time point 520 can also be determined as any time point at which the patient 502 desires monitoring, without setting up a separate treatment monitoring cycle.

[0210] In this case, the patient 502 may send a monitoring request to the analysis server 503 at any time, and the analysis server 503 may determine the time when it receives the monitoring request from the patient 502 as the first monitoring time 520, but is not limited to this.

[0211] Meanwhile, details regarding the specific actions taken at the first monitoring time point 520 are described after describing FIG.

[0212] FIG. 6 is a diagram illustrating the overall monitoring treatment process based on therapeutic drug administration, according to an exemplary embodiment.

[0213] 6 , a patient 602 may be administered a therapeutic drug when visiting a hospital 601. Specifically, when the patient 602 visits the hospital 601, medical staff assigned to the hospital 601 may administer the therapeutic drug to the patient 602. Hereinafter, it may be understood that the process of administering a therapeutic drug to a patient when visiting a hospital is performed by medical staff assigned to the hospital.

[0214] The therapeutic agent may be a therapeutic agent intended to treat thyroid eye disease and may require multiple administrations during the treatment period. Figure 6 illustrates monitoring of a therapeutic agent requiring a total of eight administrations at three-week intervals, but the administration schedule is not limited thereto.

[0215] On the other hand, with respect to the hospital 601, patient 602, and analysis server 603 in FIG. 6, the contents of the hospital 501, patient 502, and analysis server 502 described above in FIG. 5 may be applied, and duplicate descriptions will be omitted.

[0216] 6 , a patient 602 may be administered a first therapeutic drug 611 at the time 610 of a first visit to a hospital 601, and the hospital 601 may acquire actual patient data 612 from the patient 602. Specifically, at the time 610 of the patient 602's first visit to the hospital 601, medical staff assigned to the hospital 601 may administer the first therapeutic drug to the patient 602, and the medical staff assigned to the hospital 601 may perform acquisition 612 of actual patient data from the patient 602. Below, the process of a hospital acquiring patient data when a patient visits the hospital may be understood to be performed by medical staff assigned to the hospital.

[0217] In addition, the hospital 601 may perform sending 613 of the acquired patient data to the analysis server 603. Specifically, a hospital server, a medical staff server, and / or a medical staff device installed in the hospital 601 may perform sending 613 of the patient data to the analysis server 603. Hereinafter, the process of sending data to the analysis server by the hospital may be understood as being performed by the hospital server, the medical staff server, and / or the medical staff device installed in the hospital.

[0218] For details regarding the hospital 601 transmitting patient data to the analysis server 603, the details described above in the description of the hospital 501 transmitting patient data to the analysis server 503 513 in FIG. 5 may apply, and duplicate descriptions will be omitted.

[0219] 6, patient monitoring may be performed at a first monitoring time point 614 that is given for the first time after a first hospital visit time point 610, and patient monitoring may be performed at a first monitoring time point 615 that is given for a second time after the first monitoring time point 614. With regard to determining the first monitoring time point, the contents described above in the description of determining the first monitoring time point 520 in FIG. 5 may apply, and redundant description will be omitted.

[0220] Referring to FIG. 6, after the first monitoring time point 615 is administered for the second time, the patient 602 may be administered a second therapeutic drug 621 at the time of a second visit 620 to the hospital 601, and the hospital 601 may obtain actual patient data 622 from the patient 602.

[0221] The items of measured patient data acquired at the time of the second hospital visit 620 and the time of the first hospital visit 610 may be identical to one another, but are not limited to such. Some of the data items may be identical, some may be different, or the data items may be different from one another. For example, the measured patient data acquired at the time of the first hospital visit 610 may include exophthalmos measurements acquired from the patient 602, a facial image of the patient 602 at the time of the exophthalmos measurements, a thyroid dysfunction management history, patient physical information, and patient health information. The measured patient data acquired at the time of the second hospital visit 620 may include only exophthalmos measurements acquired from the patient 602 and a facial image of the patient 602 at the time of the exophthalmos measurements. That is, the thyroid dysfunction management history, patient physical information, and patient health information obtained at the time of the first hospital visit 610 can be used as is as the thyroid dysfunction management history, patient physical information, and patient health information of the patient 602 at the time of the second hospital visit 620, and the data items are not limited to the examples described above.

[0222] The hospital 601 may send 623 the acquired patient data to the analysis server 603.

[0223] The patient data transmitted by the hospital 601 to the analysis server 603 at the time of the second hospital visit 620 and the patient data transmitted by the hospital 601 to the analysis server 603 at the time of the first hospital visit 610 may be identical to each other, but are not limited to such. Some of the data items may be identical, some may be different, or the data items may be different from each other. For example, the patient data transmitted by the hospital 601 to the analysis server 603 at the time of the first hospital visit 610 may include exophthalmos measurements obtained from the patient 602, a facial image of the patient 602 at the time of the exophthalmos measurements, a thyroid dysfunction management history, physical information of the patient, and health information of the patient. The patient data transmitted by the hospital 601 to the analysis server 603 at the time of the second hospital visit 620 may include only the exophthalmos measurements obtained from the patient 602 and a facial image of the patient 602 at the time of the exophthalmos measurements. That is, the analysis server 603 may directly use the thyroid dysfunction management history, patient physical information, and patient health information obtained at the time of the first hospital visit 610 as the thyroid dysfunction management history, patient physical information, and patient health information of the patient 602 corresponding to the time of the second hospital visit 620, and the data items are not limited to the examples described above.

[0224] 6 illustrates two first monitoring time points 614 and 615 between the first hospital visit time point 610 and the second hospital visit time point 620, the number of first monitoring time points is not limited thereto, and the first monitoring time points may be arranged at various times based on the set monitoring cycle. For example, there may be one first monitoring time point between the first hospital visit time point 610 and the second hospital visit time point 620, or there may be three or more first monitoring time points.

[0225] Referring to FIG. 6, patient monitoring may be performed at a first monitoring time point 624 given for a third time after the second clinic visit time point 620.

[0226] Thereafter, each first monitoring and hospital visit can be performed on an ongoing basis.

[0227] Referring to FIG. 6, patient 602 may have an eighth therapeutic drug administration 631 at an eighth visit 630 to hospital 601 , and hospital 601 may obtain actual patient data 632 from patient 602 .

[0228] Additionally, the hospital 602 may transmit 633 the acquired patient data to the analysis server 603 .

[0229] Referring to FIG. 6, two sessions 634 and 635 for performing the first monitoring may be performed after the eighth hospital visit point 630, and the number of first monitoring sessions is not limited thereto.

[0230] Referring to FIG. 6, after the eighth therapeutic drug administration, the patient 602 may make a final visit 640 to the hospital 601, in which case the hospital 601 may acquire actual patient data 642 from the patient 602 and transmit 643 the acquired patient data to the analysis server 603.

[0231] During the therapeutic drug administration process, the first monitoring may be performed multiple times throughout the therapeutic drug administration process described in FIG. 6, but is not limited thereto, and the first monitoring may also be performed only at some intervals.

[0232] 6, the time interval between the time points of therapeutic drug administration is 3 weeks, and since the time interval between the time points of therapeutic drug administration is long, the first monitoring is described as being performed multiple times between the time points of therapeutic drug administration, but is not limited thereto. For example, if the therapeutic drug is orally administered daily according to the administration regimen, the first monitoring may be performed once after multiple time points of therapeutic drug administration, or the first monitoring may be performed once between the time points of therapeutic drug administration, but is not limited thereto.

[0233]

[0234] Referring again to FIG. 5, at the first monitoring time point 520, the patient 502 may be asked to capture a facial image and complete a questionnaire.

[0235] In this case, the entity that is requested to capture a facial image and fill out a questionnaire may be the user device of the patient 502. Hereinafter, the process by which the patient is requested to capture a facial image and fill out a questionnaire may be understood to be performed by the user device.

[0236]

[0237] The patient 502 may capture a facial image and transmit the facial image upon a request for facial image capture. For example, the patient 502 may capture a facial image in response to a facial image capture request 521 from the analysis server 503 and transmit 522 the facial image to the analysis server 503. In a more specific example, the patient 502 may capture a facial image in response to the facial image capture request 521 from the analysis server 503 by using a user device and transmit 522 the facial image to the analysis server 503 through the user device. Below, the process of the patient capturing a facial image and transmitting the facial image to the analysis server may be understood to be performed by using the user device.

[0238] In this case, facial image may refer to a frontal and / or profile image of the patient, but is not limited thereto, and may include: a panoramic image of the patient from the front to the side; footage obtained by recording the face; and / or a facial image at any angle between the front and the side of the patient.

[0239] On the other hand, if the patient's face rotates left and right and / or up and down when capturing a facial image, distortion may also occur in the facial image. If distortion exists in the facial image, the accuracy of facial image analysis may decrease, so the patient 502 may be provided with a shooting guide to acquire a preferred facial image. By capturing a facial image following the shooting guide, the patient 402 may capture a facial image with the same composition each time the facial image is captured. Specific details regarding the shooting guide will be described later.

[0240] On the other hand, since facial appearance (eg, swelling, etc.) may change according to the time of day, the patient 502 may receive a facial image capture request at a preset image capture time.

[0241] On the other hand, because facial appearance may change depending on the time of day, the patient 502 may also be required to capture his or her facial images at different times throughout the day. In this case, an analysis result for each captured facial image may be obtained, and the average value of the obtained analysis results or the like may be determined as the correction value for the day. Alternatively, the analysis result with the highest accuracy among the analysis results for each captured facial image may also be determined as the value for the day. Alternatively, the analysis result of the facial image that best satisfies the imaging guide among the captured facial images may also be determined as the value for the day.

[0242] On the other hand, the patient 502 may also receive a facial image capture request to capture multiple facial images at the time of facial image capture. In this case, the analysis result of the facial image that best meets the criteria of the imaging guide among the captured facial images may also be determined as the value at that time. Alternatively, the analysis result of each captured facial image may be obtained, and the average or median of the obtained analysis results may be determined as the correction value at that time. Alternatively, the analysis result with the highest accuracy among the analysis results for each captured facial image may also be determined as the value at that time.

[0243] On the other hand, the patient 502 may receive a facial image capture request to capture a facial image at the time of the facial image capture. In this case, the analysis result of the frame included in the captured facial image that best meets the criteria of the imaging guide may also be determined as the value at that time. Alternatively, the analysis result for each frame included in the captured facial image may be obtained, and the average or median of the obtained analysis results may be determined as the correction value at that time. Alternatively, the analysis result with the highest accuracy among the analysis results for each frame included in the captured facial image may also be determined as the value at that time.

[0244] As described above, when the patient 502 receives a facial image capture request, the impact of changes in the patient's facial appearance, which changes occur depending on the amount of force the patient applies to their face during facial image capture, the patient's condition, the patient's intentions, etc., can be reduced.

[0245]

[0246] The patient 502 may fill out the survey content and send it in response to the survey content request. For example, the patient 502 may fill out the survey content and send it to the analysis server 503 in response to a survey content request 521 from the analysis server 503. In a more specific example, the patient 502 may fill out the survey content by using his or her own user device in response to the survey content request 521 from the analysis server 503, and send the written survey result to the analysis server 503 through the user device. It will be understood below that the process of the patient filling out the survey content and sending it to the analysis server is performed by using the user device.

[0247] When completing the survey, the patient may enter text into the user device and / or check a separate box for pre-defined survey items. In addition to pre-defined survey items, the patient may also enter the survey into the user device, and the survey content may be transmitted by the patient to the analysis server 503 and / or the hospital 501.

[0248] The questionnaires required of the patient 502 may include: a questionnaire regarding determining the activity of thyroid eye disease, and a questionnaire regarding determining the severity of thyroid eye disease.

[0249] For example, questionnaire questions related to determining the activity of thyroid eye disease may include, but are not limited to, whether there is spontaneous pain in the back of the eye and whether there is pain during eye movement.

[0250] For example, questionnaire content relevant to determining the severity of thyroid eye disease may include, but is not limited to, whether diplopia is present and a quality of life questionnaire (Go-QoL).

[0251]

[0252] Based on the facial images and questionnaire content obtained from the patient 502, a personalized estimate of the patient 502 can be obtained regarding information that has been proven to be therapeutically effective through clinical trials of therapeutic drugs intended to treat thyroid eye disease.

[0253] 5 , the analysis server 503 may use facial images and questionnaire content acquired from the patient 502 to determine 523 a personalized estimate for the patient 502 regarding information proven to have therapeutic efficacy through clinical trials of therapeutic drugs intended to treat thyroid eye disease. More specifically, the analysis server 503 may use facial images and questionnaire content received from the patient's user device to determine 523 a personalized estimate for the patient 502 regarding information proven to have therapeutic efficacy through clinical trials of therapeutic drugs intended to treat thyroid eye disease. Hereinafter, the facial images and questionnaire content acquired from the patient may be understood as the facial images and questionnaire content received from the patient's user device.

[0254] The therapeutic effects proven through clinical trials of therapeutic drugs for the treatment of thyroid eye disease may include alleviation of exophthalmos, improvement of CAS values, and improvement of diplopia. Therefore, the information proven to have a therapeutic effect may be exophthalmos information, CAS information, and / or diplopia information.

[0255] On the other hand, personalized estimates for patient 502 regarding indicators proven to be effective in the approval stage of therapeutic drugs for the treatment of thyroid eye disease can be obtained based on facial images and questionnaire survey content obtained from patient 502.

[0256] Specifically, by using facial images and questionnaire survey content obtained from patient 502, analysis server 503 may determine patient 502's personalized estimates for indicators that have been proven to be effective for therapeutic drugs at the approval stage for treating thyroid eye disease.

[0257] In this case, indicators that are proven to be effective in the approval stage of a therapeutic drug for the treatment of thyroid eye disease may include exophthalmos information, CAS information, and diplopia information.

[0258]

[0259] The exophthalmos value indicated in the facial image acquired from the patient 502 may be taken as a personalized estimate of the exophthalmos information for the patient 502. For example, the analysis server 503 may determine the exophthalmos value indicated in the facial image acquired from the patient 502 as the patient's 502 personalized estimate of the exophthalmos information.

[0260] In a more specific example, when the facial image acquired from the patient 502 is a single facial image (for example, when the facial image is captured for the first time), the analysis server 503 may determine the exophthalmos numerical value by using the single facial image. Methods for determining the exophthalmos numerical value by using a single facial image will be described later in (1) Exophthalmos Degree Determination Method-1 Based on Facial Images, (2) Exophthalmos Degree Determination Method-2 Based on Facial Images, and (4) Exophthalmos Degree Determination Method-4 Based on Facial Images in 5. Exophthalmos Degree Determination Method Based on Facial Images.

[0261] When two facial images are obtained from the patient 502 (e.g., one facial image taken previously, where the two facial images may include a facial image at a first time point and a facial image at a second time point after the first time point), the analysis server 503 may determine the exophthalmos value by using the two facial images obtained from the patient 502.

[0262] Specifically, the exophthalmos value at the second time point can be determined by using the facial image at the first time point, the facial image at the second time point, and the exophthalmos value at the first time point. Alternatively, the exophthalmos value at the first time point can be determined by using the facial image at the first time point, the facial image at the second time point, and the exophthalmos value at the second time point. A method for determining the exophthalmos value by using two facial images will be described later in (3) Exophthalmos value determination method based on facial images-3 of 5. Exophthalmos value determination method based on facial images.

[0263] In the case of three facial images obtained from patient 502 (e.g., two previously taken facial images may be included, where the three facial images may include a facial image at a first time point, a facial image at a second time point after the first time point, and a facial image at a third time point after the second time point), analysis server 503 may determine the exophthalmos value at the third time point by using the facial image at the first time point, the facial image at the third time point, and the exophthalmos value at the first time point, and analysis server 503 may determine the exophthalmos value at the third time point by using the facial image at the second time point, the facial image at the third time point, and the exophthalmos value at the second time point. The average value of these two determined exophthalmos values at the third time point may then be determined as the exophthalmos value at the third time point.

[0264] That is, by using the facial image and the exophthalmos numerical value at each time point other than the third time point at which the degree of exophthalmos is determined, and by using the facial image at the third time point at which the degree of exophthalmos is determined, multiple exophthalmos numerical values at the third time point can be determined, and then the average value of the multiple determined exophthalmos numerical values at the third time point can be determined as the exophthalmos numerical value at the third time point.

[0265] Alternatively, the analysis server 503 may also determine the exophthalmos value at the third time point by using a facial image at a second time point closest to the third time point, the exophthalmos value at the second time point, and a facial image at the third time point. A method for determining the exophthalmos value by using two facial images will be described later in (3) Exophthalmos value determination method based on facial images-3 of 5. Exophthalmos value determination method based on facial images.

[0266] If there are four or more facial images acquired from patient 502, analysis server 503 may determine multiple exophthalmos numerical values, as in the case of three facial images described above, by using not only the facial images and exophthalmos numerical values at each time point other than the time point at which the degree of exophthalmos is determined, but also the facial image at the time point at which the degree of exophthalmos is determined, and may thereby determine the average value of the multiple determined exophthalmos numerical values as the exophthalmos numerical value at the time point at which the degree of exophthalmos is determined. A method for determining the exophthalmos numerical value by using two facial images will be described later in (3) Exophthalmos Degree Determination Method Based on Facial Images-3 of 5. Exophthalmos Degree Determination Method Based on Facial Images.

[0267] Alternatively, the analysis server 503 may determine the exophthalmos value at the time the exophthalmos value is determined by using the facial image at the time the exophthalmos value is determined, and by using the facial image and exophthalmos value at the time closest to the time the exophthalmos value is determined. A method for determining the exophthalmos value by using two facial images is described later in 5. Exophthalmos Degree Determination Method Based on Facial Images (3) Exophthalmos Degree Determination Method Based on Facial Images-3.

[0268] Additionally, even if there are multiple facial images obtained from the patient 502, the analysis server 503 may also determine the exophthalmos value by using a method for determining the exophthalmos value by using a single facial image.

[0269] Since the effect of the therapeutic agent is to reduce the degree of exophthalmos, the therapeutic agent can be determined to be effective if the exophthalmos value decreases at a later time point but not at an earlier time point.

[0270] Meanwhile, a trend in exophthalmos may be obtained based on facial images obtained from patient 502 as patient's 502 personalized estimate of exophthalmos information. For example, analysis server 503 may determine a trend in exophthalmos based on facial images obtained from patient 502 as patient's 502 personalized estimate of exophthalmos information.

[0271] Because the effect of the therapeutic agent is to alleviate exophthalmos, the therapeutic agent may be determined to be effective if the exophthalmos is determined to have decreased between a previous time point and a later time point, without even determining the actual exophthalmos numerical value, whereby the trend in exophthalmos may be obtained as a personalized estimate of the exophthalmos information for the patient 502. In this case, the trend in exophthalmos may refer to the trend of change in exophthalmos.

[0272] That is, if it is determined that the trend of exophthalmos based on face images acquired from the patient 502 is decreasing, it can be determined that the therapeutic drug is effective.

[0273] The trend in exophthalmos may be determined based on facial images of the patient 502 acquired at two time points. In this case, the trend in exophthalmos may refer to the trend of change in exophthalmos.

[0274] Specifically, a variable related to exophthalmos may be obtained from each of the facial images of the patient 502 acquired at two time points, and a trend in exophthalmos may be determined based on a comparison of the variable related to exophthalmos. More specific methods for determining a trend in exophthalmos are described below.

[0275] On the other hand, the two time points at which the trend in exophthalmos is determined may be two adjacent time points at which facial images are acquired from the patient 502. For example, by comparing the facial image acquired at the first monitoring time point administered at the nth time with the facial image acquired at the first monitoring time point administered at the (n+1)th time, the trend in exophthalmos between the first monitoring time point administered at the nth time and the first monitoring time point administered at the (n+1)th time can be determined. Based on the determined trend in exophthalmos, it can be determined whether the therapeutic drug is effective.

[0276] Without being limited thereto, the two time points may be freely selected depending on the purpose of determining the trend in exophthalmos. For example, by comparing a facial image acquired at a time point between the time point of administration of the first therapeutic drug and the time point of administration of the second therapeutic drug with a facial image acquired at a time point between the time point of administration of the second therapeutic drug and the time point of administration of the third therapeutic drug, the trend in exophthalmos due to administration of the second therapeutic drug may be determined. The effectiveness of the therapeutic drug may be determined based on the determined trend in exophthalmos, but is not limited thereto.

[0277] On the other hand, the trend in exophthalmos can also be determined by fixing one of the two time points for determining the trend in exophthalmos and allowing the other time point to vary. For example, based on the face image acquired at the first monitoring time point given at the nth time point, the trend in exophthalmos between the (n-1)th time point and the nth time point is determined by comparing it with the face image acquired at the (n-1)th time point, and the trend in exophthalmos between the (n-2)th time point and the nth time point is determined by comparing it with the face image acquired at the (n-2)th time point, thereby enabling trend determination for various exophthalmos, but this is not limited to the examples described above.

[0278]

[0279] The CAS value based on the facial image and questionnaire content obtained from the patient 502 may be obtained as a personalized estimate of the CAS information for the patient 502. For example, the analysis server 503 may determine the CAS value by using the facial image and questionnaire content obtained from the patient 502 as the personalized estimate of the CAS information for the patient 502.

[0280] Specifically, whether or not there is eyelid redness, whether or not there is conjunctival redness, whether or not there is eyelid swelling, whether or not there is conjunctival swelling, and whether or not there is lacrimal caruncle swelling can be determined by using the acquired face images and the trained prediction model. The CAS score can be determined based on the content of whether or not there is spontaneous pain in the posterior part of the eye and whether or not there is pain during eye movement, which is included in the questionnaire content.

[0281] Since the effect of a therapeutic agent is a reduction in CAS values, a therapeutic agent may be determined to be effective if it reduces CAS values at a later time point but not at an earlier time point.

[0282] Meanwhile, a trend in CAS may be obtained based on a facial image and a questionnaire obtained from the patient 502 as the patient's 502 personalized estimate of the CAS information. For example, the analysis server 503 may determine a trend in CAS by using the facial image and the questionnaire obtained from the patient 502 as the patient's 502 personalized estimate of the CAS information.

[0283] Since the effect of a therapeutic drug is a decrease in the CAS value, if the trend of change in CAS between a previous time point and a later time point is determined to be a decreasing trend, the therapeutic drug may be determined to be effective, and thus the trend in CAS may be obtained as a personalized estimate of the CAS information for the patient 502. In this case, the trend in CAS may refer to the trend of change in CAS.

[0284] The trend in CAS can be determined based on facial images and questionnaire survey content of the patient 502 acquired at two time points.

[0285] Specifically, the CAS value is obtained from the facial images and questionnaire survey contents of the patient 502 acquired at two time points, and the trend in the CAS can be determined based on a comparison of the acquired CAS values. The method for determining the CAS value by using the facial images and questionnaire survey contents has been described above, so a duplicate description will be omitted.

[0286] On the other hand, the two time points at which the trend in CAS is determined may be adjacent time points among the time points at which the facial image and the questionnaire survey contents are acquired from the patient 502. The description of the case where the two time points for determining the trend are adjacent time points has been given above in the description of the trend determination for exophthalmos, so duplicated description will be omitted.

[0287] The two time points may be freely selected depending on the purpose of determining the trend in CAS, and are not limited thereto. The description of the case where two time points for determining the trend are freely selected has been given above in the description of the trend determination for exophthalmos, so a duplicate description will be omitted.

[0288] On the other hand, the trend in CAS can also be determined by fixing one of the two time points for determining the trend in CAS and allowing the other time point to vary. The description of the case where one of the two time points for determining the trend is fixed has been given above in the description of the trend determination for exophthalmos, so a duplicated description will be omitted.

[0289]

[0290] The value for determining whether the patient has diplopia may be obtained based on a questionnaire obtained from the patient 502 as the patient's 502 personalized estimate of diplopia information. For example, the analysis server 503 may obtain a value for determining whether the patient has diplopia by using the questionnaire obtained from the patient 502 as the patient's 502 personalized estimate of diplopia information.

[0291] The value for determining whether a patient has diplopia may be a value indicating either the presence or absence of diplopia, whereas when determining whether a patient has diplopia by using the Gorman criteria, the value for determining whether a patient has diplopia may be a value indicating one of the following grades: no diplopia, intermittent diplopia, diplopia at extreme gaze, and persistent diplopia.

[0292] Specifically, a value for determining whether a patient has diplopia can be obtained based on the patient's response regarding the presence of diplopia, which is included in the obtained questionnaire survey content.

[0293] The effect of a therapeutic agent is an improvement in diplopia, so if diplopia is present at an earlier time point and then disappears at a later time point, the therapeutic agent may be determined to be effective.

[0294] Meanwhile, the tendency in diplopia may be obtained based on a questionnaire obtained from the patient 502 as the patient's 502 personalized estimate of diplopia information. For example, the analysis server 502 may determine the tendency in diplopia based on a questionnaire obtained from the patient 502 as the patient's 502 personalized estimate of diplopia information.

[0295] Since the effect of the therapeutic drug is an improvement in diplopia, the therapeutic drug may be determined to be effective if the trend in diplopia is determined to be a decreasing trend between the earlier and later time points, whereby the trend in diplopia may be obtained as a personalized estimate of diplopia information for the patient 502. In this case, the trend in diplopia may mean a trend of change in diplopia.

[0296] The tendency for diplopia can be determined based on questionnaires of the patient 502 taken at two time points.

[0297] Specifically, a value for determining whether a patient has diplopia is obtained from each of the questionnaire survey contents of the patient 502 obtained at two time points, and the tendency in diplopia can be determined based on a comparison of the obtained values for determining whether a patient has diplopia. The method for obtaining a value for determining whether a patient has diplopia by using the questionnaire survey contents has been described above, so a duplicated description will be omitted.

[0298] On the other hand, the two time points at which the tendency in diplopia is determined may be adjacent time points among the time points at which the questionnaire survey contents are obtained from the patient 502. The case where the two time points for determining the tendency are adjacent time points has been described above in the description of the tendency determination for exophthalmos, so duplicated description will be omitted.

[0299] The two time points may be freely selected depending on the purpose of determining the tendency of diplopia, and the two time points may be freely selected. The description of the case where two time points for determining the tendency are freely selected has been given above in the description of the tendency determination for exophthalmos, so a duplicated description will be omitted.

[0300] On the other hand, the tendency for diplopia can also be determined by fixing one of the two time points for determining the tendency for diplopia and allowing the other time point to vary. The description of the case where one of the two time points for determining the tendency is fixed has been given above in the description of the tendency determination for exophthalmos, so a duplicated description will be omitted.

[0301]

[0302] Meanwhile, the patient's 502 personalized estimate of eyelid retraction information may be obtained based on facial images obtained from the patient 502. For example, the analysis server 503 may determine the patient's 502 personalized estimate of eyelid retraction information by using the facial images obtained from the patient 502.

[0303] The numerical value of eyelid retraction indicated in the facial image acquired from the patient 502 may be obtained as the patient's 502 personalized estimate of eyelid retraction information. For example, the analysis server 503 may determine the numerical value of eyelid retraction indicated in the facial image acquired from the patient 502 as the patient's 502 personalized estimate of eyelid retraction information. Specific methods for determining the numerical value of eyelid retraction by using the facial image are described below.

[0304] Meanwhile, a trend in eyelid retraction may be obtained based on facial images obtained from the patient 502 as the patient's 502 personalized estimate of eyelid retraction information. For example, the analysis server 503 may determine a trend in eyelid retraction based on facial images obtained from the patient 502 as the patient's 502 personalized estimate of eyelid retraction information. In this case, the trend in eyelid retraction may refer to a trend of change in eyelid retraction.

[0305] The tendency in eyelid retraction can be determined based on facial images of the patient 502 taken at two time points.

[0306] Specifically, a variable related to eyelid retraction may be obtained from each of the facial images of the patient 502 acquired at two time points, and a trend in eyelid retraction may be determined based on a comparison of the obtained variables related to eyelid retraction. More specific methods for determining a trend in eyelid retraction are described below.

[0307] On the other hand, the two time points at which the tendency for eyelid retraction is determined may be two adjacent time points among the time points at which facial images are acquired from the patient 502. The description of the case where the two time points for determining the tendency are adjacent time points has been given above in the description of the tendency determination for exophthalmos, and therefore, redundant description will be omitted.

[0308] Without being limited thereto, the two time points may be freely selected depending on the purpose of determining the trend in eyelid retraction. The case where two time points for determining the trend are freely selected has been described above in the description of the trend determination for exophthalmos, and therefore, a duplicate description will be omitted.

[0309] On the other hand, the trend in eyelid retraction can also be determined by fixing one of the two time points for determining the trend in eyelid retraction and allowing the other time point to vary. The description of the case where one of the two time points for determining the trend is fixed has been given above in the description of the trend determination for exophthalmos, so a duplicated description will be omitted.

[0310]

[0311] Meanwhile, the patient 502's personalized estimate of thyroid eye disease severity information may be obtained based on facial images and questionnaire survey contents obtained from the patient 502. For example, the analysis server 503 may determine the patient 502's personalized estimate of thyroid eye disease severity information by using facial images and questionnaire survey contents obtained from the patient 502. In this case, the thyroid eye disease severity information may include, but is not limited to, one of none, mild (moderate), and severe (severe) states of thyroid eye disease severity.

[0312] Exophthalmos and eyelid retraction values present in facial images acquired from the patient 502 can be obtained to obtain a personalized estimate of the severity of thyroid eye disease for the patient 502. Additionally, a grade of soft tissue involvement can be determined by using facial images acquired from the patient 502 and the trained predictive model. Additionally, a score for whether the patient has diplopia and a score for quality of life questionnaire content can be obtained based on the questionnaire content acquired from the patient 502.

[0313] A personalized estimate of the patient's 502 severity of thyroid eye disease can be obtained based on the obtained exophthalmos score, eyelid retraction score, grade of soft tissue involvement, score on whether the patient has diplopia, and score on the content of a quality of life questionnaire.

[0314] On the other hand, a trend in the severity of thyroid eye disease may be obtained based on facial images and questionnaire survey content obtained from the patient 502 as a personalized estimate of the severity of thyroid eye disease for the patient 502. For example, the analysis server 503 may determine a trend in the severity of thyroid eye disease based on facial images and questionnaire survey content obtained from the patient 502 as a personalized estimate of the severity of thyroid eye disease for the patient 502. In this case, the trend in the severity of thyroid eye disease may mean a trend of change in thyroid eye disease.

[0315] Trends in the severity of thyroid eye disease can be determined based on facial images and questionnaires of patient 502 taken at two time points.

[0316] Specifically, thyroid eye disease severity information is obtained from facial images and questionnaire survey contents of patient 502 obtained at two points in time, and trends in the severity of thyroid eye disease can be determined based on a comparison using the obtained thyroid eye disease severity information.

[0317] On the other hand, the two time points at which the trend in the severity of thyroid eye disease is determined may be adjacent time points among the time points at which facial images and questionnaire survey contents are acquired from patient 502. The description of the case where the two time points for determining the trend are adjacent time points has been given above in the description of the trend determination for exophthalmos, so duplicated description will be omitted.

[0318] The two time points may be freely selected depending on the purpose of determining the trend in the severity of thyroid eye disease, and the two time points may be freely selected. The description of the case where two time points for determining the trend are freely selected has been given above in the description of the trend determination for exophthalmos, and therefore, a duplicate description will be omitted.

[0319] On the other hand, one of the two time points for determining the trend in the severity of thyroid eye disease can be fixed and the other time point can be allowed to change, so that the trend in the severity of thyroid eye disease can also be determined. The case where one of the two time points for determining the trend is fixed has been described above in the description of the trend determination for exophthalmos, so duplicated description will be omitted.

[0320]

[0321] Meanwhile, information regarding injection site reactions after administration of a therapeutic agent may be obtained based on a questionnaire obtained from the patient 502 .

[0322] To this end, the questionnaire required of the patient 502 may include a questionnaire regarding signs and / or symptoms following an injection site reaction. For example, the questionnaire regarding signs and / or symptoms following an injection site reaction may include, but is not limited to, questions regarding whether blood pressure increases, whether a high fever is present, whether tachycardia is present, whether shortness of breath is present, whether a headache is present, and / or whether muscle pain is present.

[0323] Alternatively, the patient 502 may first be provided with information regarding injection site reactions before being asked to complete a questionnaire about signs and / or symptoms related to injection site reactions.

[0324] Specifically, when a patient 502 visits a hospital 501 and receives a therapeutic drug, the user device of the patient 502 may display information about possible symptoms of an injection site reaction. In this case, the information about the injection site reaction may be information stored on the user device of the patient 502 or information received from the analysis server 503.

[0325] Thus, the patient 502 may first be aware of information regarding possible injection site reactions resulting from the administration of the therapeutic agent before the injection site reaction occurs, so that if an injection site reaction subsequently occurs, the patient 503 may be able to cope with it more easily.

[0326] For injection site reactions following administration of a therapeutic agent, the time to onset can vary for each therapeutic agent. For example, for therapeutic agents intended to treat thyroid eye disease, injection site reactions can occur within 1.5 hours after administration, and such time can vary depending on the results of clinical trials for each therapeutic agent.

[0327] Therefore, the time at which the patient 502 is required to complete the injection site reaction questionnaire may vary depending on the characteristics of each therapeutic agent. For example, the patient 502 may be required to complete the injection site reaction questionnaire 1.5 hours after receiving the therapeutic agent, and the time of the request is not limited to the example described above.

[0328] However, because the time at which an injection site reaction occurs for each patient 502 may vary from patient to patient depending on each patient's characteristics, the patient 502 may also complete a questionnaire about injection site reactions before receiving a separate request to complete a questionnaire. Without being limited thereto, even if the patient 502 does not have an injection site reaction at the time at which they are required to complete a questionnaire about injection site reactions and therefore does not complete the questionnaire, if any injection site reaction occurs at a later time, the patient may still be able to complete the questionnaire separately.

[0329] On the other hand, the time when the injection site reaction occurs may be determined based on a questionnaire obtained from the patient 502. For example, the time when the injection site reaction occurs may be determined based on information about the time when the injection site reaction occurs, the information being included in the questionnaire. Alternatively, the time when the injection site reaction occurs may be determined based on the time when the questionnaire is filled out, but this is not limited to the example described above.

[0330] The time interval between when the injection site reaction occurs and when the therapeutic agent is administered may be stored as data for the patient 502. For example, the analysis server 503 may determine the time interval using the time when the therapeutic agent is administered by determining when the injection site reaction occurs based on a questionnaire obtained from the patient 502, and thereby store the determined time interval as patient data.

[0331] Meanwhile, information about the injection site reaction of the patient 502 and / or information about the time interval between when the therapeutic agent is administered and when the injection site reaction occurs may be provided to the hospital 501. For example, at the time of obtaining the information about the injection site reaction and / or information about the time interval between when the therapeutic agent is administered and when the injection site reaction occurs, the hospital 501 may be provided with the information about the injection site reaction and / or the hospital 501 may be provided with the information about the time interval between when the therapeutic agent is administered and when the injection site reaction occurs. Without being limited thereto, at the time of the patient 502 visiting the hospital 501, the hospital 501 may be provided with the information about the injection site reaction and / or the hospital 501 may be provided with the information about the time interval between when the therapeutic agent is administered and when the injection site reaction occurs.

[0332] Thus, the hospital 501 may more easily respond when the patient 501 experiences an infusion site reaction by administering a therapeutic agent to the patient 501 by taking into account information about the infusion site reaction.

[0333]

[0334] On the other hand, the patient's strabismus information may be obtained based on facial images and / or questionnaires obtained from the patient 502. For this purpose, the questionnaires requested from the patient 502 may include a questionnaire regarding whether strabismus is present.

[0335] Specifically, whether a patient has strabismus can be determined from the acquired facial image. Alternatively, whether a patient has strabismus can be determined based on the content of the patient's responses to strabismus-related questions included in the acquired questionnaire content.

[0336]

[0337] According to an exemplary embodiment, the patient's personalized estimates and patient data may be displayed to the patient 502 or provided to the hospital 501 .

[0338] Specifically, referring again to FIG. 5 , the analysis server 503 may perform display 524 of the determined personalized estimate of the patient 502 to the patient 502 and provide 525 the determined personalized estimate of the patient 502 and the acquired patient data to the hospital 501.

[0339] More specifically, the patient's personalized estimate may be provided to and displayed on the patient's 502 user device, and the patient's personalized estimate and / or patient data may be provided to and displayed on the hospital's 501 medical staff device and / or hospital server. Hereinafter, displaying the patient's personalized estimate and / or patient data to the patient may be understood as displaying on the patient's user device, and providing and displaying the patient's personalized estimate and / or patient data to the hospital may be understood as providing and displaying it on the hospital's medical staff device and / or hospital server.

[0340] 5 illustrates that the patient's personalized estimate is displayed to the patient 502 at the first monitoring time point 520, the time at which the patient's personalized estimate is displayed to the patient 502 is not limited to the first monitoring time point 520. For example, the patient 502 may view the patient's personalized estimate without capturing a facial image or filling out a questionnaire. Specifically, the patient's 502 user device may display the patient's personalized estimate obtained from the analysis server 503 at any time desired by the patient 502.

[0341] Meanwhile, in FIG. 5 , the hospital 501 receives the patient's personalized estimate and patient data at the first monitoring time point 520, but it will be explained that the time point at which the hospital 501 receives the patient's personalized estimate and / or patient data is not limited to the first monitoring time point 520. For example, the hospital 501 can be provided with the patient's personalized estimate and / or patient data even if the patient 502 does not capture a facial image and fill out a questionnaire. Specifically, the medical staff device and / or the hospital server of the hospital 501 can receive the patient's personalized estimate and / or patient data from the analysis server 503 at any time desired by the medical staff. As another example, the hospital 501 can be provided with the patient's personalized estimate and / or patient data when the patient 502 visits the hospital 501. In addition, the hospital 501 can also be provided with the patient's personalized estimate and / or patient data even when the patient 502 does not visit the hospital 501.

[0342] Below, a method for displaying a personalized estimate for a patient is described.

[0343] FIG. 7 is a diagram illustrating a user interface (UI) for displaying a personalized estimate for a patient, according to an exemplary embodiment.

[0344] According to an exemplary embodiment, the obtained personalized estimates for a patient may be displayed as time series data, which may refer to data arranged in a time series.

[0345] 7, for example, the patient's personalized estimates may be displayed by date in a time series data table 710. The time series data table 710 may include, for each date, information regarding the exophthalmos value, the CAS value, and / or whether the patient has diplopia. Additionally, although not shown in FIG. 7, the time series data table may include, for each date, information regarding the eyelid retraction value and / or the severity of thyroid eye disease.

[0346] 7, the patient's personalized estimates are displayed as values by date, but are not limited thereto. The patient's personalized estimates may also be displayed as trends by date. In this case, the trend by date may refer to the trend of change up to that date.

[0347] 7 includes the exophthalmos, CAS values, and values for whether diplopia is present for each date, but the data table may also include trends in exophthalmos, trends in CAS, and / or trends in diplopia for each date. Without limitation, the data table may also include trends in eyelid retraction and / or trends in thyroid eye disease severity for each date.

[0348] More specifically, for the items regarding exophthalmos, these items may be presented as alleviated, worsened, or unchanged, rather than numerically, and may be presented as, but are not limited to, decreased, increased, or unchanged. Additionally, for the items regarding CAS, these items may be presented as alleviated, worsened, or unchanged, rather than numerically, and may be presented as, but are not limited to, decreased, increased, or unchanged. Additionally, for the items regarding diplopia, these items may be presented as alleviated, worsened, or unchanged, rather than present or absent. Additionally, for the items regarding eyelid retraction, these items may be presented as alleviated, worsened, or unchanged, rather than numerically, and may be presented as, but are not limited to, decreased, increased, or unchanged. Additionally, for the items regarding thyroid eye disease severity, these items may be presented as alleviated, worsened, or unchanged, rather than absent, mild (moderate), or severe (severe).

[0349] Meanwhile, as another example, referring to Figure 7, the obtained personalized estimates for a patient may be displayed in a time series data graph 720. A time series data graph may refer to a graph that displays data arranged in a time series.

[0350] The time series data graph 720 may include values of exophthalmos, CAS, and / or whether the patient has diplopia, taken over time. Additionally, although not shown in FIG. 7, the time series data graph may include values of eyelid retraction and / or severity of thyroid eye disease, taken over time.

[0351] On the other hand, in the data graph 720 of FIG. 7, the patient's personalized estimates are displayed as one graph, but are not limited thereto, and the patient's personalized estimates may also be displayed as separate graphs.

[0352] 7, the exophthalmos numerical value, the CAS numerical value, and the value of whether the patient has diplopia obtained over time are displayed as one graph 720, but are not limited to this, and each of a graph of the exophthalmos numerical value over time, a graph of the CAS numerical value over time, and / or a graph of the value of whether the patient has diplopia obtained over time may be displayed. In addition, each of a graph of the eyelid retraction numerical value over time and / or a graph of the value of the severity of thyroid eye disease over time may also be displayed.

[0353] On the other hand, in the data graph 720 in FIG. 7, the patient's personalized estimates are displayed as raw values, but are not limited thereto, and the patient's personalized estimates may also be displayed as smoothed values.

[0354] FIG. 8 is a diagram illustrating a UI for displaying a patient's personalized estimate, according to an exemplary embodiment.

[0355] According to an exemplary embodiment, the obtained personalized estimate for the patient may be displayed along with the comparative data.

[0356] Comparative data can refer to prospective data regarding the expected effect of a therapeutic agent.

[0357] Specifically, comparative data can be obtained based on clinical trial results.

[0358] More specifically, the comparative data may be obtained based on average data of changes in the subject's condition due to the effect of a therapeutic drug included in clinical trial results. In this case, the average data of changes in the subject's condition may refer to standard data.

[0359] More specifically, the comparative data may be obtained by adjusting the standard data according to the patient's personalized estimate. For example, the comparative data may be obtained by calculating a difference between the initial value of the standard data and the initial value of the patient's personalized estimate, and adding the calculated difference to the standard data. Specifically, the comparative data may be obtained by adding the calculated difference to each time series value included in the standard data.

[0360] Comparative data regarding exophthalmos may be obtained based on mean data of subjects' exophthalmos values over time included in clinical trial results. In this case, the mean data of subjects' exophthalmos values over time may refer to standard data regarding exophthalmos.

[0361] Specifically, the comparative data regarding the degree of exophthalmos may be obtained by adjusting the standard data regarding the degree of exophthalmos according to the numerical value of exophthalmos included in the patient's personalized estimate. For example, the comparative data regarding the degree of exophthalmos may be obtained by calculating a difference between the initial value of the standard data regarding the degree of exophthalmos and the initial value of the numerical value of exophthalmos included in the patient's personalized estimate, and adding the calculated difference value to the standard data regarding the degree of exophthalmos. In a more specific example, if the initial value of the standard data regarding the degree of exophthalmos is 17.0 mm and the initial value of the numerical value of exophthalmos included in the patient's personalized estimate is 18.0 mm, the difference value is +1 mm, and therefore the comparative data regarding the degree of exophthalmos may be obtained by adding +1 mm to the standard data regarding the degree of exophthalmos. Specifically, the comparative data regarding the degree of exophthalmos may be obtained by adding the calculated difference value to each of the time-series numerical values of exophthalmos included in the standard data regarding the degree of exophthalmos. However, the specific numerical values are not limited to the examples described above.

[0362] Without being limited thereto, comparative data may be obtained based on mean data of the slope of change in a subject's exophthalmos value over time contained in clinical trial results.

[0363] 8, the patient's obtained personalized estimates may be displayed along with the comparison data in a time series data table 810 separated by date. A time series data table may refer to a table displaying data arranged in chronological order. Specifically, the time series data table may display the patient's personalized estimates and the comparison data such that the dates on which the patient's personalized estimates are obtained correspond in order to the dates in the comparison data.

[0364] Without being limited thereto, the obtained personalized estimates for a patient may be displayed together with the comparative data as a time series data graph 820. A time series data graph may refer to a graph that displays data arranged in chronological order. Specifically, the time series data graph may display the patient's personalized estimates and the comparative data such that the dates of the obtained personalized estimates for the patient correspond in order to the dates in the comparative data, respectively.

[0365] The patient's personalized estimates obtained may be displayed along with comparative data so that it may be determined through the data whether the patient's condition is improving.

[0366] On the other hand, in Figure 8, the exophthalmos value is displayed as the comparative data, but is not limited thereto, and the gradient of change in the exophthalmos value may be displayed as the comparative data. In this case, the patient's personalized estimate may be displayed as the gradient of change in the exophthalmos value.

[0367] 8, only the comparative data regarding the exophthalmos is shown, but the comparative data may include, but is not limited to, the CAS value and / or the comparative data regarding whether the patient has diplopia. The method for obtaining the comparative data regarding the CAS value and / or the comparative data regarding whether the patient has diplopia can be applied in the same manner as the method for obtaining the comparative data regarding the exophthalmos, and therefore, a duplicate description will be omitted.

[0368]

[0369] On the other hand, although not shown in FIG. 8, the obtained personalized estimate for the patient may be displayed together with the mean value of exophthalmos for the group to which the patient belongs.

[0370] The groups to which patients belong may be distinguished by gender and / or age. For example, patients of the same gender may be recognized as belonging to the same group. As another example, patients in the same age range of a pre-set age range may be recognized as belonging to the same group. As yet another example, patients of the same race may be recognized as belonging to the same group. The groups to which patients belong are not limited to the examples described above.

[0371] The patient's personalized estimate obtained may be displayed together with the average exophthalmos value for the group to which the patient belongs, so that the patient and / or medical staff can see how the patient's condition differs from this average value.

[0372]

[0373] According to an exemplary embodiment, whether a therapeutic agent is effective may be determined based on the obtained personalized estimates for the patient.

[0374] Specifically, whether a therapeutic agent is effective may be determined if the patient's personalized estimate is included in the resulting reduction in exophthalmos values, reduction in CAS values, and / or improvement in diplopia.

[0375] Alternatively, whether a therapeutic agent is effective may be determined if the patient's personalized estimate is included in a reduced trend in exophthalmos, a reduced trend in CAS, and / or a reduced trend in diplopia.

[0376] Whether the therapeutic agent is effective can be determined at each of the monitoring time points, i.e., whether the patient's condition is such that the therapeutic agent is effective can be determined at each monitoring time point.

[0377] Without being limited thereto, whether a therapeutic agent is effective can be determined throughout the entire therapeutic agent administration process, i.e., whether a patient's current condition is such that the therapeutic agent is effective can be determined by administering the therapeutic agent.

[0378] The determined results regarding whether the therapeutic agent is effective may be displayed along with the patient's personalized estimate obtained, or, without limitation, the results regarding whether the therapeutic agent is effective may be displayed separately.

[0379]

[0380] According to an exemplary embodiment, a treatment score for a patient may be determined based on the personalized estimate of the patient that is obtained.

[0381] The treatment score is a numerical score that indicates how effective the treatment is for the current patient. The treatment score can range from 0 to 100 points, but the score range is not limited thereto.

[0382] Specifically, a high treatment score may be determined when the patient's personalized estimate includes a decrease in exophthalmos, a decrease in CAS, and / or an improvement in diplopia. More specifically, a high treatment score may be determined when the patient's personalized estimate includes a decrease in exophthalmos, a decrease in CAS, and / or an improvement in diplopia.

[0383] Alternatively, a treatment score may be determined to be high if the patient's personalized estimate obtained includes a reduced trend in exophthalmos, a reduced trend in CAS scores, and / or a reduced trend in diplopia.

[0384] A patient's treatment score can be determined at each of the monitoring time points, i.e., whether the treatment is effective for the patient can be expressed numerically at each monitoring time point.

[0385] Without being limited thereto, a treatment score for a patient may be determined throughout the course of treatment, i.e., whether or not current treatment with treatment is effective for the patient.

[0386] The determined treatment score may be displayed along with the patient's personalized estimate obtained, or, without limitation, the treatment score may be displayed separately.

[0387]

[0388] 5, hospital 501 may also be provided with patient data along with the patient's personalized estimates. Specifically, hospital 501 may also receive patient data 525 along with personalized estimates for patient 502 from analytic server 503.

[0389] The patient data provided to the hospital 501 by the analysis server 503 may include patient data acquired by the hospital 501 as actual patient data when the patient 502 visits the hospital and transmitted to the analysis server 503; injection site reaction information acquired based on questionnaire survey content acquired from the patient 502; the management history of the patient's 502's thyroid dysfunction; thyroid eye disease treatment information of the patient 502; and / or questionnaire survey content acquired from the patient 502. Without being limited thereto, the patient data provided to the hospital 501 by the analysis server 503 may include various types of patient data described above as patient data.

[0390] 5, the hospital 501 receives the patient's personalized estimate and patient data at the first monitoring time point 520, but it will be explained that the time point at which the hospital 501 receives the patient's personalized estimate and / or patient data is not limited to the first monitoring time point 520. For example, the hospital 501 may be provided with the patient's personalized estimate and / or patient data at the time that the patient 502 visits the hospital 501. In addition, the hospital 501 may also be provided with the patient's personalized estimate and / or patient data even when the patient 502 does not visit the hospital 501.

[0391]

[0392] Meanwhile, according to an exemplary embodiment, the acquired facial images of the patient may be displayed to the patient and / or the hospital in the form of comparison data, so that the process of change over time can be easily confirmed, and the method for displaying the facial images is specifically described later in 9. Display of facial image comparison data.

[0393]

[0394] Meanwhile, the patient 502 and / or hospital 501 may be provided with a message and / or additional data based on the personalized estimate of the patient that is obtained.

[0395] Specifically, the patient 502 and / or hospital 501 may be provided with a message and / or additional data if the exophthalmos value included in the patient's obtained personalized estimate increases by 1 mm or more, 2 mm or more, 3 mm or more, 4 mm or more, 5 mm or more, 6 mm or more, 7 mm or more, 8 mm or more, 9 mm or more, 10 mm or more, or 11 mm or more.

[0396] Preferably, if the exophthalmos value included in the patient's personalized estimate obtained increases by 2 mm or more, the patient 502 and / or hospital 501 may be provided with a message and / or additional data.

[0397] On the other hand, if the CAS score included in the patient's personalized estimate obtained is lower than 3 points and then becomes 3 points or higher, the patient 502 and / or hospital 501 may be provided with a message and / or additional data.

[0398] On the other hand, if the CAS number included in the patient's personalized estimate obtained increases by two or more points, the patient 502 and / or hospital 501 may be provided with a message and / or additional data. In the case of a CAS number, depending on the patient's condition, the patient may also temporarily experience eyelid redness, conjunctival redness, eyelid swelling, conjunctival swelling, caruncle swelling, spontaneous retrobulbar pain, or pain when trying to look up or down. Therefore, it may be preferable for the patient 502 and / or hospital 501 to be provided with a message and / or additional data if the CAS number increases by two or more points instead of one or more points.

[0399] However, the specific values of the exophthalmos value and the CAS value are not limited to the examples set forth above.

[0400] With regard to obtaining messages and / or additional data for the patient 502 and / or hospital 501, specifically, the patient 502 may obtain a message suggesting a hospital visit, and the hospital 501 may obtain a warning message regarding the patient's condition.

[0401] More specifically, a message suggesting a hospital visit may be displayed on the patient's 502 user device, and a warning message may be displayed on the hospital's 501 medical staff device and / or medical server, but is not limited to this.

[0402]

[0403] When administering a therapeutic drug, side effects may occur, and in some cases, prompt action may be required. Therefore, whether side effects occur in a patient may also be monitored during treatment process monitoring. Specifically, information about side effects may also be obtained from the patient during treatment process monitoring.

[0404] The side effect-related information obtained from the patient may be determined based on the results of clinical trials of the therapeutic agent. For example, the side effect information may include whether muscle spasms are present, whether nausea is present, whether hair loss is present, whether diarrhea is present, whether fatigue is present, and / or whether hyperglycemia is present, and the side effect information may vary between therapeutic agents.

[0405] On the other hand, information regarding side effects may be obtained based on the questionnaire required of the patient. Specifically, the questionnaire required of the patient may include a questionnaire regarding signs and / or symptoms related to side effects. Because each therapeutic agent may have different side effects, the questionnaire required of the patient may be determined based on the therapeutic agent administered to the patient.

[0406] Alternatively, patients may be provided with information about side effects before being asked to complete a side effect survey.

[0407] Based on the side effect information obtained from the patient, the patient and / or hospital may be provided with a message and / or additional data.

[0408] The message and / or additional data provided to the patient and / or hospital may be determined based on detailed measures according to side effects described in the administration manual for the therapeutic agent.

[0409] For example, if it is determined that the patient has experienced an adverse reaction based on the information regarding the adverse reaction obtained from the patient, the patient may be provided with a message suggesting that the patient call a governmental and / or regulatory agency responsible for regulating the safety of medical products, in which case the message may include a telephone number for the governmental and / or regulatory agency responsible for regulating the safety of medical products.

[0410] Alternatively, the patient may be provided with a message suggesting that they visit the internet site of a governmental and / or regulatory agency that oversees the safety of medical products, in which case the message may include the internet address of the website of the governmental and / or regulatory agency responsible for overseeing the safety of medical products.

[0411] Alternatively, the patient may be provided with a message suggesting that they call a hospital, where the hospital may be one that the patient plans to visit and / or has visited, and the message may include the hospital's phone number.

[0412] Alternatively, the patient may be provided with a message suggesting that they visit a hospital's internet site, where the hospital may be one that the patient plans to visit and / or has visited, and the message may include the internet address of the hospital's website.

[0413] On the other hand, hospitals may be provided with a message to warn patients that a side effect has occurred and may be provided with information about the side effect as additional data.

[0414]

[0415] 5. Method for determining exophthalmos based on facial images

[0416] Exophthalmos is a measure of how protruding the eyeball is and can be determined by the perpendicular distance between the corneal apex and the lateral orbital rim. Because exophthalmos is the perpendicular distance between the corneal apex and the lateral orbital rim, it has previously been difficult to determine exophthalmos using only a single forward face image.

[0417] However, according to an exemplary embodiment of the present disclosure, exophthalmos can be estimated by using a single frontal face image. A specific method for estimating exophthalmos is described below.

[0418] On the other hand, the facial image may be acquired from the patient. Specifically, the facial image may be acquired from the patient's user device, or the facial image may be an image captured by the patient's user device. Without being limited thereto, the facial image may also be acquired from a hospital. Specifically, the facial image may be acquired by allowing medical staff assigned to the hospital to capture the patient's face when the patient visits the hospital.

[0419] The facial image may be an image of the area between the bottom of the nose and the top of the eyebrows, and may refer to, but is not limited to, an image showing the eye area.

[0420]

[0421] (1) Method for determining exophthalmos based on facial images - 1

[0422] To estimate exophthalmos from a facial image, a trained three-dimensional (3D) facial landmark model can first be applied to the facial image.

[0423] In this case, the face image may be a front face image, face images taken from different angles, a panoramic face image, and / or an image including a video capturing a face.

[0424] The 3D facial landmark detection model is a model trained to predict 3D landmark coordinates from a facial image. Specifically, the 3D facial landmark detection model is a model trained to predict not only the x-axis and y-axis coordinates of each landmark, i.e., each feature point in a facial image, but also the z-axis coordinate, and may be a model trained by using facial images and 3D coordinate values corresponding to landmarks shown in the facial images as a training dataset. In this case, the x-axis may refer to the horizontal axis of a two-dimensional (2D) image, the y-axis may refer to the vertical axis of the 2D image, and the z-axis may refer to the axis perpendicular to the x-axis and y-axis of the 2D image.

[0425] On the other hand, the 3D facial landmark detection model according to the present disclosure may be a model configured to detect landmarks located at the boundaries between the eyeballs and eyelids, and landmarks located at the outer edges of the pupils, from a facial image.

[0426] Therefore, by applying the forward face image to the 3D face landmark detection model, it is possible to obtain not only the 3D coordinates of the landmarks located at the boundary between the eyeball and the eyelid, but also the 3D coordinates of the landmarks located at the outer edge of the pupil, which landmarks appear in the forward face image.

[0427] FIG. 9 is a diagram illustrating a method for estimating the degree of exophthalmos from landmarks detected from a front face image.

[0428] As illustrated in Figure 9, landmarks located at the boundaries between the eyes and eyelids can be detected by applying a 3D facial landmark detection model to a face image 910. Figure 9 illustrates a line 920 connecting landmarks located at the boundaries between the eyes and eyelids.

[0429] In addition, landmarks located on the outer edge of the pupil can be detected by applying a 3D facial landmark detection model to a face image 910, as illustrated in Fig. 9. Fig. 9 illustrates a line 930 connecting the landmarks located on the outer edge of the pupil. Since the landmarks located on the outer edge of the pupil are located above, to the left, right, and below the pupil, the line 930 connecting the landmarks located on the outer edge of the pupil is illustrated in the shape of a square.

[0430] Among the landmarks detected from the face image, it is possible to obtain the z-axis length between the landmark 931 located on the outer edge of the pupil and the landmark 921 located at the corner of the outer circumference of the eye. In this case, the z-axis length can be calculated as a pixel distance.

[0431] A distance 940 corresponding to the pupil radius may be calculated from the face image 910. In this case, the distance 940 corresponding to the pupil radius may be calculated as a pixel distance.

[0432] The actual length of the pupil radius is generally similar among people of the same species. Specifically, the actual length of the pupil radius may be 5.735 mm for men and 5.585 mm for women. However, the actual length of the pupil radius may vary depending on the race and / or age of the person.

[0433] Since the actual length of the pupil radius is similar among people of the same type, it is possible to predetermine the actual length of the pupil radius that varies according to gender, race and / or age.

[0434] Therefore, the actual length of the pupil radius corresponding to the face image 910 can be obtained from the actual length of the pupil radius predetermined based on the patient's gender, race, and / or age included in the patient data.

[0435] The actual z-axis distance can be calculated by using the pixel distance corresponding to the pupil radius, the actual distance of the pupil radius, and the z-axis pixel distance.

[0436] Specifically, a resolution of the pixel distance may be determined based on a ratio of a pixel distance corresponding to a pupil radius and an actual distance of the pupil radius, and the actual z-axis distance may be calculated by applying the determined resolution to the z-axis pixel distance.

[0437] On the other hand, since the calculated actual z-axis distance is the vertical distance from the outer edge of the pupil to the outer corner of the eye, an estimate of exophthalmos can be calculated by adding the actual vertical distance from the center of the pupil and iris to the outer edge of the pupil to the calculated actual z-axis distance.

[0438] However, it is generally true that the actual vertical distance from the center of the pupil and iris to the outer edge of the pupil is similar for each person. Specifically, the actual vertical distance from the center of the pupil and iris to the outer edge of the pupil may generally be between 0.2 mm and 0.3 mm.

[0439] Therefore, an estimate of exophthalmos can be calculated by adding 0.25 mm, which is the average of 0.2 mm and 0.3 mm, to the calculated actual z-axis distance.

[0440] On the other hand, the actual vertical distance from the center of the pupil and iris to the outer edge of the pupil can be measured by medical staff when a patient visits a hospital and can also be used to estimate the degree of exophthalmos, but is not limited to this.

[0441]

[0442] (2) Method for determining exophthalmos based on facial images - 2

[0443] The degree of exophthalmos can be predicted by inputting feature amounts obtainable from a face image into a trained exophthalmos prediction model.

[0444] The exophthalmos prediction model may be a model that is trained by using training data that includes, as input data, feature amounts obtainable from facial images and includes, as label values, numerical exophthalmos values corresponding to the facial images.

[0445] The exophthalmos prediction model may be a regression model, a neural network model, a machine learning model, a deep learning model, or a combination thereof. For example, the exophthalmos prediction model may be, but is not limited to, a linear regression model, a polynomial regression model, a ridge regression model, a Lasso regression model, a support vector machine (SVM) model, a decision tree regression model, a random forest regression model, a K-nearest neighbor (KNN) model, a feedforward neural network model, a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a long short-term memory (LSTM) network model, a gated recurrent unit (GRU) model, a gradient boosting (e.g., XGBoost, LightGBM, and CatBoost) model, or an AdaBoost model.

[0446] The exophthalmos prediction model can be trained by various methods such as supervised learning, unsupervised learning, reinforcement learning, and imitation learning.

[0447] For example, the exophthalmos prediction model may be trained by supervised learning, where the exophthalmos prediction model may be trained by using features obtainable from a facial image and a numerical exophthalmos value corresponding to the facial image. For a more specific example, the exophthalmos prediction model may receive as input the features obtainable from a facial image and output a predicted exophthalmos value, and the exophthalmos prediction model may be trained based on the difference between the output predicted exophthalmos value and the output numerical exophthalmos value corresponding to the facial image.

[0448] The features that can be obtained from a face image are: the z-axis length value between the landmark located at the outer edge of the pupil and the landmark located at the corner of the outer circumference of the eye, where the landmark is obtained by applying a 3D face landmark detection model to the face image (hereinafter referred to as the z-axis length value); the angle-specific distance value (Multiple Radial Mid-Pupil Lid Distance, or Multiple Radial MPLD) from the center position of the pupil and iris in the face image to the boundary between the eyeball and eyelid (hereinafter referred to as the Radial MPLD value); the sum of the distance value corresponding to 0 degrees and the distance value corresponding to 180 degrees among the Radial MPLD values (hereinafter referred to as the 0-180 distance value); the distance value corresponding to 90 degrees among the Radial MPLD values; the distance value corresponding to 270 degrees among the Radial MPLD values; Radial The MPLD values may include the sum of the distance value corresponding to 90 degrees and the distance value corresponding to 270 degrees (hereinafter referred to as the 90-270 distance value); the horizontal length of the eye connecting the left and rightmost points of the eye region (hereinafter referred to as the horizontal length of the eye); the distance between the line indicating the horizontal length of the eye and the center position of the pupil and iris (hereinafter referred to as the distance between the horizontal length and the center position of the eye); the vertical length of the eye connecting the uppermost and lowermost points of the eye region (hereinafter referred to as the vertical length of the eye); and the distance between the line indicating the vertical length of the eye and the center position of the pupil and iris (hereinafter referred to as the distance between the vertical length and the center position of the eye).

[0449] Among the features obtainable from a facial image, the value of the z-axis length between the landmark located at the outer edge of the pupil and the landmark located at the corner of the outer circumference of the eye, which is a landmark obtained by applying a 3D facial landmark detection model to the facial image, can be obtained from: a frontal facial image, a facial image taken from a different angle, a panoramic facial image, and / or an image including a video capturing the face. The specific method for obtaining the z-axis length value was described above in (1) Facial Image-Based Exophthalmos Degree Determination Method-1, so duplicated description will be omitted.

[0450] Meanwhile, to obtain other features from the face image, an image segmentation model trained on the forward face image can be first applied.

[0451] The image segmentation model may be an artificial neural network model trained to identify objects shown in an image and detect regions of the objects in the image. Since the image segmentation model is a model commonly used to detect specific object regions from an image, a detailed description of the structure of the image segmentation model will be omitted.

[0452] On the other hand, the image segmentation model according to the present disclosure may be a model that detects eyeball regions and pupil and iris regions contained in a front face image.

[0453] By applying the forward face image to an image segmentation model, the eye regions and pupil and iris regions shown in the forward face image can be detected.

[0454] FIG. 10 is a diagram illustrating the eyeball region, pupil region, and iris region detected from a front face image.

[0455] As illustrated in FIG. 10, a forward face image 1020 in which an eyeball region 1021 is detected and a forward face image 1030 in which a pupil and iris region 1031 is detected can be obtained by applying an image segmentation model to the forward face image 1010.

[0456] In FIG. 10, the eyeball region 1021 is illustrated as not including the caruncle region, but the segmentation model may detect the caruncle region as the eyeball region 1021.

[0457] On the other hand, the segmentation model used to detect the eyeball region and the segmentation model used to detect the pupil and iris region may be different models. Specifically, the segmentation model used to detect the eyeball region and the segmentation model used to detect the pupil and iris region may be separate segmentation models trained by using different training data sets. In this case, the structures of the segmentation model used to detect the eyeball region and the segmentation model used to detect the pupil and iris region may be identical to each other before they are trained. Alternatively, the structures of the segmentation model used to detect the eyeball region and the segmentation model used to detect the pupil and iris region may also be different from each other before they are trained.

[0458] Alternatively, the image segmentation model may be a single segmentation model that is trained to detect both the eye region and the pupil and iris region.

[0459] Boundaries 1022 and 1023 of the eyeball and eyelid may be identified based on the detected eyeball region 1021. Specifically, the boundary 1022 between the eyeball and the upper eyelid may refer to the upper boundary of the eyeball region 1021, and the boundary 1023 between the eyeball and the lower eyelid may refer to the lower boundary of the eyeball region 1021.

[0460] A pupil and iris center position 1032 may be identified based on the detected pupil and iris region 1031. Specifically, the pupil and iris center position may be a center position of a circle corresponding to the detected pupil and iris region 1031.

[0461] The distances from the identified pupil and iris center locations 1032 to both the eyeball and eyelid boundaries 1022 and 1023 can be calculated.

[0462] Specifically, Radial MPLD values may be calculated from the identified pupil and iris center position 1032 to both the eyeball and eyelid boundaries 1022 and 1023. In this case, each distance value may be calculated as a pixel distance.

[0463] Meanwhile, the pixel distance corresponding to the pupil radius 1033 can be calculated based on the detected pupil and iris region 1031 .

[0464] FIG. 11 is a diagram illustrating feature amounts that can be acquired from a face image.

[0465] 11, to obtain the Radial MPLD value from the center position 1110 of the pupil and iris on the face image to the boundary 1120 between the eyeball and eyelid, which is one of the feature quantities, the angle from the center position 1110 of the pupil and iris to the boundary 1120 between the eyeball and eyelid may be distinguished in 15-degree increments, while the x-axis direction of the image is set to 0 degree, thereby calculating the Radial MPLD value for each distinguished angle. In this case, the 0-degree direction may be the direction from the center position 1110 of the pupil and iris on the face image to the caruncle.

[0466] On the other hand, since the Radial MPLD value for each angle is calculated with the x-axis direction of the image set to 0 degrees, the facial image needs to be aligned horizontally before calculating the Radial MPLD value for each angle. To align the horizontal level of the facial image, the center positions of the pupils and irises of both eyes on the facial image can be determined and the horizontal level of the image can be adjusted so that the line connecting the center positions of the pupils and irises of both eyes is horizontal.

[0467] Among the features that can be obtained from a facial image, the feature corresponding to the Radial MPLD value can be determined as a feature for all angle values of the Radial MPLD value, but for the feature corresponding to the Radial MPLD value, it is also possible that one value obtained from each angle value of the Radial MPLD value can be determined as a feature.

[0468] For example, the feature amount corresponding to the Radial MPLD value can be determined based on the correlation of the values of the Radial MPLD value for each angle.

[0469] In another example, the feature corresponding to the Radial MPLD value may be obtained by inputting the value for each angle of the Radial MPLD value into a separately provided value extraction model, which may be a model trained to output a single feature from multiple input values.

[0470] Of the feature amounts obtainable from a face image, the sum 1130 of the distance value corresponding to 0 degrees and the distance value corresponding to 180 degrees in the Radial MPLD value can be obtained based on the calculated Radial MPLD value.

[0471] Of the feature amounts obtainable from a face image, the sum 1130 of the distance value corresponding to 90 degrees and the distance value corresponding to 270 degrees in the Radial MPLD value can be obtained based on the calculated Radial MPLD value.

[0472] Among the features obtainable from a face image, the horizontal eye length 1140 connecting the leftmost and rightmost points of the eyeball region can be obtained based on the eyeball region detected through an image segmentation model.

[0473] Among the features that can be obtained from a facial image, a distance 1150 between a line representing the horizontal length 1140 of the eye and the center position 1110 of the pupil and iris can be obtained based on the obtained horizontal length 1140 of the eye and the center position 1110 of the pupil and iris.

[0474] Among the features obtainable from a face image, the vertical eye length 1160 connecting the uppermost and lowermost points of the eyeball region can be obtained based on the eyeball region detected through an image segmentation model.

[0475] Among the features that can be obtained from a facial image, the distance between a line representing the vertical length 1160 of the eye and the center position 1110 of the pupil and iris can be obtained based on the obtained vertical length 1160 of the eye and the center position 1110 of the pupil and iris.

[0476] The degree of exophthalmos can be determined based on an output value obtained by inputting feature amounts obtained from a face image into a trained exophthalmos prediction model.

[0477] FIG. 12 illustrates experimental results from the trained exophthalmos prediction model.

[0478] The experimental results in Fig. 12 are obtained from an exophthalmos prediction model trained by using 655 face images of 348 patients.

[0479] Specifically, the exophthalmos prediction model is trained using a training dataset that includes, as input data, feature values obtained from each of 655 facial images and, as label values, the actual exophthalmos numerical values corresponding to each facial image.

[0480] The experimental results in Fig. 12 are experimental results using different feature amounts obtained from face images. That is, the input data that is input to the exophthalmos prediction model and considered in each experimental result is different for each experimental result.

[0481] 12, a first experimental result 1210 is an experimental result for an exophthalmos prediction model trained by using training data including the z-axis length value of a face image and the Radial MPLD value of the face image as input data. In the first experimental result 1210, the MAE is 1.91135, and r 2 is confirmed to be 0.32806.

[0482] 12, the second experimental result 1220 is an experimental result for an exophthalmos prediction model trained by using training data including the z-axis length value of the face image, the Radial MPLD value of the face image, and the 0-180 distance value of the face image as input data. In the second experimental result 1220, the MAE is 1.90232, and r 2 was confirmed to be 0.33316.

[0483] 12, a third experimental result 1230 is an experimental result for an exophthalmos prediction model trained by using training data including the z-axis length value of the face image; the Radial MPLD value of the face image; the 0-180 distance value of the face image; and the horizontal length of the eye of the face image as input data. In the third experimental result 1230, the MAE is 1.87273, and r 2 is confirmed to be 0.35574.

[0484] 12, a fourth experimental result 1240 is an experimental result for an exophthalmos prediction model trained by using training data including the z-axis length value of the face image; the Radial MPLD value of the face image; the 0-180 distance value of the face image; the horizontal length of the eye of the face image; and the distance between the horizontal length of the eye and the center position of the eye of the face image as input data. In the fourth experimental result 1240, the MAE is 1.86234, and r2 is confirmed to be 0.35671.

[0485] According to the experimental results, it can be confirmed that the accuracy of exophthalmos prediction increases as various feature quantities are set as input data. On the other hand, in the case of comparing the third experimental result 1230 and the fourth experimental result 1240, it can be confirmed that the accuracy increases relatively significantly when the horizontal length of the eye is included in the input data. In other words, it can be confirmed that the horizontal length of the eye is an important feature quantity in predicting exophthalmos.

[0486] Meanwhile, the exophthalmos prediction model can be trained by using training data including the z-axis length value of the face image, the Radial MPLD value of the face image, and the 90-270 distance value of the face image as input data.

[0487] Alternatively, the exophthalmos prediction model may be trained by using training data including the z-axis length value of the face image, the Radial MPLD value of the face image, the 90-270 distance value of the face image, and the vertical eye length of the face image as input data.

[0488] Alternatively, the exophthalmos prediction model may be trained by using training data including as input data the z-axis length value of the facial image, the Radial MPLD value of the facial image, the 90-270 distance value of the facial image, the vertical eye length of the facial image, and the distance between the vertical eye length and the eye center position of the facial image.

[0489] Alternatively, the exophthalmos prediction model can also be trained by using training data that includes all of the features described above as input data, and the types of features included in the input data are not limited to the examples described above.

[0490] On the other hand, the exophthalmos prediction model may receive input of a face image together with features obtainable from the face image, where the face image may be a frontal face image, or alternatively, the face image may be an image including eye regions and pupil and iris regions obtained by performing image segmentation.

[0491]

[0492] (3) Exophthalmos Determination Method Based on Facial Images-3

[0493] The degree of exophthalmos can be predicted by inputting the difference values between the feature amounts obtainable from the respective facial images at two points in time into a trained exophthalmos prediction model.

[0494] The exophthalmos prediction model may be a model that is trained using training data that includes as input data the difference value between features obtainable from each of the facial images at two points in time and an exophthalmos numerical value corresponding to the facial image at one of the two points in time, and also includes as label values the exophthalmos numerical value corresponding to the facial image at the other of the two points in time.

[0495] The exophthalmos prediction model may be a regression model, a neural network model, a machine learning model, a deep learning model, or a combination thereof. Specific details are described in (2) Exophthalmos Determination Method Based on Facial Images-2, so duplicate descriptions will be omitted.

[0496] The types of features that can be obtained from facial images and the methods for obtaining the features are described above in (2) Method for determining exophthalmos based on facial images-2, so duplicate descriptions will be omitted.

[0497] The exophthalmos prediction model may receive not only a difference value between a first feature value obtained from a face image at a first time point and a second feature value obtained from a face image at a second time point, but also an exophthalmos value at the first time point, and output an exophthalmos value at the second time point. In this case, the first feature value and the second feature value considered to obtain the difference value are values for the same feature.

[0498] FIG. 13 illustrates experimental results from the trained exophthalmos prediction model.

[0499] The experimental results in Fig. 13 are obtained from an exophthalmos prediction model trained by using 844 face image sets of 196 patients, where one face image set includes two face images, i.e., a face image at a first time point and a face image at a second time point.

[0500] Specifically, the exophthalmos prediction model is trained by using a training dataset that includes as input data not only the difference values between features obtained from a facial image at one time point and features obtained from a facial image at another time point (the facial images are included in each of the 844 facial image sets), but also the exophthalmos numerical values corresponding to the facial image at one time point, and the exophthalmos numerical values corresponding to the facial image at the other time points as label values.

[0501] The experimental results in Fig. 13 are the results of experiments performed by applying different feature quantities obtained from face images. That is, the input data input to the exophthalmos prediction model and considered in each experimental result are different for each experimental result.

[0502] 13, a fifth experimental result 1310 is an experimental result for an exophthalmos prediction model trained by using training data including as input data: an exophthalmos value corresponding to a face image at one time point; a difference value between z-axis length values obtained from each face image at two time points; and a difference value between Radial MPLD values obtained from each face image at two time points. In the fifth experimental result 1310, the MAE is 1.85316, and r 2 is confirmed to be 0.36599.

[0503] 13, the sixth experimental result 1320 is an experimental result for an exophthalmos prediction model trained by using training data including as input data: an exophthalmos value corresponding to a face image at one time point; a difference value between z-axis length values obtained from each face image at two time points; a difference value between Radial MPLD values obtained from each face image at two time points; and a difference value between distance values of 0-180 obtained from each face image at two time points. In the sixth experimental result 1320, the MAE is 1.85043, and r 2 is confirmed to be 0.36660.

[0504] 13, the seventh experimental result 1330 is an experimental result for an exophthalmos prediction model trained by using training data including as input data: an exophthalmos value corresponding to a face image at one time point; a difference value between z-axis length values obtained from each face image at two time points; a difference value between Radial MPLD values obtained from each face image at two time points; a difference value between 0-180 distance values obtained from each face image at two time points; and a difference value between horizontal eye lengths obtained from each face image at two time points. In the seventh experimental result 1330, the MAE is 1.78171, and r 2 is confirmed to be 0.39728.

[0505] 13, the eighth experimental result 1340 is an experimental result for an exophthalmos prediction model trained by using training data including as input data: an exophthalmos value corresponding to a face image at one time point; a difference value between z-axis length values obtained from each face image at two time points; a difference value between Radial MPLD values obtained from each face image at two time points; a difference value between distance values of 0-180 obtained from each face image at two time points; a difference value between horizontal eye lengths obtained from each face image at two time points; and a difference value between horizontal eye lengths and distances between center positions obtained from each face image at two time points. In the eighth experimental result 1340, the MAE was 1.78160, and r 2 is confirmed to be 0.39584.

[0506] According to the experimental results, it can be confirmed that the more features are set, the higher the accuracy of exophthalmos prediction can be obtained. On the other hand, when comparing the seventh experimental result 1330 and the eighth experimental result 1340, it can be confirmed that the accuracy increases relatively greatly when the horizontal length of the eye is included in the input data. In other words, it can be confirmed that the horizontal length of the eye is an important feature in exophthalmos prediction.

[0507] Comparing the experimental results in FIG. 13 with those in FIG. 12, it can be seen that using the difference between features obtained from facial images at two time points provides higher accuracy than using only features obtained from facial images at one time point.

[0508] On the other hand, the exophthalmos prediction model can be trained by using training data including as input data: an exophthalmos value corresponding to a facial image at one time point; a difference value between z-axis length values obtained from each facial image at two time points; a difference value between Radial MPLD values obtained from each facial image at two time points; and a difference value between 90-270 distance values obtained from each facial image at two time points.

[0509] On the other hand, the exophthalmos prediction model can be trained by using training data including as input data: an exophthalmos value corresponding to a facial image at one time point; a difference value between z-axis length values obtained from each facial image at two time points; a difference value between Radial MPLD values obtained from each facial image at two time points; a difference value between 90-270 distance values obtained from each facial image at two time points; and a difference value between vertical eye lengths obtained from each facial image at two time points.

[0510] On the other hand, the exophthalmos prediction model can be trained by using training data including as input data: an exophthalmos value corresponding to a facial image at one time point; a difference value between the z-axis length values obtained from each facial image at two time points; a difference value between the Radial MPLD values obtained from each facial image at two time points; a difference value between the 90-270 distance values obtained from each facial image at two time points; a difference value between the vertical eye lengths obtained from each facial image at two time points; and a difference value between the vertical eye lengths and distances between the center positions obtained from each facial image at two time points.

[0511] Alternatively, the exophthalmos prediction model can also be trained by using training data including, as input data, an exophthalmos numerical value corresponding to a facial image at one time point; and differential values of various types of features obtained from each of the facial images at two time points. The types of features included in the input data are not limited to the examples described above. Meanwhile, the exophthalmos prediction model can also receive input of facial images at two time points along with differential values of features obtainable from each of the facial images at the two time points. In this case, the facial image can be a forward facial image. Alternatively, the facial image can be an image including eyeball regions and pupil and iris regions obtained by performing image segmentation.

[0512]

[0513] (4) Exophthalmos Determination Method Based on Facial Images-4

[0514] On the other hand, since exophthalmos is the perpendicular distance between the corneal apex and the lateral orbital rim, exophthalmos can also be estimated based on profile images.

[0515] To estimate exophthalmos from a profile image, a corresponding forward face image may be acquired along with the profile image.

[0516] FIG. 14 is a diagram illustrating a method for estimating exophthalmos by using a profile image.

[0517] 14, the distance 1411 between the corneal end point and the eye end point can be calculated from the profile image 1410. In this case, the distance 1411 between the corneal end point and the eye end point can be a pixel distance. An image segmentation model can be used to calculate the distance 1411 between the corneal end point and the eye end point from the profile image 1410, but is not limited to such.

[0518] Meanwhile, a distance 1421 corresponding to the pupil radius may be calculated from a front face image 1420 corresponding to the profile face image 1410. In this case, the distance 1421 corresponding to the pupil radius may be a pixel distance. An image segmentation model may be used to calculate the distance 1421 corresponding to the pupil radius from the front face image 1420, but is not limited to this.

[0519] The profile face image 1410 and the front face image 1420 correspond to each other may mean that the resolution of each of the images is the same or the difference in the resolution is within a threshold. Specifically, it may mean that the ratio of the pixel distance and the actual distance on the profile face image 1410 is the same as the ratio of the pixel distance and the actual distance on the front face image 1420, or the difference between the ratios is within a threshold.

[0520] To obtain a front face image 1420 corresponding to the profile face image 1410, the profile face image 1410 and the front face image 1420 may be captured under the condition that the distance between the face and the photographing device when capturing the profile face image 1410 is the same as the distance between the face and the photographing device when capturing the front face image 1420. In this case, the photographing device may be, but is not limited to, a patient's user device, and may also be a medical staff device in a hospital.

[0521] On the other hand, as described above, the actual length of the pupil radius is generally similar for each person, and specifically, the actual length of the pupil radius may be 0.00 mm.

[0522] That is, the resolution of the front face image 1420 can be determined by using the pupil radius length 1421 calculated from the front face image 1420 and the actual length of the pupil radius.

[0523] Because the resolution of each of the profile face image 1410 and the front face image 1420 is the same, the actual distance corresponding to the distance 1411 between the end point of the cornea and the end point of the eye on the profile face image 1410 can be calculated based on the resolution determined from the front face image 1420.

[0524] Specifically, the actual distance between the end point of the cornea and the end point of the eye can be calculated by multiplying the ratio of the length 1421 of the pupil radius calculated from the front face image 1420 and the actual length of the pupil radius, which is 00 mm, by the distance 1411 between the end point of the cornea and the end point of the eye calculated from the profile face image 1410.

[0525] The calculated actual distance between the corneal endpoint and the ocular endpoint can be an estimate of exophthalmos.

[0526] On the other hand, in FIG. 14, the front face image 1420 and the side face image 1410 are shown as separate images, but the face image used to estimate the degree of exophthalmos may be a face image captured panoramic from the front to the side of the face.

[0527] Specifically, the distance corresponding to the pupil radius and the distance between the corneal endpoint and the ocular endpoint can be calculated from a panoramic facial image captured from the front to the side of the face, in which case the distance corresponding to the pupil radius and the distance between the corneal endpoint and the ocular endpoint can each be calculated as a pixel distance.

[0528] The actual distance between the corneal end point and the ocular end point can be calculated by multiplying the ratio of the distance corresponding to the calculated radius of the pupil and the actual distance of the pupil radius by the distance between the corneal end point and the ocular end point, i.e., the exophthalmos can be estimated as the distance between the calculated corneal end point and the ocular end point.

[0529]

[0530] 6. Method for determining exophthalmos tendency based on facial images

[0531] Even if the numerical value of exophthalmos itself is not determined from a facial image, the trend in exophthalmos can be determined based on a comparison of facial images at two time points. That is, when it is difficult to calculate the numerical value of exophthalmos itself from a facial image, by comparing facial images at two time points and determining the trend in exophthalmos, it is possible to monitor whether a therapeutic drug for treating thyroid eye disease is effective following its administration. In this case, the trend in exophthalmos may mean the trend in changes in exophthalmos.

[0532] Meanwhile, the facial image may be acquired from the patient. Specifically, the facial image may be acquired from the patient's user device, and the facial image may be, but is not limited to, an image captured by the patient's user device. The facial image may also be acquired from a hospital. Specifically, the facial image may be acquired by allowing medical staff assigned to the hospital to capture the patient's face when the patient visits the hospital.

[0533] The facial image may be, but is not limited to, an image of the area between the bottom of the nose and the top of the eyebrows, and the facial image may refer to an image illustrating the eye area.

[0534]

[0535] FIG. 15 is a diagram illustrating a method for determining a tendency in exophthalmos based on a face image.

[0536] 15 , obtaining 1511 a first value of an exophthalmos-related variable from a facial image 1510 acquired at a first time point may be performed, and obtaining 1521 a second value of the exophthalmos-related variable from a facial image 1520 acquired at a second time point may be performed, where the second time point may be a time point later than the first time point.

[0537] In this case, the same imaging guide may be provided for capturing facial images at the first and second time points. That is, the patient's face shown in both facial image 1510 acquired at the first time point and facial image 1520 acquired at the second time point may have the same composition. Therefore, facial image 1510 acquired at the first time point and facial image 1520 acquired at the second time point may be compared with each other. Specifically, a first value of a variable related to exophthalmos acquired from facial image 1510 acquired at the first time point and a second value of a variable related to exophthalmos acquired from facial image 1520 acquired at the second time point may be compared with each other. Specific details regarding the imaging guide will be described later in 8. Capturing and Transmitting Facial Images.

[0538] The variable related to the degree of exophthalmos is a variable that can be obtained by analyzing a facial image, and may refer to a variable whose value increases as the degree of exophthalmos increases. Specifically, the variable related to the degree of exophthalmos may increase in direct proportion or square proportion as the degree of exophthalmos increases, and an increase or decrease in the variable related to the degree of exophthalmos due to an increase or decrease in the degree of exophthalmos is not limited thereto.

[0539] Thus, a trend in exophthalmos between a first time point and a second time point can be determined by performing a comparison 1530 of a first value of an exophthalmos-related variable obtained from a facial image 1510 acquired at a first time point and a second value of an exophthalmos-related variable obtained from a facial image 1520 acquired at a second time point.

[0540]

[0541] Whether the degree of exophthalmos has improved, worsened, or remained unchanged between the first and second time points can be determined based on the determined trend of the degree of exophthalmos.

[0542] Specifically, if the second value is less than the first value, it can be determined that the trend in exophthalmos is decreasing between the first and second time points, and since the trend in exophthalmos is decreasing, it can be determined that the exophthalmos is alleviated between the first and second time points, and therefore it can be determined that the therapeutic agent is effective between the first and second time points.

[0543] Alternatively, if the second value is greater than the first value, the trend in exophthalmos between the first and second time points may be determined to be an increasing trend, and because the trend in exophthalmos is an increasing trend, it may be determined that the exophthalmos worsened between the first and second time points, and therefore that the therapeutic agent is not effective between the first and second time points.

[0544] Alternatively, if the second value and the first value are the same, the trend in exophthalmos between the first and second time points may be determined to be a trend of no change, and since the trend in exophthalmos is a trend of no change, it may be determined that the exophthalmos has not changed between the first and second time points, and therefore it may be determined that there is no effect of the therapeutic agent between the first and second time points.

[0545]

[0546] On the other hand, the exophthalmos-related variable may be a variable for a value including a numerical value of exophthalmos, a Multiple Radial Mid-Pupil Lid Distance (Radial MPLD) value, a horizontal length of the eye, a 3D facial landmark coordinate value, and / or a value for the distance between the end point of the cornea and the end point of the eye. The exophthalmos-related variable is not limited to the examples described above.

[0547]

[0548] Among the exophthalmos-related variables, the exophthalmos numerical value can be obtained by the facial image-based exophthalmos determination method described above.

[0549] Specifically, a first exophthalmos numerical value may be obtained as a first value of an exophthalmos-related variable from a face image 1510 at a first time point, and a second exophthalmos numerical value may be obtained as a second value of an exophthalmos-related variable from a face image 1520 at a second time point. Specific details regarding obtaining exophthalmos numerical values from face images have been described above in the face image-based exophthalmos determination method, and therefore, redundant description will be omitted.

[0550] The exophthalmos value obtained from the facial images belongs to a variable whose value increases as the degree of exophthalmos increases and whose value decreases as the degree of exophthalmos decreases. Thus, a trend in the degree of exophthalmos between a first time point and a second time point can be determined by comparing the first exophthalmos value and the second exophthalmos value.

[0551]

[0552] Among the exophthalmos-related variables, the Radial MPLD value can be obtained by the method described above in 5. Method for determining exophthalmos based on facial images.

[0553] Specifically, a first eyeball region and a first pupil and iris region shown in the first time-point face image 1510 may be detected by applying the first time-point face image 1510 to an image segmentation model. A first boundary between the eyeball and the eyelid in the first time-point face image 1510 may be identified based on the detected first eyeball region. A first center position of the pupil and the iris shown in the first time-point face image 1510 may be identified based on the detected first pupil and iris region. A first Radial MPLD value may be calculated as an angle-specific distance value from a first position of the pupil center to the first boundary between the eyeball and the eyelid. A specific method for calculating the Radial MPLD value has been described above in 5. Method for Determining Exophthalmos Based on a Facial Image, so a duplicated description will be omitted.

[0554] A second eyeball region and a second pupil and iris region shown in the face image 1520 at the second time point may be detected by applying the face image 1520 at the second time point to an image segmentation model. A second boundary between the eyeball and the eyelid shown in the face image 1520 at the second time point may be identified based on the detected second eyeball region. A second center position of the pupil and the iris shown in the face image 1520 at the second time point may be identified based on the detected second pupil and iris region. A second Radial MPLD value may be calculated as an angle-specific distance value from the second position of the pupil center to the second boundary between the eyeball and the eyelid. A specific method for calculating the Radial MPLD value has been described above in 5. Facial Image-Based Exophthalmos Determination Method, so a duplicated description will be omitted.

[0555] The Radial MPLD values obtained from the facial images belong to a variable whose value increases as the degree of exophthalmos increases and whose value decreases as the degree of exophthalmos decreases. Thus, a trend in the degree of exophthalmos between a first time point and a second time point can be determined by comparing the first Radial MPLD value and the second Radial MPLD value.

[0556] In this case, the trend in exophthalmos can be determined by comparing the Radial MPLD values between 195 degrees and 345 degrees. Specifically, the trend in exophthalmos can be determined by comparing the value between 195 degrees and 345 degrees determined from the first Radial MPLD values obtained from the face image at a first time point with the value between 195 degrees and 345 degrees determined from the second Radial MPLD values obtained from the face image at a second time point. The angle determined from the first Radial MPLD values and the angle determined from the second Radial MPLD values can be equal to each other.

[0557] The use of Radial MPLD values between 195 degrees and 345 degrees to determine trends in exophthalmos is described with reference to FIGS.

[0558] 16 and 17 are graphs illustrating the relationship between Radial MPLD values and exophthalmos.

[0559] FIG. 16 illustrates exophthalmos distribution graphs for Radial MPLD values, including: exophthalmos distribution graph 1610 for values corresponding to 0 degrees; exophthalmos distribution graph 1615 for values corresponding to 15 degrees; exophthalmos distribution graph 1620 for values corresponding to 30 degrees; exophthalmos distribution graph 1625 for values corresponding to 45 degrees; exophthalmos distribution graph 1630 for values corresponding to 60 degrees; exophthalmos distribution graph 1635 for values corresponding to 75 degrees; exophthalmos distribution graph 1640 for values corresponding to 90 degrees; exophthalmos distribution graph 1645 for values corresponding to 105 degrees; exophthalmos distribution graph 1650 for values corresponding to 120 degrees; exophthalmos distribution graph 1655 for values corresponding to 135 degrees; exophthalmos distribution graph 1660 for values corresponding to 150 degrees; and exophthalmos distribution graph 1665 for values corresponding to 165 degrees.

[0560] FIG. 17 illustrates exophthalmos distribution graphs for Radial MPLD values, including: exophthalmos distribution graph 1710 for values corresponding to 180 degrees; exophthalmos distribution graph 1715 for values corresponding to 195 degrees; exophthalmos distribution graph 1720 for values corresponding to 210 degrees; exophthalmos distribution graph 1725 for values corresponding to 225 degrees; exophthalmos distribution graph 1730 for values corresponding to 240 degrees; exophthalmos distribution graph 1735 for values corresponding to 255 degrees; exophthalmos distribution graph 1740 for values corresponding to 270 degrees; exophthalmos distribution graph 1745 for values corresponding to 285 degrees; exophthalmos distribution graph 1750 for values corresponding to 300 degrees; exophthalmos distribution graph 1755 for values corresponding to 315 degrees; exophthalmos distribution graph 1760 for values corresponding to 330 degrees; and exophthalmos distribution graph 1765 for values corresponding to 345 degrees.

[0561] 16 and 17, the Radial MPLD values corresponding to 0-180 degrees in graphs 1610, 1615, 1620, 1625, 1630, 1635, 1640, 1645, 1650, 1655, 1660, 1665, and 1710 do not increase or decrease with an increase or decrease in exophthalmos. However, the Radial MPLD values corresponding to 195-345 degrees in graphs 1715, 1720, 1725, 1730, 1735, 1740, 1745, 1750, 1755, 1760, and 1765 show a distribution in a diagonal direction upward to the upper right, where the Radial MPLD values increase as the exophthalmos increase and decrease as the exophthalmos decrease. Therefore, trends in exophthalmos can be determined by comparing Radial MPLD values between 195 and 345 degrees.

[0562] On the other hand, each Radial MPLD value obtained from a face image may be calculated as an actual distance, but is not limited thereto, and each Radial MPLD value obtained from a face image may also be calculated as a pixel distance.

[0563] In this case, to compare the first Radial MPLD value and the second Radial MPLD value, a first pupil radius pixel distance shown in the face image 1510 at the first time point and a second pupil radius pixel distance shown in the face image 1520 at the second time point may be used. In this case, the face image 1510 at the first time point and the face image 1520 at the second time point may be forward face images.

[0564] Specifically, the ratio of the first Radial MPLD value to the first pupil radius pixel distance may be compared to the ratio of the second Radial MPLD value to the second pupil radius pixel distance.

[0565] That is, the trend in exophthalmos can be determined by comparing the ratio of the first Radial MPLD value to the first pupil radius pixel distance and the ratio of the second Radial MPLD value to the second pupil radius pixel distance.

[0566]

[0567] Among the exophthalmos-related variables, the horizontal eye length may refer to the horizontal length of the eyeball region shown in the forward face image. Specifically, the horizontal eye length may refer to the distance between the left and right end points of the eyeball region. Alternatively, the horizontal eye length may refer to the length of the eyeball region in the x-axis direction. Without being limited thereto, the horizontal eye length may also refer to the length between the position of the corner of the outer periphery of the eye and the position of the caruncle.

[0568] Meanwhile, the horizontal length of the eye can be calculated based on the horizontal length of the eye region obtained by applying the front face image to the image segmentation model. Since the details of obtaining the eye region by applying the front face image to the image segmentation model have been described above, the duplicated description will be omitted.

[0569] Specifically, a first eye region shown in the first time point face image 1510 may be detected by applying the first time point face image 1510 to an image segmentation model. A horizontal length of a first eye shown in the first time point face image 1510 may be calculated based on the horizontal length of the first eye region.

[0570] The second eye region shown in the second time point face image 1520 may be detected by applying the second time point face image 1520 to an image segmentation model. The horizontal length of the second eye shown in the second time point face image 1520 may be calculated based on the horizontal length of the second eye region.

[0571] The horizontal eye length obtained from the facial images belongs to a variable whose value increases as the degree of exophthalmos increases and whose value decreases as the degree of exophthalmos decreases. Thus, a trend in the degree of exophthalmos between a first time point and a second time point can be determined by comparing the horizontal eye length of the first eye and the horizontal eye length of the second eye.

[0572] FIG. 18 is a graph illustrating the relationship between horizontal eye length and exophthalmos.

[0573] In FIG. 18, the horizontal length of the eye is set as the distance between the left and rightmost points of the eyeball region.

[0574] 18, a distribution in the upper right direction is shown in which the horizontal length of the eye increases as the degree of exophthalmos increases and decreases as the degree of exophthalmos decreases. Thus, trends in the degree of exophthalmos can be determined by comparing the horizontal lengths of the eyes.

[0575] Meanwhile, the horizontal length of the eye obtained from the face image may be calculated as, but is not limited to, an actual distance, and the horizontal length of the eye obtained from the face image may also be calculated as a pixel distance.

[0576]

[0577] In this case, a first pupil radius pixel distance shown in the face image 1510 at the first time point and a second pupil radius pixel distance shown in the face image 1520 at the second time point may be used to compare the horizontal length of the first eye and the horizontal length of the second eye. In this case, the face image 1510 at the first time point and the face image 1520 at the second time point may be forward face images.

[0578] Specifically, the ratio of the horizontal length of the first eye to the first pupil radius pixel distance may be compared to the ratio of the horizontal length of the second eye to the second pupil radius pixel distance.

[0579] That is, the trend in exophthalmos can be determined by comparing the ratio of the horizontal length of the first eye to the pixel distance of the first pupil radius and the ratio of the horizontal length of the second eye to the pixel distance of the second pupil radius.

[0580]

[0581] Three-dimensional (3D) facial landmark coordinate values can be obtained by applying a facial image to a 3D facial landmark detection model. Among the obtained 3D facial landmark coordinate values, the z-axis distance value between the z-axis coordinate value of the landmark representing the outer edge of the pupil and the z-axis coordinate value of the landmark representing the corner of the outer circumference of the eye can be considered as a value for an exophthalmos-related variable. Details regarding the 3D facial landmark detection model have been described above in 5. Exophthalmos Determination Method Based on Facial Image, so duplicated description will be omitted.

[0582] Specifically, a first z-axis coordinate value of a landmark representing the outer edge of the pupil and a second z-axis coordinate value of a landmark representing the outer corner of the eye may be obtained by applying the 3D facial landmark detection model to the first time point face image 1510. A first z-axis distance value may be obtained based on the distance between the first z-axis coordinate value and the second z-axis coordinate value.

[0583] By applying the 3D facial landmark detection model to the second time point face image 1520, a third z-axis coordinate value of the landmark representing the outer edge of the pupil and a fourth z-axis coordinate value of the landmark representing the outer corner of the eye can be obtained. A second z-axis distance value can be obtained based on the distance between the third z-axis coordinate value and the fourth z-axis coordinate value.

[0584] The z-axis distance value between the z-axis coordinate value of the landmark representing the outer edge of the pupil and the z-axis coordinate value of the landmark representing the corner of the circumference of the eye, obtained from the facial image, belongs to a variable whose value increases as the degree of exophthalmos increases and whose value decreases as the degree of exophthalmos decreases. Thus, the trend in the degree of exophthalmos between the first time point and the second time point can be determined by comparing the first z-axis distance value and the second z-axis distance value.

[0585] Meanwhile, each of the z-axis distance values obtained from the face image may be calculated as, but is not limited to, an actual distance, and each of the z-axis distance values obtained from the face image may also be calculated as a pixel distance.

[0586] In this case, the first pupil radius pixel distance shown in the face image 1510 at the first time point and the second pupil radius pixel distance shown in the face image 1520 at the second time point can be used to compare the first z-axis distance value and the second z-axis distance value.

[0587] Specifically, a ratio of the first z-axis distance value and the first pupil radius pixel distance may be compared to a ratio of the second z-axis distance value and the second pupil radius pixel distance.

[0588] That is, the trend in exophthalmos can be determined by comparing the ratio of the first z-axis distance value and the first pupil radius pixel distance with the ratio of the second z-axis distance value and the second pupil radius pixel distance.

[0589]

[0590] Among the exophthalmos-related variables, the distance between the corneal end point and the ocular end point can be obtained by using the method described above in the method for determining exophthalmos based on a profile image.

[0591] Specifically, a first distance between the corneal end point and the eye end point may be obtained as a first value of the exophthalmos-related variable from the face image 1510 at a first time point, and a second distance between the corneal end point and the eye end point may be obtained as a second value of the exophthalmos-related variable from the face image 1520 at a second time point. A specific method for obtaining the distance between the corneal end point and the eye end point from a face image has been described above in the face image-based exophthalmos determination method, so a duplicated description will be omitted.

[0592] The distance between the corneal endpoint and the ocular endpoint obtained from the facial images belongs to a variable whose value increases as the degree of exophthalmos increases and whose value decreases as the degree of exophthalmos decreases. Thus, a trend in the degree of exophthalmos between a first time point and a second time point can be determined by comparing a first distance between the corneal endpoint and the ocular endpoint and a second distance between the corneal endpoint and the ocular endpoint.

[0593] Meanwhile, each of the distances between the corneal end points and the eye end points obtained from the face image may be calculated as, but is not limited to, an actual distance, and each of the distances between the corneal end points and the eye end points obtained from the face image may also be calculated as, but is not limited to, a pixel distance.

[0594] In this case, a first pupil radius pixel distance shown in the face image 1510 at the first time point and a second pupil radius pixel distance shown in the face image 1520 at the second time point can be used to compare a first distance between the corneal endpoint and the ocular endpoint and a second distance between the corneal endpoint and the ocular endpoint. In this case, the first pupil radius pixel distance can be obtained from a forward face image included in the face image 1510 at the first time point, and the second pupil radius pixel distance can be obtained from a forward face image included in the face image 1520 at the second time point.

[0595] Specifically, the ratio of a first distance between the corneal endpoint and the ocular endpoint to a first pupil radius pixel distance can be compared to the ratio of a second distance between the corneal endpoint and the ocular endpoint to a second pupil radius pixel distance.

[0596] That is, the trend in exophthalmos can be determined by comparing the ratio of a first distance between the corneal endpoint and the ocular endpoint to a first pupil radius pixel distance with the ratio of a second distance between the corneal endpoint and the ocular endpoint to a second pupil radius pixel distance.

[0597]

[0598] 7. Eyelid retraction detection method based on facial images

[0599] Eyelid retraction is an index of how much of the sclera is exposed when the upper eyelid is pulled upward or the lower eyelid is lowered. Eyelid retraction can be determined by the distance between the center positions of the pupil and iris and the upper eyelid, and the distance between the center positions of the pupil and iris and the lower eyelid. That is, to determine eyelid retraction, the center positions of the pupil and iris on the face image and the boundary between the eyeball and the eyelid need to be determined.

[0600] Since eyelid retraction is determined by the distance between the center of the pupil and the upper eyelid or the distance between the center of the pupil and the lower eyelid, and exophthalmos is determined by the vertical distance between the corneal apex and the lateral orbital rim, eyelid retraction and exophthalmos can be understood as mutually different indicators.

[0601] The pupil center position can be determined based on the pupil and iris regions detected by applying an image segmentation model to the front face image, as described above. The method for determining the pupil and iris center positions from the front face image has been described above in 5. Method for Determining Exophthalmos Based on Facial Images, so a duplicate description will be omitted.

[0602] The boundary between the eyeball and the eyelid can be determined based on the eyeball region detected by applying an image segmentation model to the front face image, as described above. The method for determining the boundary between the eyeball and the eyelid from the front face image has been described above in 5. Method for determining exophthalmos based on face image, so a duplicated description will be omitted.

[0603] Meanwhile, the facial image may be acquired from the patient. Specifically, the facial image may be acquired from the patient's user device, and the facial image may be, but is not limited to, an image captured by the patient's user device. The facial image may also be acquired from a hospital. Specifically, the facial image may be acquired by allowing medical staff assigned to the hospital to capture the patient's face when the patient visits the hospital.

[0604] The facial image may be an image of the area between the bottom of the nose and the top of the eyebrows, and may refer to, but is not limited to, an image illustrating the eye area.

[0605] FIG. 19 is a diagram illustrating a method for determining eyelid retraction from a front face image.

[0606]

[0607] 19 , the center positions of the pupil and iris 1920 can be determined based on the pupil and iris regions detected by applying an image segmentation model to the front face image 1910. The upper eyelid boundary 1931 and the lower eyelid boundary 1932 can be determined based on the eyeball regions detected by applying an image segmentation model to the front face image 1910. The method for determining the center positions of the pupil and iris and the boundaries between the eyeball and the eyelid from the front face image has been described above in 5. Method for Determining Exophthalmos Degree Based on a Facial Image, so a duplicated description will be omitted.

[0608] Referring to FIG. 19, a value for eyelid retraction, the marginal corneal reflex distance 1 (MRD1) 1941, can be calculated based on the center positions of the pupil and iris 1920 and the upper eyelid boundary 1931.

[0609] Meanwhile, the marginal corneal reflex distance 2 (MRD2) 1942, which is a value related to eyelid retraction, can be calculated based on the center positions of the pupil and iris 1920 and the lower eyelid boundary 1932.

[0610] MRD1 may refer to the distance from the center of the patient's pupil to the center of the upper eyelid when the patient is looking straight ahead, and MRD2 may refer to the distance from the center of the patient's pupil to the center of the lower eyelid when the patient is looking straight ahead.

[0611] Meanwhile, each of MRD1 and MRD2 obtained from the face image can be calculated as a pixel distance, and the actual distance of each of MRD1 and MRD2 can be calculated by using the radius of the pupil.

[0612] Specifically, the pixel distance corresponding to the pupil radius is obtained from the front face image, and the resolution of the image is determined by considering the pixel distance corresponding to the pupil radius and the actual length of the pupil radius, and then the actual distance corresponding to the pixel distance of MRD1 and MRD2 can be calculated by considering the resolution of the image.

[0613]

[0614] 19 , the center positions of the pupil and iris 1960 can be determined based on the pupil and iris regions detected by applying an image segmentation model to the front face image 1950. The boundary 1970 between the eyeball and eyelid can be determined based on the eyeball region detected by applying an image segmentation model to the front face image 1950. The method for determining the center positions of the pupil and iris and the boundary between the eyeball and eyelid from the front face image has been described above in 5. Method for Determining Exophthalmos Degree Based on Facial Image, so a duplicated description will be omitted.

[0615] 19, the Radial MPLD value can be calculated as the distance from the center position 1960 of the pupil and iris as the center to the boundary 1970 between the eyeball and eyelid. The angle can be distinguished in 15-degree increments, while the x-axis direction of the image is set to 0 degrees. Details regarding the calculation of the Radial MPLD value have been described above in 5. Method for determining exophthalmos based on facial images, so a duplicated explanation will be omitted.

[0616] Among the Radial MPLD values, the values corresponding to 90 degrees and / or 270 degrees may be calculated as values related to eyelid retraction. Specifically, among the Radial MPLD values, the value corresponding to 90 degrees may be a value related to upper eyelid retraction, and among the Radial MPLD values, the value corresponding to 270 degrees may be a value related to lower eyelid retraction.

[0617] Meanwhile, each Radial MPLD value obtained from a face image may be calculated as a pixel distance, and the actual distance of each Radial MPLD value may be calculated by using the radius of the pupil.

[0618] Specifically, a pixel distance corresponding to the pupil radius is obtained from the front face image, the resolution of the image is determined by considering the pixel distance corresponding to the pupil radius and the actual length of the pupil radius, and then the actual distance corresponding to the pixel distance of the Radial MPLD value can be calculated by considering the resolution of the image.

[0619]

[0620] 8. Capturing and sending facial images

[0621] If the patient's face is rotated left and right and / or up and down when capturing a facial image, distortion may occur in the facial image. When distortion occurs in the facial image, the accuracy of facial image analysis may decrease, so the patient may be provided with a shooting guide to obtain a preferred facial image.

[0622] The imaging guide provided to the patient may be provided from a user device, may be pre-stored on the user device, or may be retrieved from the analysis server and provided to the patient when the patient is requested to capture an image, without being limited thereto.

[0623] The patient may capture a facial image according to the provided photographing guide and store the captured facial image on the user device or transmit the captured facial image to an analysis server.

[0624] Without being limited thereto, when a medical staff member captures a facial image, the medical staff member may capture the facial image by following the provided photographing guide, and may store the captured facial image on a medical staff device, a hospital server, and / or a medical server, or may transmit the captured facial image to an analysis server.

[0625] Below, a method for capturing and transmitting a facial image is described.

[0626] FIG. 20 is a flowchart illustrating a process for capturing and transmitting a facial image according to an example embodiment.

[0627] Referring to FIG. 20, a method 2000 for capturing and transmitting a facial image may include a step S2010 of requesting a facial image capture, a step S2020 of providing a facial image capture guide, a step S2030 of determining whether the capture guide is satisfied, a step S2040 of capturing an image when the capture guide is satisfied, and a step S2050 of verifying and transmitting the captured image to an analysis server.

[0628] The step S2010 of requesting facial image capture may be a step of requesting capturing a facial image that is the target of analysis to obtain a personalized estimate of the patient and patient data.

[0629] For example, the analysis server may send a facial image capture request to the patient's user device at the time of monitoring. As another example, the analysis server may send a facial image capture request to the patient's user device when receiving a monitoring request from the patient's user device. Without being limited thereto, the analysis server may also send a facial image capture request to a hospital medical staff device, or may send a facial image capture request to a medical staff device when receiving a monitoring request from the hospital medical staff device. That is, step S2010 of requesting facial image capture may be performed at the time of monitoring, or may be performed at any time according to the needs of the patient and / or medical staff.

[0630] Step S2020 of providing a facial image photographing guide may be a step of providing a photographing guide to assist in capturing a facial image so as to obtain a desirable facial image.

[0631] For example, if the patient's user device needs to capture a facial image, the patient's user device may display a photography guide. As a specific example, a display on the patient's user device may display the photography guide along with a preview image.

[0632] The photography guide may include text, audio, indicators, and / or graphics that guide the angle of the user device, the distance between the user device and the user's face, the angle of the face, the position of the face, the position of the eyes, and / or facial expressions. In this case, whether the photography guide is met may be determined based on whether the angle of the user device, the distance between the user device and the user's face, the vertical angle of the face, the horizontal angle of the face, the position of the face in the image, the position of the eyes in the image, and / or facial expressions meet the criteria.

[0633] The photography guide may include text, audio, indicators, and / or graphics that guide whether the face is angled properly, whether the face is centered, whether the face is angled properly left to right, whether the eye position is proper, and / or whether the facial expression is neutral.

[0634] For example, a shooting guide indicating whether the vertical angle of the face is appropriate may be displayed as a horizontal line on the preview image, and a shooting guide indicating whether the horizontal angle of the face is appropriate may be displayed as a vertical line on the preview image. The horizontal line may be at the center of the preview image, and the vertical line may be at the center of the preview image. In this case, when the vertical angle of the face is appropriate, the color of the horizontal line on the preview image may change, and when the horizontal angle of the face is appropriate, the color of the vertical line on the preview image may change. Therefore, the patient can easily visually confirm whether the shooting guide is met.

[0635] In addition, a shooting guide indicating whether the eye position is appropriate may be displayed as a crosshair at the appropriate eye position on the preview image. In this case, the color of the crosshair on the preview image may change when the eye position is appropriate. Therefore, the patient may easily visually confirm whether the shooting guide is satisfied.

[0636] In addition, a shooting guide indicating whether the eye position is appropriate may be displayed as an indicator and / or the shape of a pupil's appearance near the preview image, and a shooting guide indicating whether the facial expression is neutral may be displayed as an indicator and / or the shape of a neutral facial expression near the preview image. In this case, when the pupil position is appropriate, the color of the indicator and / or the shape of the pupil's appearance may change, and when the facial expression is neutral, the color of the indicator and / or the shape of the neutral facial expression may change. Thus, the patient can easily visually confirm whether the shooting guide is satisfied.

[0637] On the other hand, since the conditions to be met by the front face image and the profile face image may be different from each other, the shooting guide provided for capturing the front face image and the shooting guide provided for capturing the profile face image may be different from each other.

[0638] For example, since information about pupils obtained from forward facial images is used in various ways in image analysis, the imaging guides provided for capturing forward facial images may include imaging guides for aligning pupil position.

[0639] On the other hand, when a medical staff device in a hospital needs to capture a face image, the medical staff device may display a shooting guide. Specifically, a display equipped on the medical staff device may display the shooting guide together with a preview image. Specific details regarding the shooting guide have been described above, so a duplicate description will be omitted.

[0640] The step S2030 of determining whether the imaging guide is satisfied may be a step of determining whether the preview image captured by the patient's user device satisfies the provided imaging guide.

[0641] For example, if the shooting guide is for aligning the left-right angle and the up-down angle of the face, the user device may determine whether the angle of the face in the captured preview image is aligned in accordance with the shooting guide. In a more specific example, the user device may determine the roll, pitch, and / or yaw of the face in the captured preview image, and determine whether the angle of the face is facing forward, facing left, or facing right based on the determined result.

[0642] Without being limited thereto, depending on what the photography guide is intended to guide, the user device may determine whether the face in the preview image satisfies the photography guide.

[0643] The step S2040 of capturing an image when the photography guide is satisfied may be a step of capturing a face image when a face in a preview image captured by the user device satisfies the photography guide.

[0644] For example, the user device may determine whether the preview facial image satisfies the photography guide, as described above, and may capture the facial image if the photography guide is satisfied.

[0645] The facial image capture may be performed by the patient interacting with the imaging interface of the user device, including, but not limited to, displaying text, audio, and / or shapes on the user device to indicate that the imaging guide has been met.

[0646] On the other hand, capturing of facial images may also be performed by the user device to automatically capture facial images when the imaging guide is satisfied, without the patient having to operate the imaging interface of the user device.

[0647] Step S2050 of checking the captured image and sending the captured image to the analysis server may be a step of checking whether the captured facial image is a preferred facial image for use as an analysis image, and then sending the captured facial image to the analysis server if it is determined to be a preferred facial image.

[0648] The patient may determine whether the captured facial image is a preferred facial image to be used as an analysis image. For example, the patient may review the captured facial image through the user device and determine whether the angle of the face is off, whether the facial image is displayed blurred, and / or whether the eye area is not completely displayed in the facial image. Factors considered to determine whether the captured facial image is a preferred facial image to be used as an analysis image are not limited to the examples described above. The patient may determine whether the captured facial image is a preferred facial image to be used as an analysis image by considering whether the accuracy of the analysis results is expected to be low when performing image analysis using the captured facial image.

[0649] If the patient determines that the facial image is not a preferred facial image to be used as the analysis image, the patient may recapture the facial image. In this case, the user device may perform an operation to recapture the facial image. For example, but not limited to, the user device may delete the captured facial image and display the facial photography guide again.

[0650] If the patient determines that the captured facial image is a preferred facial image to be used as the analysis image, the captured facial image may be transmitted to the analysis server via the user device.

[0651] Meanwhile, the user device may also determine whether the captured facial image is a preferred facial image to be used as an analysis image. For example, even if a facial image is captured after the shooting guide is deemed to be satisfied, it may take some time until the actual facial image is captured, so the user device may recheck whether the captured facial image satisfies the shooting guide, but is not limited to this.

[0652] In this case, if the user device determines that the captured facial image is a preferred facial image to be used as an analysis image, the user device may transmit the captured facial image to the analysis server.

[0653]

[0654] 9. Display of face image comparison data

[0655] The acquired facial images of the patient can be pre-processed and displayed so that they can be easily compared with each other.

[0656] Below, a method for displaying a patient's facial image is described.

[0657]

[0658] FIG. 21 is a diagram illustrating adjustment of a patient's facial image, according to an exemplary embodiment.

[0659] 21 , facial images 2110, 2120, and 2130 acquired from a patient may each show the patient's eyes. Because the facial images 2110, 2120, and 2130 acquired from a patient are facial images captured by the patient using their user device, the eye sizes and pupil positions shown in each of the facial images 2110, 2120, and 2130 may differ from each other.

[0660] All of the reference pupil positions 2111, 2121, and 2131 are illustrated together on the facial images 2110, 2120, and 2130, respectively, acquired from the patient in Figure 21. However, this is illustrated to show that the eye sizes and pupil positions in each of the facial images 2110, 2120, and 2130 are different from each other, and when the facial images 2110, 2120, and 2130 acquired from the patient are displayed on a user device and / or medical staff device, the reference pupil positions 2111, 2121, and 2131 may not be illustrated together.

[0661] Each of the reference pupil positions 2111, 2121, and 2131 may be at a position having fixed coordinates relative to the frame in which the facial image is displayed. Specifically, each of the reference pupil positions on the frame in which the facial image is displayed may be at a position whose coordinates are fixed regardless of the actual position of the pupil appearing on the displayed facial image.

[0662] Referring to FIG. 21, for each of facial images 2110, 2120 and 2130 obtained from a patient, the size and / or position of facial images 2110, 2120 and 2130 may be adjusted so that pupils 2112, 2122 and 2132 on facial images 2110, 2120 and 2130 are located at reference pupil positions 2111, 2121 and 2131, respectively.

[0663] For example, the first face image 2110 may be adjusted in size and / or position such that the pupil 2112 on the first face image 2110 acquired from the patient is located at the reference pupil position 2111. The first adjusted face image 2140 may be acquired by adjusting the size and / or position of the first face image 2110. That is, the pupil position 2141 shown on the first adjusted face image 2140 may correspond to the reference pupil position.

[0664] Additionally, the second face image 2120 may be adjusted in size and / or position such that the pupil 2122 on the second face image 2120 acquired from the patient is located at the reference pupil position 2121. The second adjusted face image 2150 may be acquired by adjusting the size and / or position of the second face image 2120. That is, the pupil position 2151 shown on the second adjusted face image 2150 may correspond to the reference pupil position.

[0665] Additionally, the third face image 2130 may be adjusted in size and / or position such that the pupil 2132 on the third face image 2130 acquired from the patient is located at the reference pupil position 2131. The third adjusted face image 2160 may be acquired by adjusting the size and / or position of the third face image 2130. That is, the pupil position 2161 shown on the third adjusted face image 2160 may correspond to the reference pupil position.

[0666] To allow the acquired facial images to be easily compared with each other, adjusted facial images 2140, 2150, and 2160 may be displayed such that the positions of the pupils shown in acquired facial images 2110, 2120, and 2130 are adjusted to the reference pupil positions.

[0667] For example, adjusted facial images 2140, 2150, and 2160 may be displayed sequentially starting from the adjusted facial image obtained from the facial image corresponding to the earliest time point and ending with the adjusted facial image obtained from the facial image corresponding to the latest time point. Because the positions of the pupils in adjusted facial images 2140, 2150, and 2160 are identical, patients and / or medical staff may more intuitively recognize how the facial state changes over time when adjusted facial images 2140, 2150, and 2160 are displayed sequentially.

[0668] As another example, two adjusted face images including an adjusted face image acquired from a face image corresponding to the earliest time point among the adjusted face images and an adjusted face image acquired from a face image corresponding to the latest time point may be displayed sequentially. This is not limiting, and the two adjusted face images may be displayed overlapping each other.

[0669] As a specific example, two adjusted face images may be displayed, including an adjusted face image acquired from a face image corresponding to the start of treatment and an adjusted face image acquired from a face image corresponding to the current time.

[0670] Since the two adjusted facial images, including the adjusted facial image obtained from the facial image corresponding to the earliest time point and the adjusted facial image obtained from the facial image corresponding to the latest time point, are displayed with the same pupil position, patients and / or medical staff can more intuitively recognize how the facial condition has changed between specific time points.

[0671]

[0672] FIG. 22 is a diagram illustrating a method for displaying a patient's facial image, according to an exemplary embodiment.

[0673] Referring to FIG. 22, eye contours 2213, 2223 and 2233 can be obtained from adjusted face images 2211, 2221 and 2231 in which the respective pupil positions 2212, 2222 and 2232 are adjusted to the reference pupil positions.

[0674] In this case, the contour of the eyeball may refer to the boundary between the eyeball and the eyelid. The contour of the eyeball may be obtained based on image segmentation, and the specific details have been described above in the method for determining exophthalmos based on a face image, so a duplicate description will be omitted.

[0675] 22 , a first eyeball contour 2213 may be obtained from a first adjusted face image 2211 in which the pupil position is adjusted to a reference pupil position 2212. In addition, a second eyeball contour 2223 may be obtained from a second adjusted face image 2221 in which the pupil position is adjusted to a reference pupil position 2222. In addition, a third eyeball contour 2233 may be obtained from a third adjusted face image 2231 in which the pupil position is adjusted to a reference pupil position 2232.

[0676] To allow the acquired facial images to be easily compared with each other, the acquired eye contours 2213, 2223 and 2233 may be displayed superimposed on each other.

[0677] Specifically, as shown in FIG. 22, eyeball contours 2213, 2223, and 2233 obtained from face images corresponding to a plurality of time points can be displayed in a state 2243 where they overlap each other on a single image 2241.

[0678] The eye contours 2213, 2223 and 2233 are displayed in an overlapping state 2243 based on the reference pupil position 2242, so that the patient and / or medical staff can more intuitively recognize how the facial condition has changed.

[0679] The acquired eye contours 2213, 2223, and 2233 may be displayed sequentially, starting from the eye contour acquired from the face image corresponding to the earliest time point and ending with the eye contour acquired from the face image corresponding to the most recent time point. Because the eye contours 2213, 2223, and 2233 are aligned based on the reference pupil position, patients and / or medical staff may more intuitively recognize how the facial state changes over time when the eye contours 2213, 2223, and 2233 are displayed sequentially.

[0680] The obtained eye contours may be displayed in the adjusted face image.

[0681] For example, a first eyeball outline 2213 may be displayed in the first adjusted face image 2211, a second eyeball outline 2223 may be displayed in the second adjusted face image 2221, and a third eyeball outline 2233 may be displayed in the third adjusted face image 2231.

[0682] Without being limited to this, eyeball contours obtained from face images at multiple points in time may be displayed in one face image corresponding to one point in time.

[0683] For example, the first eyeball outline 2213 and the second eyeball outline 2223 may be displayed in the first adjusted face image 2211 or the second adjusted face image 2221, and the first eyeball outline 2213 and the third eyeball outline 2233 may be displayed in the first adjusted face image 2211 or the third adjusted face image 2231.

[0684] Therefore, the current eye contour and the previous eye contour can be displayed together on the current facial image, so that the patient and / or medical staff can more intuitively recognize how the facial condition has changed over a specific period of time.

[0685]

[0686] The patient's eye contours obtained from the patient's face image and the average eye contours of the group to which the patient belongs can be displayed together with the patient's face image, where the patient's eye contours and the average eye contours of the group to which the patient belongs can be displayed on the patient's face image based on the same reference pupil position.

[0687] The groups to which patients belong may be distinguished by gender and / or age. For example, patients of the same gender may be recognized as belonging to the same group. As another example, patients in the same age range of a pre-set age range may be recognized as belonging to the same group. As yet another example, patients of the same race may be recognized as belonging to the same group. The groups to which patients belong are not limited to the examples described above.

[0688] The acquired eye contour of the patient and the average eye contour of the group to which the patient belongs can be displayed together on the patient's facial image, so that the patient and / or medical staff can intuitively recognize how different the patient's facial condition is compared to the average.

[0689] The patient's eye contour obtained from the patient's face image and an area corresponding to the deviation between the eye contour of the group to which the patient belongs can be displayed together on the patient's face image, where the patient's eye contour and an area corresponding to the deviation between the eye contour of the group to which the patient belongs can be displayed on the patient's face image based on the same reference pupil position.

[0690] The area corresponding to the deviation between the patient's acquired eye contour and the eye contour of the group to which the patient belongs can be displayed together on the patient's facial image, so that the patient and / or medical staff can more intuitively recognize the level at which the patient's facial condition is located.

[0691]

[0692] 10. Clinical Trial Monitoring

[0693] In a clinical trial, a therapeutic agent may be administered to each of multiple patients, and the effect of the therapeutic agent on the multiple patients may be monitored over a monitoring period.

[0694] In existing clinical trials, patient data can only be obtained when patients visit the hospital, i.e., only limited clinical trial data can be obtained, and therefore existing clinical trial results can only be used to a limited extent for drug research and / or development.

[0695] Therefore, there is a need for monitoring methods that will obtain more data in clinical trials.

[0696] FIG. 23 illustrates a method for monitoring the effectiveness of a therapeutic drug in a clinical trial, according to an exemplary embodiment.

[0697] 23 , each of a plurality of patients 2302 may be administered a therapeutic drug when visiting a hospital 2301. Specifically, when each of the plurality of patients 2302 visits the hospital 2301, medical staff assigned to the hospital 2301 may administer the therapeutic drug to each of the plurality of patients 2302. Hereinafter, the process of administering a therapeutic drug to a patient when visiting the hospital may be understood to be performed by medical staff assigned to the hospital.

[0698] The therapeutic agent may be a therapeutic agent intended for clinical trials. For example, the therapeutic agent may be a therapeutic agent intended for the treatment of thyroid eye disease, and the clinical trial may be a trial to verify whether the therapeutic agent is effective in treating thyroid eye disease. In this case, the therapeutic effect of thyroid eye disease may be relief of exophthalmos, improvement of CAS scores, and / or relief of diplopia. On the other hand, the therapeutic agent may be a therapeutic agent that requires multiple administrations during the clinical trial period.

[0699] Hospital 2301 may refer to a hospital where medical staff are assigned, while actions performed by hospital 2301 may be understood to be performed by medical staff assigned to the hospital, a hospital server, a medical staff server, and / or a medical staff device, and redundant descriptions will be omitted below.

[0700] The plurality of patients 2302 may be subjects of a clinical trial, i.e., each patient is administered a placebo or a therapeutic drug for the treatment of thyroid eye disease. Hereinafter, each patient administered a placebo may also undergo the same monitoring as each patient administered a therapeutic drug, and therefore, each patient administered a therapeutic drug may include each patient administered a placebo and each patient administered a therapeutic drug for the treatment of thyroid eye disease. Meanwhile, hereinafter, actions performed by the patient 2302 may be understood to be performed by the patient and / or the patient's user device, and redundant descriptions will be omitted below.

[0701] 23 , each of a plurality of patients 2302 may receive an administration 2311 of a first therapeutic agent during a first visit 2310 to a hospital 2301, and the hospital 2301 may acquire actual patient data 2312 for each of the plurality of patients 2302. Specifically, when each of the plurality of patients 2302 makes a first visit 2310 to the hospital 2301, medical staff assigned to the hospital 2301 may administer the first therapeutic agent to each of the plurality of patients 2302, and the medical staff assigned to the hospital 2301 may perform acquiring 2312 actual patient data from each of the plurality of patients 2302. Below, the process of the hospital acquiring patient data when each patient visits the hospital may be understood to be performed by the medical staff assigned to the hospital.

[0702] In addition, the hospital 2301 may execute sending 2313 the acquired patient data to the analysis server 2303. Specifically, the hospital server, a medical staff server and / or a medical staff device installed in the hospital 2301 may execute sending 513 the patient data to the analysis server 2303. Hereinafter, the process of the hospital sending data to the analysis server may be understood to be executed by the hospital server, a medical staff server and / or a medical staff device installed in the hospital.

[0703] The analysis server 2303 may be a device that transmits and receives data between the hospital 2301 and the multiple patients 2302, obtains patient data for the multiple patients 2302, determines and / or estimates the status of each of the multiple patients 2302, and provides the determined and / or estimated patient data to the hospital 2301 and / or the multiple patients 2302.

[0704]

[0705] The patient data transmitted by the hospital 2301 to the analysis server 2303 may include exophthalmos measurements obtained from each of the multiple patients 2302, facial images of each patient at the time the exophthalmos are measured, thyroid dysfunction management history, thyroid eye disease treatment information, patient physical information, patient health information, and the like.

[0706] Each exophthalmometry measurement may be an actual exophthalmometry value obtained directly from the patient by medical staff, or may be, but is not limited to, an exophthalmometry value estimated from a facial image of the patient captured during the patient's hospital visit.

[0707] The patient's thyroid dysfunction management history may include, but is not limited to: the diagnosis of the thyroid dysfunction, the time of the thyroid dysfunction diagnosis, blood test results, information about surgeries and / or procedures resulting from the thyroid dysfunction, and the type, dosage, and / or duration of medication used to treat the thyroid dysfunction.

[0708] Thyroid dysfunction may refer to, but should not be limited to, hypothyroidism, and may be understood to include symptoms related to thyroid dysfunction. Specifically, symptoms related to thyroid dysfunction may include, but are not limited to, hypothyroidism, thyroiditis, and / or thyroid nodules, examples of which are described herein.

[0709] Blood test results may include hormone and antibody levels.

[0710] The hormone levels can be values for hormones related to hyperthyroidism, for example, hormones related to hyperthyroidism can include, but are not limited to, free T4, TSH, free T3, and / or total T3.

[0711] The antibody level can be a value for antibodies related to hyperthyroidism. For example, antibodies related to hyperthyroidism can include, but are not limited to, anti-TSH receptor Ab, anti-TPO Ab, and / or anti-Tg Ab.

[0712] Additionally, blood test results may include thyroglobulin (TG) and thyroxine-binding globulin (TBG) levels.

[0713] The thyroid eye disease treatment information may include, but is not limited to, the type, dosage, and duration of treatment administered to treat the thyroid eye disease, the date of steroid prescription; the amount of steroid prescription, the date of radiation therapy, the date of thyroid eye disease surgery, and / or the date of triamcinolone administration.

[0714] Patient physical information and patient health information may include, but is not limited to, information based on the patient's physical characteristics, such as the patient's age, sex, race, and weight.

[0715] On the other hand, while FIG. 23 shows that the hospital 2301 performs the transmission 2313 of patient data to the analysis server 2303 after administering 2311 the therapeutic drug to each of the multiple patients 2302, the hospital 2301 may also transmit the patient data to the analysis server 2303 before administering the therapeutic drug to each patient 2302, without being limited thereto.

[0716]

[0717] Referring to FIG. 23, patient monitoring may be performed at each second monitoring time point 2320.

[0718] According to an exemplary embodiment, the second monitoring time point 2320 may be a time point during a period when each of the plurality of patients 2302 visits the hospital 2301 to administer a therapeutic agent. Specifically, the second monitoring time point 2320 may be a time point during a clinical trial when each of the plurality of patients 2302 visits the hospital 2301 to administer any therapeutic agent that requires multiple doses.

[0719] Additionally, the second monitoring time point 2320 may be a time point within a clinical trial where each of multiple patients 2302 visits the hospital 2301 for status observation after receiving all of the therapeutic drugs that require multiple doses.

[0720] Additionally, the second monitoring time point 2320 may be a time point within a period that further includes a period of time after each of the plurality of patients 2302 has been administered all of the therapeutic agents requiring multiple doses, in this case, the period of time may be, but is not limited to, one year.

[0721] The second monitoring time point 2320 may be within a period that further includes a period of time after all therapeutic agent has been administered, so that it can be determined whether therapeutic effect has disappeared and / or whether side effects have occurred after therapeutic agent administration has ceased, where the period of time may be determined according to a clinical trial design.

[0722] The second monitoring time point 2320 according to an exemplary embodiment may be determined based on a pre-established clinical trial monitoring cycle.

[0723] In this case, the clinical trial monitoring cycle can be set according to the clinical trial design.

[0724] For example, the clinical trial monitoring cycle can be set according to how much clinical trial data needs to be obtained, specifically, how much clinical trial data a pharmaceutical company of a therapeutic drug needs to obtain and how much clinical trial data needs to be obtained during the clinical trial.

[0725] As another example, the clinical trial monitoring cycle can be set to perform monitoring at set times during the administration of the therapeutic agent according to the clinical trial design. For example, if the therapeutic agent is administered every three weeks, the clinical trial monitoring cycle can be set to perform monitoring every one week, but is not limited thereto.

[0726] On the other hand, the clinical trial monitoring cycle may be set so that monitoring is performed at a specific interval. For example, the clinical trial monitoring cycle may be set so that monitoring is performed every two days, but is not limited thereto, and the specific period may be freely determined.

[0727] Meanwhile, the specific actions taken at the second monitoring time point 2320 will be described after describing FIG.

[0728] FIG. 24 is a diagram illustrating an overall method for monitoring the effectiveness of therapeutic drugs in clinical trials, according to an exemplary embodiment.

[0729]

[0730] 24 , each of a plurality of patients 2402 may be administered a therapeutic drug when visiting a hospital 2401. Specifically, when each of the plurality of patients 2402 visits the hospital 2401, medical staff assigned to the hospital 2401 may administer the therapeutic drug to each of the plurality of patients 2402. Hereinafter, it may be understood that the process of administering the therapeutic drug to each patient when visiting the hospital is performed by medical staff assigned to the hospital.

[0731] The therapeutic agent may be a therapeutic agent for which a clinical trial is being conducted. For example, the therapeutic agent may be a therapeutic agent intended to treat thyroid eye disease, and the clinical trial may be a trial to verify whether the therapeutic agent is effective in treating thyroid eye disease. The dosing regimen may be determined based on the clinical trial design.

[0732] On the other hand, the details described above for the hospital 2301, multiple patients 2302, and analysis server 2303 in Figure 23 can also be applied to the hospital 2401, multiple patients 2402, and analysis server 2403 in Figure 24, so duplicate descriptions will be omitted.

[0733] 24 , each of a plurality of patients 2402 may receive administration 2411 of a first therapeutic agent at time 2410 of their first visit to the hospital 2401, and the hospital 2401 may execute acquiring 2412 actual patient data from each of the plurality of patients 2402. Specifically, when each of the plurality of patients 2402 makes their first visit to the hospital 2401 at time 2410, medical staff assigned to the hospital 2401 may administer the first therapeutic agent to each of the plurality of patients 2402, and the medical staff assigned to the hospital 2401 may execute acquiring 2412 actual patient data from each patient 2402. Below, the process of the hospital acquiring patient data when a patient visits the hospital may be understood to be performed by medical staff assigned to the hospital.

[0734] In addition, the hospital 2401 may execute sending 2413 of the acquired patient data to the analysis server 2403. Specifically, a hospital server, a medical staff server, and / or a medical staff device installed in the hospital 2401 may execute sending 2413 of the patient data to the analysis server 2403. Hereinafter, the process of the hospital sending data to the analysis server may be understood to be executed by the hospital server, the medical staff server, and / or the medical staff device installed in the hospital.

[0735] For details regarding the hospital 2401 transmitting patient data to the analysis server 2403, the details described above in the description of the hospital 2401 transmitting patient data to the analysis server 2403 2313 in FIG. 23 may apply, and duplicate descriptions will be omitted.

[0736] 24, monitoring of each of a plurality of patients may be performed at a second monitoring time point 2414 administered a first time after the first hospital visit time point 2410, and patient monitoring may be performed at a second monitoring time point 2415 administered a second time after the first monitoring time point 2414. With regard to determining the second monitoring time point, the details described above in the description of determining the second monitoring time point 2320 in FIG. 23 may apply, and therefore redundant description will be omitted.

[0737] Referring to FIG. 24, after the second monitoring time point 2415 is administered for the second time, each of the multiple patients 2402 may make a second visit 2420 to the hospital 2401 and receive a second therapeutic drug 2421, and the hospital 2401 may obtain actual patient data 2422 from each of the multiple patients 2402.

[0738] The items of measured patient data acquired at the time of the second clinic visit 2420 and the items of measured patient data acquired at the time of the first clinic visit 2410 may be identical to one another, but are not limited to such. Some of the data items may be identical, some may be different, or the data items may be different from one another. For example, the measured patient data acquired at the time of the first clinic visit 2410 may include exophthalmos measurements acquired from each of the plurality of patients 2402, facial images of the patients at the time of the exophthalmos measurements, thyroid dysfunction management history, patient physical information, and patient health information. The measured patient data acquired at the time of the second clinic visit 2420 may include only exophthalmos measurements acquired from each of the plurality of patients 2402 and facial images of the patients at the time of the exophthalmos measurements. That is, the thyroid dysfunction management history, patient physical information, and patient health information obtained at the time of the first hospital visit 2410 can be used as is as the thyroid dysfunction management history, patient physical information, and patient health information of each of the multiple patients 2402 at the time of the second hospital visit 2420, and the data items are not limited to the examples described above.

[0739] The hospital 2401 may send 2423 the acquired patient data to the analysis server 2403.

[0740] The patient data transmitted by the hospital 2401 to the analytic server 2403 at the time of the second hospital visit 2420 and the patient data transmitted by the hospital 2401 to the analytic server 2403 at the time of the first hospital visit 2410 may be identical to one another, but are not limited to such. Some of the data items may be identical, some may be different, or the data items may be different from one another. For example, the patient data transmitted by the hospital 2401 to the analytic server 2403 at the time of the first hospital visit 2410 may include exophthalmos measurements obtained from each of the multiple patients 2402, facial images of the patients at the time of the exophthalmos measurements, thyroid dysfunction management history, patient physical information, and patient health information. The patient data transmitted by the hospital 2401 to the analytic server 2403 at the time of the second hospital visit 620 may include only the exophthalmos measurements obtained from each of the multiple patients 2402 and facial images of the patients at the time of the exophthalmos measurements. That is, the analysis server 2403 can directly use the thyroid dysfunction management history, patient physical information, and patient health information obtained at the time of the first hospital visit 2410 as the thyroid dysfunction management history, patient physical information, and patient health information of each of the multiple patients 2402 corresponding to the time of the second hospital visit 2420, and the data items are not limited to the examples described above.

[0741] 24 illustrates two second monitoring time points 2414 and 2415 between the first hospital visit time point 2410 and the second hospital visit time point 2420, the number of second monitoring time points is not limited thereto, and the second monitoring time points may be located at various times based on the set monitoring cycle. For example, there may be one second monitoring time point between the first hospital visit time point 2410 and the second hospital visit time point 2420, or there may be three or more second monitoring time points.

[0742] Referring to FIG. 24, after the second hospital visit time 2420, patient monitoring may be performed at a second monitoring time 2424 given for a third time, and patient monitoring may be performed at a second monitoring time 2425 given for a fourth time.

[0743] Thereafter, the clinic visit and each second monitoring may be performed repeatedly.

[0744]

[0745] Referring again to FIG. 23, at a second monitoring time point 2320, each of the plurality of patients 2302 may be asked to capture a facial image and complete a questionnaire.

[0746] In this case, the subject being requested to capture a facial image and complete a questionnaire may be the user device of each of the plurality of patients 2303. Hereinafter, the process of requesting a patient to capture a facial image and complete a questionnaire may be understood to be performed on the user device.

[0747]

[0748] Each of the plurality of patients 2302 may capture a facial image and transmit the facial image in response to a facial image capture request. For example, each of the plurality of patients 2302 may capture a facial image in response to a facial image capture request 2321 from the analysis server 2303 and transmit the facial image to the analysis server 2303.

[0749] In this case, the facial image may refer to, but is not limited to, a frontal facial image and / or a profile facial image of the patient, and the facial image may include: a panoramic image from the front to the side of the patient; a video captured by recording the face; and / or a facial image at any angle between the front and the side of the patient. In a more specific example, each of the multiple patients 2302 may use a user device to capture a facial image in response to a facial image capture request 2321 from the analysis server 2303, and transmit 2322 the facial image to the analysis server 2303 through the user device. In the following, the process of a patient capturing a facial image and transmitting the facial image to the analysis server may be understood to be performed by using the user device.

[0750] On the other hand, if the patient's face rotates left and right and / or up and down when capturing a facial image, distortion may occur in the facial image. If distortion occurs in the facial image, the accuracy of facial image analysis may decrease. Therefore, in order to obtain a preferable facial image, each of the multiple patients 2302 may be provided with a shooting guide. By capturing a facial image following the shooting guide, the patient 402 may capture a facial image with the same composition each time the facial image is captured. Specific details regarding the shooting guide are described in 8. Capturing and Sending Facial Images, so duplicate description will be omitted.

[0751] On the other hand, since facial appearance may change depending on the time of day, each of the multiple patients 2302 may also receive a facial image capture request at a pre-set image capture time.

[0752] On the other hand, because facial appearance may change depending on the time of day, each of the multiple patients 2302 may be required to capture their own facial images at multiple different times throughout the day. In this case, an analysis result for each of the acquired facial images may be obtained, and an average value or the like of the obtained analysis results may be determined as the correction value for that day. Alternatively, the analysis result with the highest accuracy among the analysis results for each of the acquired facial images may also be determined as the value for that day. Alternatively, the analysis result of the facial image that best satisfies the imaging guide among the acquired facial images may also be determined as the value for that day.

[0753] On the other hand, each of the multiple patients 502 may also receive a facial image capture request to capture multiple facial images at the time of facial image capture. In this case, an analysis result for each of the acquired facial images may be acquired, and the average, median, or similar value of the acquired analysis results may be determined as the correction value at that time. Alternatively, the analysis result with the highest accuracy among the analysis results for each of the acquired facial images may also be determined as the value at that time. Alternatively, the analysis result of the facial image that best satisfies the imaging guide among the acquired facial images may also be determined as the value at that time.

[0754] On the other hand, each of the multiple patients 2302 may also receive a facial image capture request to capture a facial video at the time of facial image capture. In this case, an analysis result for each frame included in the acquired facial video may be acquired, and the average, median, or similar of the acquired analysis results may be determined as the correction value at that time. Alternatively, the analysis result with the highest accuracy among the analysis results for each frame included in the acquired facial video may also be determined as the value at that time. Alternatively, the analysis result for the frame included in the acquired facial video that best satisfies the imaging guide may also be determined as the value at that time.

[0755] As described above, as each of multiple patients 2302 receives a facial image capture request, the effects of variations in the patient's facial appearance, variations that occur depending on the degree of force the patient applies to their face during facial image capture, the patient's condition, the patient's intent, and the like, can be reduced.

[0756]

[0757] Each of the multiple patients 2302 may fill out the questionnaire content and send the questionnaire content in response to the questionnaire content request. For example, each of the multiple patients 2302 may fill out the questionnaire content in response to a questionnaire content request 2321 from the analysis server 2303 and execute transmission 2322 of the questionnaire content to the analysis server 2303. In a more specific example, each of the multiple patients 2302 may fill out the questionnaire content by using a user device in response to the questionnaire content request 2321 from the analysis server 2303 and send the written questionnaire content to the analysis server 2303 through the user device. It will be understood below that the process of a patient filling out the questionnaire content and sending the questionnaire content to the analysis server is executed by using a user device.

[0758] When completing the survey, the patient may enter text into the user device and / or check a separate check box for pre-defined survey items. In addition to the pre-defined survey items, the patient may also enter the survey information they wish to send to the analysis server 2303 and / or the hospital 2301 into the user device.

[0759] The questionnaires required of the patient 2302 may include: a questionnaire regarding determining the activity of thyroid eye disease, and a questionnaire regarding determining the severity of thyroid eye disease.

[0760] For example, questionnaire questions related to determining the activity of thyroid eye disease may include, but are not limited to, whether there is spontaneous pain in the back of the eye and whether there is pain during eye movement.

[0761] For example, questionnaire content relevant to determining the severity of thyroid eye disease may include, but is not limited to, whether diplopia is present and a quality of life questionnaire (Go-QoL).

[0762] Additionally, the questionnaires required of each of the plurality of patients may include a questionnaire about signs and / or symptoms associated with side effects.

[0763] To this end, each of the plurality of patients 2302 may be provided with information regarding side effects before being required to complete a side effect survey.

[0764] Information regarding side effects may include, but is not limited to, information regarding the likelihood of occurrence of whether muscle spasms are present, whether nausea is present, whether hair loss is present, whether diarrhea is present, whether fatigue is present, and / or whether hyperglycemia is present.

[0765]

[0766] Based on the facial images and questionnaire survey contents obtained from each of the multiple patients 2302, personalized estimates can be obtained for each of the multiple patients 2302 regarding information that is desired to be proven as the effectiveness of a therapeutic drug through clinical trials.

[0767] 23 , the analysis server 2303 may perform the following: determining 2323 a personalized estimate for each of the plurality of patients 2302 regarding information that will be proven as the effectiveness of a therapeutic drug through a clinical trial, by using facial images and questionnaire content obtained from each of the plurality of patients 2302. More specifically, the analysis server 2303 may perform the following: determining 2323 a personalized estimate for each of the plurality of patients 2302 regarding information that will be proven as the effectiveness of a therapeutic drug through a clinical trial of a therapeutic drug intended to treat thyroid eye disease, by using facial images and questionnaire content obtained from the plurality of patients 2302. Hereinafter, the facial images and questionnaire content obtained from the patients may be understood as the facial images and questionnaire content received from each patient's user device.

[0768] Information that may be desired to prove as the effectiveness of a therapeutic drug through a clinical trial may include a reduction in exophthalmos, an improvement in CAS scores, and / or an improvement in diplopia. Thus, the personalized estimate for each of the plurality of patients 2302 may include exophthalmos information, CAS information, and / or diplopia information.

[0769] Without being limited thereto, the personalized estimate for each of the plurality of patients 2302 may include information regarding eyelid retraction and / or information regarding the severity of thyroid eye disease.

[0770]

[0771] The numerical value shown in the facial image acquired from each of the plurality of patients 2302 may be obtained as a personalized estimate of the exophthalmos information of each of the plurality of patients 2302. For example, the analysis server 2303 may determine the exophthalmos numerical value shown in the facial image acquired from each of the plurality of patients 2302 as a personalized estimate of the exophthalmos information for each of the plurality of patients 2302. Specific methods for determining the exophthalmos numerical value by using facial images have been described above in 4. Treatment Process Monitoring and 5. Exophthalmos Determination Method Based on Facial Images, and therefore, duplicate descriptions will be omitted.

[0772] Meanwhile, a trend in exophthalmos may be obtained based on facial images obtained from each of the plurality of patients 2302 as a personalized estimate of exophthalmos information for each of the plurality of patients 2302. For example, the analysis server 2303 may determine a trend in exophthalmos based on facial images obtained from each of the plurality of patients 2302 as a personalized estimate of exophthalmos information for each of the plurality of patients 2302. Specific methods for determining exophthalmos numerical values by using facial images have been described above in 4. Treatment Process Monitoring and 6. Exophthalmos Trend Determination Method Based on Facial Images, and therefore, repeated description will be omitted.

[0773]

[0774] As a personalized estimate of the CAS information for each of the plurality of patients 2302, the CAS numerical value may be obtained based on the facial image and questionnaire content obtained from each of the plurality of patients 2302. For example, the analysis server 2303 may determine the CAS numerical value by using the facial image and questionnaire content obtained from each of the plurality of patients 2302 as a personalized estimate of the CAS information for each of the plurality of patients 2302. Specific methods for determining the CAS numerical value based on the facial image and questionnaire content have been described in 2. Thyroid Eye Disease Activity and 4. Treatment Process Monitoring, so duplicate descriptions will be omitted.

[0775] On the other hand, as a personalized estimate of the CAS information for each of the multiple patients 2302, a trend in CAS can be obtained based on facial images and questionnaire survey contents obtained from each of the multiple patients 2302. For example, the analysis server 2303 can determine a trend in CAS by using the facial images and questionnaire survey contents obtained from each of the multiple patients 2302 as a personalized estimate of the CAS information for each of the multiple patients 2302. Specific methods for determining a trend in CAS based on facial images and questionnaire survey contents have been described above in 2. Thyroid Eye Disease Activity and 4. Treatment Process Monitoring, so duplicated descriptions will be omitted.

[0776]

[0777] As a personalized estimate of the diplopia information for each of the plurality of patients 2302, the diplopia presence / absence value may be obtained based on the questionnaire survey content obtained from each of the plurality of patients 2302. For example, the analysis server 2303 may obtain the diplopia presence / absence value by using the questionnaire survey content obtained from each of the plurality of patients 2302 as a personalized estimate of the diplopia information for each of the plurality of patients 2302.

[0778] The diplopia presence / absence value may be a value indicating either the presence or absence of diplopia, whereas when determining the presence or absence of diplopia by using the Gorman criteria, the diplopia presence / absence value may be a value indicating one of the values for no diplopia, intermittent diplopia, diplopia at extreme gaze, and persistent diplopia.

[0779] The specific method for obtaining a diplopia presence / absence determination value based on the questionnaire survey contents has been described in 4. Treatment Process Monitoring, so duplicate descriptions will be omitted.

[0780] On the other hand, a trend in diplopia may be obtained based on questionnaire survey content obtained from each of the plurality of patients 2302 as a personalized estimate of diplopia information for each of the plurality of patients 2302. For example, the analysis server 2302 may determine a trend in diplopia based on questionnaire survey content obtained from each of the plurality of patients 2302 as a personalized estimate of diplopia information for each of the plurality of patients 2302. A specific method for determining a trend in diplopia based on questionnaire survey content has been described in 4. Treatment Process Monitoring, so a duplicated description will be omitted.

[0781]

[0782] A personalized estimate of eyelid retraction information for each of the plurality of patients 2302 may be obtained based on facial images obtained from each of the plurality of patients 2302. For example, the analysis server 2303 may determine a personalized estimate of eyelid retraction information for each of the plurality of patients 2302 by using facial images obtained from each of the plurality of patients 2302.

[0783] A numerical value of eyelid retraction indicated in a facial image acquired from each of the plurality of patients 2302 may be obtained as a personalized estimate of eyelid retraction information for each of the plurality of patients 2302. For example, the analysis server 2303 may determine a numerical value of eyelid retraction indicated in a facial image acquired from each of the plurality of patients 2302 as a personalized estimate of eyelid retraction information for each of the plurality of patients 2302. A specific method for determining an eyelid retraction numerical value by using a facial image has been described above in 7. Eyelid Retraction Determination Method Based on Facial Image, so a duplicated description will be omitted.

[0784] On the other hand, a trend in eyelid retraction may be obtained based on facial images obtained from each of the plurality of patients 2302 as a personalized estimate of eyelid retraction information for each of the plurality of patients 2302. For example, the analysis server 2303 may determine a trend in eyelid retraction based on facial images obtained from each of the plurality of patients 2302 as a personalized estimate of eyelid retraction information for each of the plurality of patients 2302. Specific methods for determining a trend in eyelid retraction by using facial images have been described in 4. Treatment Process Monitoring and 7. Eyelid Retraction Determination Method Based on Facial Images, so duplicated descriptions will be omitted.

[0785]

[0786] A personalized estimate of thyroid eye disease severity information for each of the plurality of patients 2302 may be obtained based on facial images and questionnaire survey contents obtained from each of the plurality of patients 2302. For example, the analysis server 2303 may determine a personalized estimate of thyroid eye disease severity information for each of the plurality of patients 2302 by using facial images and questionnaire survey contents obtained from each of the plurality of patients 2302. In this case, the thyroid eye disease severity information may include, but is not limited to, none, mild (moderate), and severe (severe) severity of thyroid eye disease. Specific methods for obtaining a personalized estimate of thyroid eye disease severity based on facial images and questionnaire survey contents have been described in 3. Severity of Thyroid Eye Disease and 4. Treatment Process Monitoring, and therefore, duplicate descriptions will be omitted.

[0787] On the other hand, a trend in the severity of thyroid eye disease may be obtained based on facial images and questionnaire survey contents obtained from each of the plurality of patients 2302 as a personalized estimate of the severity of thyroid eye disease for each of the plurality of patients 2302. For example, the analysis server 2303 may determine a trend in the severity of thyroid eye disease based on facial images and questionnaire survey contents obtained from each of the plurality of patients 2302 as a personalized estimate of the severity of thyroid eye disease for each of the plurality of patients 2302. Specific methods for determining a trend in the severity of thyroid eye disease based on facial images and questionnaire survey contents have been described in 3. Severity of Thyroid Eye Disease and 4. Treatment Process Monitoring, and therefore, duplicate descriptions will be omitted.

[0788]

[0789] Side effect occurrence information can be acquired for each of the multiple patients 2302 based on the contents of a questionnaire survey acquired from each of the multiple patients 2302. For example, the analysis server 2303 can determine whether a side effect will occur for each of the multiple patients 2302 by using the contents of the questionnaire survey acquired from each of the multiple patients 2302.

[0790] In this case, the pharmaceutical company may be notified of the side effect occurrence information if it is determined that a side effect has occurred in at least some of the multiple patients 2302. For example, if the analysis server 2303 determines that at least some of the multiple patients 2302 have experienced a side effect based on the determination of whether a side effect has occurred for each of the multiple patients 2302, the analysis server 2303 may transmit the side effect occurrence information to the pharmaceutical company server.

[0791]

[0792] According to an exemplary embodiment, clinical trial data may be generated based on the personalized estimates obtained for each of the plurality of patients. For example, analytic server 2303 may perform generating 2324 clinical trial data based on the personalized estimates obtained for each of the plurality of patients.

[0793] The clinical trial data may include time series data regarding average values for all of the multiple patients 2302 regarding the degree of exophthalmos, the CAS numerical value, whether the patient has diplopia, and / or whether there are side effects. In addition, the clinical trial data may include time series data for each of the multiple patients 2302 regarding the degree of exophthalmos, the CAS numerical value, whether the patient has diplopia, and / or whether there are side effects. Time series data may mean data arranged in a time series. Specific details regarding the time series data are described in 4. Treatment Process Monitoring, so duplicate description will be omitted.

[0794] In addition, the clinical trial data may include information regarding drug efficacy at each time point. Information regarding drug efficacy at each time point may refer to the number of patients who experience drug efficacy compared to the total number of patients at each time point. Whether drug efficacy is demonstrated can be determined based on information desired to be proven as the efficacy of a therapeutic drug through clinical trials. Specifically, it can be determined that drug efficacy is demonstrated when the efficacy of a therapeutic drug desired to be proven is demonstrated for patients through clinical trials.

[0795] In addition, the clinical trial data may include information regarding the incidence of side effects at each time point. Information regarding the incidence of side effects at each time point may refer to the number of patients who experienced side effects compared to the total number of patients at each time point. Whether a side effect has occurred may be determined based on information that is desired to be proven as a side effect of the therapeutic agent through the clinical trial. Specifically, if a patient experiences a side effect that is desired to be proven as a side effect of the therapeutic agent through the clinical trial, it may be determined that a side effect has occurred.

[0796]

[0797] By monitoring each of multiple patients, pharmaceutical companies can obtain much more data in clinical trials.

[0798] Pharmaceutical companies may use the clinical trial data obtained for drug research and / or drug development.

[0799] Pharmaceutical companies may use the obtained clinical trial data to determine appropriate dosages and / or administration regimens, such as administration cycles, for therapeutic agents.

[0800]

[0801] 11. Post-treatment monitoring

[0802] Even when a patient is administered a therapeutic drug intended to treat thyroid eye disease and the patient's symptoms are cured at the end of treatment, the patient's thyroid eye disease may recur after the end of treatment. Therefore, after treatment is completed, the patient's thyroid eye disease must be monitored to see if it worsens again.

[0803] FIG. 25 is a diagram illustrating a post-treatment monitoring method, according to an exemplary embodiment.

[0804] Referring to FIG. 25, patient 2502 may be a patient who visited hospital 2501, received all of the medications intended to treat thyroid eye disease, and then completed his or her treatment.

[0805] Hospital 2501 may refer to a hospital where medical staff are assigned, while actions performed by a hospital may be understood to be performed by medical staff assigned to the hospital, a hospital server, a medical staff server, and / or a medical staff device, and redundant descriptions will be omitted below.

[0806] The patient 2502 may be a patient with thyroid eye disease or may be a patient who has completed administration of a therapeutic drug intended to treat thyroid eye disease. Meanwhile, hereinafter, actions performed by the patient 2502 may be understood to be performed by the patient and / or the patient's user device, and redundant descriptions will be omitted below.

[0807] 25 , after the administration of the therapeutic drug is completed, the patient 2502 may make a final visit 2510 to the hospital 2501, and the hospital 2501 may execute acquisition 2511 of actual patient data from the patient 2502. Specifically, medical staff assigned to the hospital 2501 may execute acquisition 2511 of actual patient data from the patient 2502. In this case, the end of treatment may be understood as the time of the final visit to the hospital.

[0808] On the other hand, patient 2502 may also be administered a therapeutic drug at the time of his / her final visit to hospital 2501. Specifically, when patient 2502 visits hospital 2501, medical staff assigned to hospital 2501 may administer a therapeutic drug to patient 2502. In this case, the end of this treatment may be understood as the time when administration of the therapeutic drug is completed.

[0809] The hospital 2501 may execute transmission 2512 of the acquired patient data to the analysis server 2503. Specific details regarding the patient data transmitted from the hospital 2501 to the analysis server 2503 have been described above in 4. Treatment Process Monitoring, and therefore, duplicate description will be omitted.

[0810] The analysis server 2503 may be a device that performs data transmission and reception between the hospital 2501 and the patient 2502, acquires patient data about the patient 2502, determines and / or estimates the patient's condition, and provides the determined and / or estimated patient data to the hospital 2501 and / or the patient 2502. Without being limited thereto, the analysis server 2503 may be a device that stores patient data and / or information related to thyroid eye disease and the like acquired from the hospital 2501 and / or the patient 2502, and provides the stored information to the hospital 2501 and / or the patient 2502. Specific details of the data transmission and reception between the hospital 2501 and the patient 2502 performed by the analysis server 2503 have been described in 4. Treatment Process Monitoring, and therefore, a duplicate description will be omitted.

[0811]

[0812] Referring to FIG. 25, patient monitoring may be performed every third monitoring time point 2520 and 2530 .

[0813] According to an exemplary embodiment, the third monitoring time point 2520 and 2530 may be a time point after the end of the treatment period associated with the end of administration of the therapeutic agent to the patient 2502 .

[0814] According to an exemplary embodiment, the third monitoring time points 2520 and 2530 may be determined based on a set post-treatment monitoring cycle.

[0815] In this case, the post-treatment monitoring cycle may be set based on the characteristics of the therapeutic agent and / or the patient's condition.

[0816] For example, a treatment monitoring cycle may be set so that monitoring is performed at specific intervals.

[0817] As a specific example, the post-treatment monitoring cycle may be set so that monitoring is performed every 5 days, but is not limited thereto, and the specific period may be freely determined. Meanwhile, in this case, the post-treatment monitoring cycle may be longer than the treatment monitoring cycle described in 4. Treatment Process Monitoring.

[0818] In another example, since the time of symptom recurrence may vary for each therapeutic agent, a post-treatment monitoring cycle may be established by considering the time of symptom recurrence after the end of treatment. Alternatively, a post-treatment monitoring cycle may be established by considering the time of symptom recurrence after the end of treatment.

[0819] Specifically, the post-treatment monitoring cycle may be set to perform more frequent monitoring at a time point where the clinical trial results indicate a high recurrence rate of symptoms. Alternatively, the post-treatment monitoring cycle may be set to perform more frequent monitoring at intervals where the clinical trial results indicate a rapid increase in the recurrence rate of symptoms. For example, if a therapeutic drug is effective after the end of treatment, but the therapeutic drug's effectiveness wears off one year after the end of treatment and the recurrence rate of symptoms is high, the post-treatment monitoring cycle may be set to perform more frequent monitoring one year after the end of treatment, but is not limited to this.

[0820] On the other hand, the treatment monitoring cycle can also be divided into a monitoring cycle set by the hospital and a monitoring cycle desired by the patient.

[0821] Specifically, the monitoring cycle set by the hospital may be set according to the characteristics of the therapeutic drug and / or the patient's condition, as described above. The monitoring cycle desired by the patient is not limited to the monitoring cycle set by the hospital, but may be freely set according to the patient's needs.

[0822] For example, the monitoring cycle set by the hospital may be set to once every three weeks after the end of treatment, and the monitoring cycle desired by the patient may be set to once a week, but is not limited to this.

[0823] Alternatively, the monitoring cycle set by the hospital may be set according to the characteristics of the therapeutic drug and / or the patient's condition, as described above, but the patient-desired monitoring cycle may not be set separately, and monitoring may be performed arbitrarily at any time desired by the patient.

[0824] For example, if the monitoring cycle set by the hospital is set to once every three weeks after the end of treatment, monitoring will be performed according to the monitoring cycle set by the hospital, but additional monitoring may be performed at any time desired by the patient, including, but not limited to, this.

[0825] However, according to an exemplary embodiment, the third monitoring time point 2520 may also be determined at any time the patient 2502 desires to be monitored without setting up a separate post-treatment monitoring cycle.

[0826] In this case, the patient 2502 may send a monitoring request to the analysis server 2503 at any time, and the analysis server 2503 may determine the time at which it receives the monitoring request from the patient 2502 as the third monitoring time 2520, but is not limited to this.

[0827] Referring to FIG. 25, at a third monitoring time point 2520, the patient 2502 may be asked to capture a facial image and complete a questionnaire.

[0828] In this case, the object on which the patient is asked to capture a facial image and complete a survey may be the user device of the patient 2502. Hereinafter, the process of the patient being asked to capture a facial image and complete a survey may be understood to be performed on the user device.

[0829]

[0830] In response to a request to capture a facial image and fill out a questionnaire, the patient 2502 may capture a facial image, fill out the questionnaire, and transmit them. For example, in response to a request 2521 to capture a facial image and fill out a questionnaire, which is requested from the analysis server 2503, the patient 2502 may capture a facial image, fill out the questionnaire, and transmit 2522 the facial image and the questionnaire to the analysis server 2503. Details including the content related to the request to capture a facial image and fill out the questionnaire, the content related to capturing a facial image and filling out the questionnaire, and the content related to transmitting the facial image and the questionnaire have been described in 4. Treatment Process Monitoring, so duplicate description will be omitted.

[0831]

[0832] The patient's personalized estimate corresponding to the exophthalmos information may be obtained based on a facial image obtained from the patient 2502. For example, the analysis server 2503 may execute determining 2523 the patient's personalized estimate corresponding to the exophthalmos information by using a facial image obtained from the patient 2502. Specific methods for determining the patient's personalized estimate corresponding to the exophthalmos information are described in 4. Treatment Process Monitoring, 5. Exophthalmos Determination Method Based on Facial Images, and 6. Exophthalmos Trend Determination Method Based on Facial Images, and therefore, duplicate descriptions will be omitted.

[0833] The personalized estimate of the patient corresponding to the CAS information may be obtained based on the facial image and questionnaire content obtained from the patient 2502. For example, the analysis server 2503 may determine the personalized estimate of the patient corresponding to the CAS information by using the facial image and questionnaire content obtained from the patient 2502. A specific method for determining the personalized estimate of the patient corresponding to the CAS information has been described in 4. Treatment Process Monitoring, and therefore, a duplicate description will be omitted.

[0834]

[0835] The obtained personalized estimate for the patient may be displayed to the patient 2502 or provided to the hospital 2501.

[0836] Specifically, referring to FIG. 25 , the analysis server 2503 may perform the following: display 2524 to the patient 2502 a personalized estimate corresponding to the determined exophthalmos information; and provide 2525 to the hospital 2501 a personalized estimate corresponding to the determined exophthalmos information and the acquired patient data.

[0837] More specifically, the patient's personalized estimates may be provided and displayed on the patient's 2502 user device, and the patient's personalized estimates and / or patient data may be provided and displayed on the hospital's 2501 medical staff device and / or hospital server.

[0838] 25 illustrates that the patient's personalized estimate is displayed to the patient 2502 at the third monitoring time point 2520, the time at which the patient's personalized estimate is displayed to the patient 2502 is not limited to the third monitoring time point 2520. For example, the patient 2502 may view the patient's personalized estimate without capturing a facial image or filling out a questionnaire. Specifically, the patient's 2502 user device may display the patient's personalized estimate obtained from the analysis server 2503 at any time desired by the patient 2502.

[0839] 25, while hospital 2501 receives the patient's personalized estimate and patient data at third monitoring time point 2520, it will be explained that the time point at which hospital 2501 receives the patient's personalized estimate and / or patient data is not limited to third monitoring time point 2520. For example, hospital 2501 can be provided with the patient's personalized estimate and / or patient data even if patient 2502 does not capture a facial image and fill out a questionnaire survey. Specifically, medical staff devices and / or hospital servers at hospital 2501 can receive the patient's personalized estimate and / or patient data from analysis server 2503 at any time desired by the medical staff.

[0840] Specific methods for displaying and providing patient personalized estimates and patient data have been described above in 4. Treatment Process Monitoring, so duplicate descriptions will be omitted.

[0841] Meanwhile, the patient's personalized estimate corresponding to the obtained exophthalmos information may be displayed along with the value corresponding to the exophthalmos information given at the end of the thyroid eye disease treatment.

[0842] The value corresponding to the exophthalmos information at the end of thyroid eye disease treatment may refer to an exophthalmos numerical value corresponding to the point at which exophthalmos is alleviated as a result of thyroid eye disease treatment.

[0843] For example, the value corresponding to the exophthalmos information at the end of thyroid eye disease treatment may be, but is not limited to, an average value including: a value corresponding to the exophthalmos information at the end of administration of the therapeutic drug; a value corresponding to the exophthalmos information at the last time the therapeutic drug is administered; a value corresponding to the exophthalmos information at the last visit to the hospital; and / or a value corresponding to the exophthalmos information included in patient data obtained from the hospital at the time of the last visit to the hospital.

[0844] The patient's personalized estimate corresponding to the acquired exophthalmos information may be displayed to confirm the patient's personalized estimate by comparing it with a value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment. Specifically, the patient's personalized estimate corresponding to the acquired exophthalmos information may be displayed not to compare it with the patient's exophthalmos before the onset of exophthalmos, but to compare it with a value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment. That is, even if there is a difference between the patient's exophthalmos value before the onset of exophthalmos and the exophthalmos value at the end of the thyroid eye disease treatment, the patient's personalized estimate may be displayed to compare it with the exophthalmos value at the end of the treatment.

[0845] FIG. 26 is a diagram illustrating a UI for displaying a patient's personalized estimate, according to an exemplary embodiment.

[0846] 26 , displaying 2610 the obtained personalized patient estimate 2630 may be performed, such that the obtained personalized patient estimate may be verified by comparison with values including values corresponding to exophthalmos information, values corresponding to CAS information, and / or values corresponding to diplopia information at the end of thyroid eye disease treatment 2620. In this case, the obtained personalized patient estimate 2630 may be the currently obtained personalized estimate or the most recently obtained personalized patient estimate.

[0847] Referring to FIG. 26, difference values 2640 may also be displayed to facilitate comparing the patient's obtained personalized estimate 2630 with values including values corresponding to exophthalmos information, values corresponding to CAS information, and / or values corresponding to diplopia information 2620 at the end of thyroid eye disease treatment.

[0848] The patient's obtained personalized estimate is displayed along with values including values corresponding to the patient's exophthalmos information, values corresponding to CAS information, and / or values corresponding to diplopia information at the end of thyroid eye disease treatment, so that the patient and / or hospital can easily determine whether the thyroid eye disease has worsened.

[0849] The patient's personalized estimate corresponding to the exophthalmos information obtained after the end of thyroid eye disease treatment may be displayed as time series data. Time series data may refer to data arranged in a time series. Specific details regarding time series data are described in 4. Treatment Process Monitoring, so duplicate descriptions will be omitted.

[0850] The patient's personalized estimate corresponding to the exophthalmos information obtained after the end of the thyroid eye disease treatment may be displayed as time series data along with the value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment. Time series data may refer to data arranged in a time series. Specific details regarding time series data are described in 4. Treatment Process Monitoring, so duplicate descriptions will be omitted.

[0851] The patient's personalized estimate corresponding to the obtained exophthalmos information is displayed along with the value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment, so that the patient and / or hospital can determine whether the thyroid eye disease has worsened.

[0852] The patient's personalized estimates corresponding to the exophthalmos information obtained after the end of thyroid eye disease treatment are displayed as time series data so that the patient and / or hospital can determine whether the thyroid eye disease has worsened.

[0853] The patient's personalized estimates corresponding to the obtained CAS information may be displayed along with values corresponding to the CAS information at the end of the thyroid eye disease treatment.

[0854] A value corresponding to the CAS information at the end of thyroid eye disease treatment may mean CAS information corresponding to the time when the CAS value is alleviated as a result of thyroid eye disease treatment.

[0855] For example, the value corresponding to the CAS information at the end of thyroid eye disease treatment may be, but is not limited to, a value corresponding to the CAS information at the end of therapeutic drug administration; a value corresponding to the CAS information at the end of therapeutic drug administration; a value corresponding to the CAS information at the end of the hospital visit; and / or an average value including a value corresponding to the CAS information included in patient data obtained from the hospital at the end of the hospital visit.

[0856] A patient's personalized estimate corresponding to the obtained CAS information may be displayed as confirmed by comparison with the value corresponding to the CAS information at the end of thyroid eye disease treatment.

[0857] The patient's personalized estimates corresponding to the CAS information obtained after the end of thyroid eye disease treatment may be displayed as time series data. Time series data may refer to data arranged in a chronological order. Specific details regarding time series data are described in 4. Treatment Process Monitoring, so duplicate descriptions will be omitted.

[0858] The patient's personalized estimates corresponding to the CAS information obtained after the end of thyroid eye disease treatment may be displayed as time series data along with values corresponding to the CAS information at the end of thyroid eye disease treatment. Time series data may refer to data arranged in a time series. Specific details regarding time series data are described in 4. Treatment Process Monitoring, so duplicate descriptions will be omitted.

[0859] The patient's personalized estimate corresponding to the obtained CAS information is displayed along with the value corresponding to the CAS information at the end of the thyroid eye disease treatment, so that the patient and / or hospital can determine whether the thyroid eye disease has worsened.

[0860] The patient's personalized estimates corresponding to the CAS information obtained after the end of thyroid eye disease treatment are displayed as time series data so that the patient and / or hospital can determine whether the thyroid eye disease has worsened.

[0861] On the other hand, the content described above in 4. Treatment Process Monitoring can be applied to specific methods for displaying the obtained personalized estimates for a patient, so duplicate descriptions will be omitted.

[0862]

[0863] Whether thyroid eye disease has worsened can be determined based on a comparison of the patient's personalized estimate corresponding to the acquired exophthalmos information with a value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment. Specifically, whether thyroid eye disease has worsened can be determined not by comparing the patient's personalized estimate corresponding to the acquired exophthalmos information with the exophthalmos before the patient developed exophthalmos, but by comparing the patient's personalized estimate corresponding to the acquired exophthalmos information with a value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment. That is, even when there is a difference between the exophthalmos value before the patient developed exophthalmos and the exophthalmos value at the end of the thyroid eye disease treatment, whether thyroid eye disease has worsened can be determined based on the exophthalmos value at the end of the thyroid eye disease treatment.

[0864] Referring again to FIG. 25, the analysis server 2503 may determine 2526 whether thyroid eye disease has worsened based on a comparison of the patient's personalized estimate corresponding to the determined exophthalmos information and the value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment.

[0865] Specifically, thyroid eye disease may be determined to have worsened if the patient's personalized estimate corresponding to the acquired exophthalmos information increases by more than a threshold value compared to the value corresponding to the exophthalmos information at the end of thyroid eye disease treatment.

[0866] For example, thyroid eye disease may be determined to have worsened if the patient's personalized estimate corresponding to the acquired exophthalmos information increases by a threshold value of 1 mm, 2 mm, 3 mm, 4 mm, 5 mm, 6 mm, 7 mm, 8 mm, 9 mm, 10 mm, or 11 mm or more compared to the value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment. Preferably, thyroid eye disease may be determined to have worsened if the patient's personalized estimate corresponding to the acquired exophthalmos information increases by 2 mm or more compared to the value corresponding to the exophthalmos information at the end of the thyroid eye disease treatment, and the threshold value for the increase in exophthalmos value is not limited to the examples described above.

[0867] Alternatively, thyroid eye disease may be determined to have worsened if the trend in exophthalmos included in the patient's personalized estimate corresponding to the obtained exophthalmos information shows an increase of at least a threshold value compared to the value at the end of the thyroid eye disease treatment, in which case the two time points used to determine the trend in exophthalmos may be the current time and the end of the thyroid eye disease treatment.

[0868] Whether thyroid eye disease has worsened may be determined based on a comparison of the patient's personalized estimate corresponding to the obtained CAS information and a value corresponding to the CAS information at the end of the thyroid eye disease treatment. For example, analysis server 2503 may determine whether thyroid eye disease has worsened based on a comparison of the patient's personalized estimate corresponding to the determined CAS information and a value corresponding to the CAS information at the end of the thyroid eye disease treatment.

[0869] Specifically, thyroid eye disease may be determined to have worsened if the patient's personalized estimate corresponding to the acquired CAS information increases by more than a threshold value compared to the value corresponding to the CAS information at the end of thyroid eye disease treatment.

[0870] For example, thyroid eye disease may be determined to have worsened if the patient's personalized estimate corresponding to the acquired CAS information increases by two or more points compared to the value corresponding to the CAS information at the end of thyroid eye disease treatment. Depending on the patient's condition, the patient may also temporarily experience eyelid redness, conjunctival redness, eyelid swelling, conjunctival swelling, caruncle swelling, spontaneous retrobulbar pain, or pain when trying to look up or down. Therefore, it may be preferable to determine that thyroid eye disease has worsened if the CAS value increases by two or more points, rather than one or more points, and the threshold for the increase in the CAS value is not limited to the examples described above.

[0871] Alternatively, thyroid eye disease may be determined to have worsened if the patient's personalized estimate corresponding to the obtained CAS information is equal to or greater than a threshold value.

[0872] For example, thyroid eye disease may be determined to have worsened if the patient's personalized estimate corresponding to the acquired CAS information is 3 points or more, and the threshold for the CAS number is not limited to th...

Claims

1. 1. A method for monitoring overall thyroid eye disease treatment, comprising: During the treatment period, the user is prescribed and administered a therapeutic drug that has been proven effective in treating thyroid eye disease through clinical trials. requesting the user device to take a facial image and input a questionnaire in accordance with a treatment monitoring cycle; acquiring the facial image and the survey content from the user device; Using the facial image and the questionnaire survey content acquired from the user device, a personalized estimate of the user corresponding to exophthalmos information, CAS (Clinical Activity Score) information, and diplopia information among the indicators proven in the approval stage of the therapeutic drug is obtained; and displaying, including visualizing on the user device, the personalized estimates for the user in chronological order from the time the user began administering the therapeutic agent; After the treatment period is over, requesting the user device to capture the facial image in accordance with a post-treatment monitoring cycle; acquiring the facial image from the user device; obtaining the personalized estimate of the user corresponding to at least the exophthalmos information by using the facial image obtained from the user device; a displaying step, comprising visualizing on the user device the personalized estimate of the user corresponding to the exophthalmos information by comparing it with the exophthalmos information at the end of the administration of the therapeutic agent; and displaying a message suggesting a visit to a clinic based on a difference between the personalized estimate of the user corresponding to the exophthalmos information and the exophthalmos information at the end of the administration of the therapeutic agent. A method for providing

2. The method of claim 1 , wherein the post-treatment monitoring cycle is different from the treatment monitoring cycle.

3. The method of claim 2 , wherein the treatment monitoring cycle and the post-treatment monitoring cycle are determined according to the therapeutic agent.

4. The therapeutic monitoring cycle is determined by taking into account the administration cycle of the therapeutic agent; The post-treatment monitoring cycle is determined by considering the time point of recurrence of symptoms after administration of the therapeutic agent has ceased. The method of claim 3.

5. The questionnaire survey includes questions about side effects based on the results of the clinical trial, determining whether side effects have occurred by using the questionnaire obtained from the user device during the treatment period; The method of claim 1 further comprising:

6. If the user experiences any side effects during the treatment period, displaying a message on the user device suggesting at least one of calling the hospital and visiting an internet site of the hospital. The method of claim 5 further comprising:

7. displaying comparative data obtained during the treatment period based on the results of the clinical trial on the user device in correspondence with the date of obtaining the personalized estimate for the user. The method of claim 1 , comprising:

8. The comparative data is obtained by adjusting standard data included in the results of the clinical trial according to the personalized estimate. The method of claim 7.

9. the personalized estimate of the user corresponding to the exophthalmos information includes a numerical value for exophthalmos; the personalized estimate of the user corresponding to CAS information includes a numerical value for CAS; the personalized estimate of the user corresponding to the diplopia information includes a grade of diplopia; and displaying the visualization of the personalized estimate for the user in chronological order from the time the user began administering the therapeutic agent includes displaying the numerical value for exophthalmos, the numerical value for CAS, and the grade for diplopia on the user device in at least one of a chronological data table and a chronological data graph. The method of claim 1.

10. displaying a message suggesting a visit to a clinic based on a difference between the personalized estimate of the user corresponding to the exophthalmos information and the exophthalmos information at the end of administration of the therapeutic agent, determining whether the difference between the personalized estimate of the user corresponding to the exophthalmos information and the exophthalmos value included in the exophthalmos information at the end of administration of the therapeutic agent is greater than 2 mm; and displaying a message on the user device about a suggestion to visit a hospital if the difference in the exophthalmos values is equal to or greater than 2 mm. The method of claim 1 , comprising:

11. After the treatment period is over, requesting and obtaining input of the survey content into the user device in accordance with the post-treatment monitoring cycle; obtaining the personalized estimate of the user corresponding to the CAS information by using the facial image obtained from the user device and the survey content; determining whether the personalized estimate of the user corresponding to the CAS information is equal to or greater than 3; and If the personalized estimate of the user corresponding to the CAS information is equal to or greater than 3, displaying the message about the suggestion to visit the hospital on the user device. The method of claim 1.

12. The step of obtaining personalized estimates of the user corresponding to exophthalmos information, CAS (Clinical Activity Score) information, and diplopia information using the facial images obtained from the user device and the questionnaire content includes: obtaining the personalized estimate of the user corresponding to the exophthalmos information by using the facial image; obtaining the personalized estimate of the user corresponding to the CAS information by using the facial image; and obtaining the personalized estimate of the user corresponding to the double vision information by using the facial image; The method of claim 1 , comprising:

13. acquiring an actual measurement value of the user's exophthalmos and a face image corresponding to the actual measurement value of the exophthalmos; wherein the personalized estimate of the user corresponding to the exophthalmos information is obtained by using the face image obtained from the user device, the face image corresponding to an actual measurement of the exophthalmos numerical value, and the actual measurement of the exophthalmos. The method of claim 1 further comprising:

14. The step of acquiring the facial image and the questionnaire content from the user device during the treatment period comprises: providing a photography guide to a user device; capturing a facial image when the capture guide is satisfied; displaying the survey content on the user device; and Obtaining the results of the survey Including; wherein the step of acquiring a facial image from the user device after the treatment period has ended comprises: providing the imaging guide to the user device; and capturing a face image when the capture guide is satisfied. The method of claim 1 , comprising:

15. The method of claim 14 , wherein the imaging guide is a guide for guiding at least one of a left-right angle of a face, a vertical angle of the face, an expression of the face, and an eye position in an image.

16. the photographing guide includes indicators indicating whether the left-right angle of the face, the up-down angle of the face, the facial expression of the face, and the position of the eyes in the image are each satisfied; The step of capturing a face image when the capture guide is satisfied includes: determining whether the imaging guide is satisfied according to whether the horizontal angle of the face, the vertical angle of the face, and the position of the eyes on the image satisfy criteria; When the photographing guide is satisfied, changing the display status of the indicators indicating whether the horizontal angle of the face, the vertical angle of the face, the facial expression of the face, and the position of the eyes in the image are satisfied; and capturing the face image if the capture guide is satisfied.

16. The method of claim 15, comprising:

17. the user visits a hospital to receive the therapeutic drug; The method for monitoring the overall treatment of thyroid eye disease treatment comprises: obtaining a thyroid disorder management history for the user; and providing a medical staff device with a history of thyroid dysfunction management when the user visits a hospital; The method of claim 1 , comprising:

18. A non-transitory computer-readable recording medium storing a computer program for executing the method of claim 1.

Images