Online collaboration system and health monitoring method in online collaboration system

Abstract

We provide an online collaboration system that aims to improve the accuracy of online medical consultations by linking with a health monitoring system that uses in-vivo cybernetic avatars. [Solution] An online collaboration system that provides online medical care by linking the subject terminal to the virtual doctor terminal or the real doctor terminal, comprising a subject terminal for receiving online medical care, a virtual doctor terminal for a virtual doctor, a real doctor terminal for specialists, an online collaboration server, and the subject terminal to the virtual doctor terminal or the real doctor terminal, the online collaboration server having a database of subject information and in-vivo cybernetic avatar information, and when it receives a virtual doctor selection request from the subject terminal, it transfers the subject information and in-vivo cybernetic avatar information to the virtual doctor terminal, and when it receives a real doctor selection request, it transfers the subject information and in-vivo cybernetic avatar information to the real doctor terminal 9, and the online medical care is provided.

Description

[Technical Field]

[0001] The present invention relates to an online collaboration system and a health monitoring method in the online collaboration system, and in particular to an online collaboration system and a health monitoring method in the online collaboration system that collaborates with a health monitoring system that uses an in vivo cybernetic avatar and contributes to improving the accuracy of online medical care. [Background technology]

[0002] Recently, research into the relationship between lifestyle-related diseases and sleep has progressed, and it has been shown that lack of sleep has a wide range of effects, including daytime sleepiness and fatigue, an increase in psychosomatic complaints such as headaches, emotional instability, and reduced work efficiency related to impaired attention and judgment. Furthermore, it has become clear that when various sleep problems, including lack of sleep, become chronic, they are associated with an increased risk of developing obesity, high blood pressure, type 2 diabetes, heart disease, and cerebrovascular disease, as well as worsening symptoms, and are involved in an increased mortality rate.

[0003] In the daily health monitoring method, the subject's biological information can be monitored using a biometric device, such as a blood pressure monitor, a weight scale, a smartphone-compatible predictive thermometer, a wristband-type life recorder, a pocket-type activity monitor, an upper arm hoseless blood pressure monitor, or a body composition monitor.

[0004] It is desirable to achieve a reduction in medical expenses and a long and healthy life by using a health monitoring method that makes it possible to achieve a long and healthy life by performing health monitoring as simply as possible.

[0005] The inventor has previously proposed a health monitoring system that uses an in-vivo cybernetic avatar to monitor the spatiotemporal internal environment, making it possible to visualize the health monitoring results for health and pre-disease, encouraging behavioral changes in subjects, and enabling them to live long, healthy lives (Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 7614679 [Patent Document 2] Japanese Patent Application Laid-Open No. 2024-171169 Summary of the Invention [Problem to be solved by the invention]

[0007] The above-mentioned Patent Document 1 encouraged subjects to change their behavior, improve lifestyle-related diseases, and achieve healthy longevity, but did not consider utilizing the measurements of health monitoring results.If virtual doctors, or real doctors such as family doctors or hospital doctors, could use the measurements of health monitoring results, it is believed that the accuracy of online medical consultations could be improved.

[0008] The above-mentioned Patent Document 2 discloses a medical information processing device that uses a virtual doctor (AI doctor) to suggest appropriate actions for each patient, but does not teach or suggest the use of data measuring spatiotemporal internal body environment information using an in vivo cybernetic avatar.

[0009] The present invention was devised in light of the above-mentioned problems, and the purpose of the present invention is to provide an online collaboration system that links with a health monitoring system that uses an in-vivo cybernetic avatar and aims to improve the accuracy of online medical consultations.

[0010] Another object of the present invention is to provide a health monitoring method in an online linked system that can predict the current health condition or the location where a disease will worsen in the future. [Means for solving the problem]

[0011] In order to achieve the above object, the online collaboration system of the first invention is an online collaboration system in which a subject terminal for a subject receiving online medical care, a virtual doctor terminal for at least a virtual doctor who is a virtual doctor, a real doctor terminal for specialists including at least a doctor, an online collaboration server, and a management server having a virtual doctor section and an in-vivo cybernetic avatar section are connected via the cloud, and the subject terminal is linked to the virtual doctor terminal or the real doctor terminal by the online collaboration server at the subject's selection to provide online medical care, the online collaboration server has a database that stores subject information and in-vivo cybernetic avatar information, and when the online collaboration server receives a virtual doctor selection request from the subject terminal, it transfers the subject information and in-vivo cybernetic avatar information stored in the database to the virtual doctor terminal to provide virtual online medical care, and when the online collaboration server receives a real doctor selection request from the subject terminal, it transfers the subject information and in-vivo cybernetic avatar information stored in the database to the real doctor terminal to provide online medical care.

[0012] The online linkage system according to a second aspect of the present invention is the system according to the first aspect of the present invention, wherein the subject information includes body temperature data, blood pressure data, weight data, and blood glucose data of the subject measured by a medical measurement device, and system The data is characterized by including the subject's exercise record data, sleep status data, and dietary data measured by the device.

[0013] The online collaboration system according to the third invention is characterized in that, in the first or second invention, the in vivo cybernetic avatar information is spatiotemporal internal body environment information measured by a measuring means that measures spatiotemporal internal body environment information using an in vivo cybernetic avatar.

[0014] The online linkage system according to the fourth invention is 2In the invention, the database is characterized in that the measurement data from the medical measurement system device and the measurement data from the non-medical measurement system device are converted into common data with the same data structure and stored.

[0015] The health monitoring method in the online linkage system according to the fifth invention is the same as that in the online linkage system according to the second invention. A health monitoring method in The medical measurement system device includes at least a blood pressure monitor, a thermometer, and a weight scale, and includes a biological information acquisition step of acquiring biological information of the subject using the blood pressure monitor, the thermometer, and the weight scale; system The device includes a body-wearable device and is characterized by comprising: a behavioral information acquisition step for acquiring behavioral information, which is physical activity information, from the body-wearable device; a dietary data acquisition step for acquiring dietary data obtained by calculating nutritional information from dietary image data of the subject captured by a dietary image sensor; a health checkup result information acquisition step for acquiring health checkup result information including at least brain images and in-vivo endoscopic images; an analysis step for analyzing the acquired biometric information, behavioral information, dietary data, and health checkup result information of the subject; and an estimation step for estimating the subject's future health condition using the results of the analysis and the subject's biometric information, behavioral information, dietary data, and health checkup result information.

[0016] In the online linkage system according to the sixth invention Ken The health monitoring method according to the fifth aspect of the present invention further comprises an electroencephalogram (EEG) measurement step of measuring the electroencephalogram (EEG) of the subject's sleep state at home, and the electroencephalogram (EEG) measurement step is used to perform a sleep test equivalent to a polysomnogram test.

[0017] A seventh aspect of the present invention provides a health monitoring method in an online collaboration system according to the fifth aspect of the present invention, wherein the medical measurement system device further includes a blood glucose meter, ofand an estimation step of estimating the future health condition of the subject using the results of the analysis, biological information including the blood glucose data of the subject, behavioral information, dietary data, and health check result information, and the blood glucose data acquired by the blood glucose data acquisition step in the analysis.

[0018] The health monitoring method in an online collaboration system according to an eighth aspect of the present invention is the method according to the fifth aspect of the present invention, wherein the virtual doctor is a doctor avatar, and the avatar is connected to family, friends, and doctors. wind It is characterized by the fact that it can be changed to an avatar.

[0019] Online collaboration according to the ninth invention Sis In the eighth aspect of the present invention, the health monitoring method for a system is characterized in that the avatar is configured to be able to converse by voice using a multimodal generation AI. [Effects of the Invention]

[0020] According to the present invention having the above configuration, it is possible to link with a health monitoring system that uses an in vivo cybernetic avatar, and to realize an online linkage system that improves the accuracy of online medical care.

[0021] According to the present invention, a health monitoring method can be realized that can predict the current health state or the location where a disease will worsen in the future. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is a functional block diagram of an online collaboration system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a conceptual diagram of virtual doctor collaboration and real doctor collaboration in the online collaboration system according to the embodiment of the present invention. [Figure 3] FIG. 3 is a processing flowchart of online collaboration according to an embodiment of the present invention. [Figure 4] Figure 4 is a diagram explaining the flow of measurement data from the medical measurement system and non-medical measurement system, and data from the in-vivo cybernetic avatar. [Figure 5] FIG. 5 is a diagram showing an example of common data conversion of measurement data obtained by the medical measurement system and the non-medical measurement system shown in FIG. [Figure 6] FIG. 6 is a flowchart showing the common data conversion and transfer of the measurement data and dietary data from the sensors of the medical measurement system shown in FIG. [Figure 7] FIG. 7 is a flowchart showing the common data conversion and transfer of the measurement data and dietary data from the non-medical measurement system sensor group in FIG. [Figure 8] Figure 8 is an explanatory diagram of the common data collection and AI analysis of measurement results from sensors from different manufacturers. [Figure 9] FIG. 9 is a functional block diagram showing an example of the AI ​​analysis unit 81 of FIG. [Figure 10] FIG. 10 is a diagram showing an example of a food image. [Figure 11] FIG. 11 shows an example of images of meals stored for each dish on a certain day. [Figure 12] FIG. 12 is a diagram showing an example of nutritional components of dietary data. [Figure 13] FIG. 13 is a diagram showing an example of an avatar corresponding to a health advice level given by a virtual doctor. [Figure 14] FIG. 14 is a diagram illustrating an example of a hardware configuration of a portable communication terminal serving as a subject terminal. [Figure 15] FIG. 15 is a diagram showing an example of a display screen of the health monitoring application and the online link application installed on the mobile communication terminal of FIG. [Figure 16] FIG. 16 is a functional block diagram of an online cooperative application according to an embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing an example of an in vivo cybernetic avatar display area of ​​the present invention. [Figure 18]FIG. 18 is a diagram showing an example of the three major disease avatar display. [Figure 19] FIG. 19 is a diagram showing criteria to be displayed in the physical activity display area. [Figure 20] FIG. 20 is a flowchart illustrating a health monitoring method according to an embodiment of the present invention. [Figure 21] FIG. 21 is a flowchart illustrating a health monitoring method according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0023] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

[0024] First, an online collaboration system 100 according to an embodiment of the present invention will be described with reference to FIGS.

[0025] <Embodiment> FIG. 1 is a functional block diagram of an online collaboration system according to an embodiment of the present invention, FIG. 2 is a conceptual diagram of virtual doctor collaboration and real doctor collaboration in the online collaboration system according to an embodiment of the present invention, and FIG. 3 is a processing flowchart of online collaboration according to an embodiment of the present invention.

[0026] As shown in Figure 1, online collaboration system 100 includes subject terminals 1, 2, 3, ..., administrator terminal 4, cloud 5, management server 6, online collaboration server 7, virtual doctor terminal 8, and real doctor terminal 9. Online collaboration server 7 includes virtual doctor collaboration unit 71, real doctor collaboration unit 72, and database 73.

[0027] The online collaboration system 100 is connected via a cloud 5 to subject terminals 1, 2, 3, ... for subjects receiving online medical care, a virtual doctor terminal 8 for at least a virtual doctor who is a virtual doctor, a real doctor terminal 9 for specialists including at least a doctor, an online collaboration server 7, and a management server 6 having a virtual doctor unit 61 and an in-vivo cybernetic avatar unit 62, and the subject terminal is linked to the virtual doctor terminal 8 or the real doctor terminal 9 by the online collaboration server according to the subject's selection to perform online medical care. The online collaboration server 7 has a database 73 storing subject information and in-vivo cybernetic avatar information,

[0028] When the online collaboration server 7 receives a virtual doctor selection request from the subject terminals 1, 2, 3, ..., it transfers the subject information and in-vivo cybernetic avatar information stored in the database 73 to the virtual doctor terminal 8 and performs a virtual online medical examination, and when it receives a real doctor selection request from the subject terminals 1, 2, 3, ..., it transfers the subject information and in-vivo cybernetic avatar information stored in the database to the real doctor terminal and performs an online medical examination.

[0029] Subject terminal 1 is a mobile communication terminal for subjects receiving online medical care. Examples include smartphones, tablet terminals, laptop PCs, and desktop PCs. During online medical care, the subject can talk to a virtual doctor or a real doctor via video call using a camera installed on subject terminal 1. Subject terminals 2, 3, etc. also have a similar device configuration, so they can receive the above-mentioned online medical care.

[0030] The administrator terminal 4 is a terminal that performs operation management of the management server 6 and the online cooperation server 7 on the cloud 5, and performs user registration for various terminals and acts as a help desk for the cloud 5.

[0031] The cloud 5 provides cloud services through an online collaboration server 7 and a management server 6. Membership registration can be performed via the subject's terminal 1, the virtual doctor terminal 8, the real doctor terminal 9 (hospital terminal 9a) or the family doctor terminal 9b, and cloud services can be accessed. Membership registration is performed by registering the email addresses and passwords of the subject (patient), hospital doctor, family doctor, etc., and cloud services can be accessed. Security levels can be increased by using biometric authentication, etc.

[0032] The management server 6 has a virtual doctor unit 61 and an in-vivo cybernetic avatar unit 62. The virtual doctor unit 61 stores a plurality of virtual doctors and provides the virtual doctors to the virtual doctor terminal 8. The in-vivo cybernetic avatar unit 62 uses the technology of Japanese Patent No. 7614679 to store in-vivo cybernetic avatar information, which is spatiotemporal internal body environment information measured by a measuring means that measures spatiotemporal internal body environment information using an in-vivo cybernetic avatar.

[0033] The online collaboration server 7 has a virtual doctor collaboration unit 71, a real doctor collaboration unit 72, and a database 73. When the virtual doctor collaboration unit 71 of the online collaboration server 7 receives a virtual doctor selection request from the subject terminal 1, it transfers the subject information and in-vivo cybernetic avatar information stored in the database 73 to the virtual doctor terminal 8, and performs a virtual online medical examination. When the real doctor collaboration unit 72 of the online collaboration server 7 receives a real doctor selection request from the subject terminal 1, it transfers the subject information and in-vivo cybernetic avatar information stored in the database 73 to the real doctor terminal 9, and performs an online medical examination by a doctor.

[0034] Fig. 2 is a conceptual diagram of virtual doctor collaboration and real doctor collaboration in an online collaboration system according to an embodiment of the present invention, and Fig. 3 is a processing flowchart for online collaboration according to an embodiment of the present invention. The online collaboration process will be described below with reference to Figs. 2 and 3.

[0035] As shown in FIG. 2, when the subject starts the online collaboration app on the subject terminal 1 (step 301), a doctor selection screen is displayed (step 302). When the subject (patient) checks and selects a virtual doctor (step 303), a virtual doctor selection request is sent to the online collaboration server 7. The online collaboration server 7 receives the virtual doctor selection request and instructs the virtual doctor collaboration unit 71 to collaborate with the virtual doctor (step 304). The virtual doctor collaboration unit 71 transfers the subject information and in-vivo cybernetic avatar information stored in the database 73 to the virtual doctor terminal 8 (steps 305, 306). Here, the subject information includes the subject's body temperature data, blood pressure data, weight data, and blood glucose data measured by medical measurement devices. Non-medical measurement system The data includes the subject's exercise record data, sleep state data, and dietary data measured by the device. The in-vivo cybernetic avatar information is spatiotemporal internal environment information measured by a measurement means that measures spatiotemporal internal environment information using an in-vivo cybernetic avatar. The acquisition of spatiotemporal internal environment information using an in-vivo cybernetic avatar is performed using the technology disclosed in Patent Publication No. 7614679, previously proposed by the present inventor. Database 73 stores measurement data from medical measurement devices and non-medical measurement devices, converted into common data with the same data structure.

[0036] The virtual doctor terminal 8 performs a virtual online medical examination by adding the above-mentioned subject information and in-vivo cybernetic avatar information (step 307). This allows a virtual online medical examination to be performed that takes into account the subject's real measurement data, thereby improving the accuracy of the virtual online medical examination.

[0037] On the doctor selection screen shown in Figure 2 (step 302), the subject (patient) checks the real doctor's "family doctor" or "hospital" (step 303). If "hospital" is selected, the names of registered doctors who provide online medical care are displayed. Select a registered doctor's name. A real doctor selection request is sent to the online collaboration server 7. The online collaboration server 7 receives the real doctor selection request and instructs the real doctor collaboration unit 72 to collaborate with a real doctor (step 308). The real doctor collaboration unit 72 transfers the above-mentioned subject information and in-vivo cybernetic avatar information stored in the database 73 to the hospital terminal 9a or the family doctor terminal 9b (steps 309, 310). Here,

[0038] The hospital terminal 9a or the family doctor terminal 9b performs online medical treatment by the family doctor or a selected doctor at the hospital, adding the above-mentioned subject information and in-vivo cybernetic avatar information (step 311). This allows online medical treatment to be performed taking into account the subject's real measurement data, thereby improving the accuracy of online medical treatment. Since online medical treatment can be performed using real measurement data, patients can receive online medical treatment without having to fill out a medical questionnaire, thereby shortening the time required for online medical treatment.

[0039] In database 73, measurement data from medical measurement devices and measurement data from non-medical measurement devices are converted into common data with the same data structure and stored. By storing such common data, personal information of patients can be used by primary care physicians and various hospitals, leading to improved medical services and reduced medical costs. However, since this information involves personal information of the patients who are the subjects, consent from the patients must be obtained. In addition, sufficient security measures must be taken. This common data will be explained below.

[0040] Figure 4 is a diagram explaining the flow of measurement data from the medical measurement system and non-medical measurement system, and data from the in-vivo cybernetic avatar.

[0041] In Figure 4, medical measurement devices include at least a blood pressure monitor, a thermometer, a weight scale, and a blood glucose meter. Many blood pressure monitors can also simultaneously measure pulse, so in such cases, the subject's pulse is also measured and recorded. Non-medical measurement devices are treated as health devices and include, for example, wearable devices, smartwatches, ring-type devices, watch-type devices, and food imaging cameras (smartphone cameras are also acceptable). Food imaging cameras can be equipped with cameras that are standard on smartphones. Medical measurement data measured by medical measurement devices is collected by the subject's terminal 1 and temporarily stored as medical measurement data in a database 73 on the online collaboration server 7. The database 73 is then transferred from the database 73 to the real doctor terminal 9 and the virtual doctor terminal 8. The real doctor terminal 9 uses the medical measurement data for medical examinations. Furthermore, information from the in vivo cybernetic avatar unit 62 is also provided to the real doctor terminal 9. The real doctor terminals (9a, 9b) can use the patient's real measurement data for medical examinations.

[0042] Other examples include measuring biometric data such as blood pressure, heart rate, blood sugar, body temperature, and oxygen saturation in real time and linking it to a smartphone or cloud platform. The data can be analyzed by AI and configured to generate alerts in the event of a health risk. This technology can be used not only for daily health management, but also for monitoring athletes' performance and as a monitoring system for the elderly.

[0043] Another example is the use of ultra-small sensors implanted in the body as nanosensors to monitor blood components, thereby enhancing the management of chronic diseases such as diabetes and high blood pressure. Nanosensors can detect changes in blood and bodily fluid components in real time and transmit the data wirelessly. This allows for earlier detection of changes in physical condition, leading to the early detection and prevention of diseases.

[0044] On the other hand, non-medical measurement data measured by non-medical measurement devices is collected by the subject terminal 1, temporarily stored as non-medical measurement data in a database 73 of the online collaboration server 7, and then transferred from the database 73 to the virtual doctor terminal 8. In the virtual doctor terminal 8, the non-medical measurement data is used for virtual consultations.

[0045] Furthermore, a food image capture camera captures food images, and the food images for each dish are saved on the subject terminal 1. A publicly known food image analysis app installed on the subject terminal 1 calculates nutritional information and temporarily stores it as food data in a database 73 on the online collaboration server 7. The data is then transferred from the database 73 to a virtual doctor terminal 8 or a real doctor terminal 9. The virtual doctor terminal 8 uses the food data for health advice (nutritional advice) based on AI diagnosis. The real doctor terminal 9 uses the food data for nutritional consultations by registered dietitians. By using the food data for medical examinations and AI diagnosis, for example, when used for online medical consultations for diabetes patients, online consultations can be performed using nutritional information that is more in line with actual diets. Conventional nutritional consultations for diabetes involve recording the daily meal menu on paper, but analysis based on accurate food data is now possible, improving the accuracy of nutritional consultations.

[0046] Fig. 5 is a diagram showing an example of common data conversion of measurement data from the medical measurement system and non-medical measurement system in Fig. 4, Fig. 6 is a flowchart of common data conversion and transfer of measurement data and dietary data from the sensor group of the medical measurement system in Fig. 5, and Fig. 7 is a flowchart of common data conversion and transfer of measurement data and dietary data from the sensor group of the non-medical measurement system in Fig. 5. Below, common data will be explained using Figs. 5 to 7.

[0047] Each sensor group is divided into a medical sensor group and a non-medical sensor group. As shown in FIG. 5, the medical sensor group includes at least a medical daily activity sensor group 31, a medical exercise sensor group 32, a medical sleep sensor group 33, and a meal imaging sensor group 34. Measurement data from the medical sensor group is converted by the common data converter 11 of the common data converter 10, and the converted common data is transferred to the real doctor terminal 9. As shown in FIG. 5, the non-medical sensor group includes at least a non-medical daily activity sensor group 35, a non-medical exercise sensor group 36, a non-medical sleep sensor group 37, and a meal imaging sensor group 38. Measurement data from the non-medical sensor group is converted by the common data converter 12 of the common data converter 10, and the converted common data is transferred to the virtual doctor terminal 8. The function of the common data converter 10 may be provided in the online collaboration server 7, or the common data conversion may be performed in the subject terminal 1.

[0048] As shown in FIG. 6, measurements using the medical sensor group include measurements of the subject using the medical activity sensor group 31 (step 601). Measurements are then made using the medical movement sensor group 32 (step 602), the medical sleep sensor group 33 (step 603), and the food image sensor group 34 (step 604). The frequency of measurements using these sensor groups may be changed depending on the subject's health condition, such as several times a day or in real time. Measurements using the food image sensor group are made at three meal times a day (breakfast, lunch, and dinner), and if a snack is taken, food images of the snack are also taken. The order of the above-mentioned measurement and food image capture steps may be changed as appropriate.

[0049] At least the data is converted into common data on a daily basis and transferred (steps 605 and 606). The common data is accumulated as trend data for one day, one week, one month, three months, six months, one year, etc.

[0050] As shown in FIG. 7, measurements using the non-medical sensor group include measurements using the non-medical daily activity sensor group 35 (step 701), measurements using the non-medical exercise sensor group 36 (step 702), measurements using the non-medical sleep sensor group 37 (step 703), and measurements using the food image sensor group 38 (step 704). The frequency of measurements using these sensor groups may be changed depending on the subject's health condition, such as several times a day or in real time. Measurements using the food image sensor group are taken three times a day (breakfast, lunch, and dinner), and if a snack is taken, food images of the snack are also taken. The order of the above-mentioned measurement and food image capture steps may be changed as appropriate.

[0051] At least the data is converted into common data on a daily basis and transferred (steps 705 and 706). The common data is accumulated as trend data for one day, one week, one month, three months, six months, one year, etc.

[0052] Figure 8 is an explanatory diagram of the commonization of measurement results from sensors from different manufacturers and AI analysis. The sensors consist of sensors for daily life, exercise, sleep, and diet. The measurement data for each of these is converted by the corresponding data converter to obtain common daily life data, common exercise data, common sleep data, and common diet data. The common data obtained is input to the AI ​​analysis unit 81, which outputs analysis results based on AI analysis rules.

[0053] FIG. 9 is a functional block diagram of the AI ​​analysis unit 81 in FIG. 8, FIG. 10 is a diagram showing an example of photographing a food image, and FIG. 11 is a diagram showing an example of images of each dish stored as food images for a certain day. The example in FIG. 10 shows an example in which a subject holds a smartphone (subject terminal) 97 in their left hand 95 and takes a photo of a food image 94 with their right hand 96. The example in FIG. 11 shows an example in which food images 112-119 for each dish for a certain day are stored in a storage unit on the screen of a subject terminal 111. The date and time of each food image 112-119 is recorded, allowing the subject to know the exact time when each dish was eaten.

[0054] FIG. 12 is a diagram showing an example of nutritional components of dietary data, and FIG. 13 is a diagram showing an example of avatars corresponding to health advice levels from a virtual doctor. The example in Figure 12 shows an example of dietary data of nutritional components calculated by a known nutrition calculation app that calculates nutritional value by performing AI image analysis on the meal images of each dish shown in Figure 11. As shown in Figure 12, the data includes at least numerical data for "protein," "fat," "carbohydrate," "sugar," "dietary fiber," and "salt," but other nutritional components may also be included.

[0055] The example in Figure 13 shows the health advice level and corresponding avatar when a virtual doctor gives health advice based on the AI ​​analysis results in Figure 9. In other words, the subject can freely choose from three types of advice: a doctor avatar with a gentle expression when they want to receive "gentle" health advice; a doctor avatar with a calm expression when they want to receive "normal" health advice; and a doctor avatar with a stern expression when they want to receive "strict" health advice. In this way, by changing the health advice level, the advice from the virtual doctor can be varied in three stages. For subjects who prefer a more realistic image, the avatar image can be a virtual human doctor image, which is a realistically generated version of an actual doctor.

[0056] In the online collaboration described above, a still image of a virtual doctor is displayed. However, multimodal generation AI can be used to combine and correlate text, natural language, voice, numerical values, table data, image vision, and video, allowing the virtual doctor avatar and the patient to converse via voice. Online health management can also be configured to enable collaboration between family members or friends. The virtual doctor avatar can be configured as a 3D still image or 3D video, and can be freely changed to look like a family member, friend, or doctor. Similarly, multimodal generation AI can be used to configure avatars of family members and friends to converse with the subject via voice. For example, when monitoring an elderly person in a remote location, a child can become an avatar and converse with the elderly person (parent) via voice. This makes it easier for children to communicate with the elderly than directly, and is expected to facilitate health management for the elderly.

[0057] Hereinafter, virtual AI analysis including dietary data will be described with reference to FIGS. In Figure 9, the AI ​​analysis unit 81 has common daily activity data (L) 82, a daily activity state rule group 83, common exercise data (T) 84, an exercise state rule group 85, common sleep data (S) 86, a sleep state rule group 87, common dietary data (E) 88, a dietary state rule group 89, (L+T+S+E) evaluation data 90, a comprehensive evaluation rule group 91, an AI analysis means 92, and a trend database 93.

[0058] The common life activity data (L) 82 includes at least the subject's body temperature data, blood pressure data, weight data, and blood glucose data measured by a medical measurement device. Other data, such as pulse data, may also be included.

[0059] The common exercise data (T) 84 includes exercise record value data of the subject measured by a non-medical measurement device. The exercise record value data may be automatically collected from wearable devices (body-worn devices) such as pedometers and smart watches via near-field wireless communication such as a Bluetooth (registered trademark) interface, or may be numerical data input by voice input. In addition, in the case of strength training or aerobic exercise, if the numerical data cannot be input directly, it may be input into the subject terminal 1 by voice input or the like. The input data at that time records the exercise time and the number of floors.

[0060] The common sleep data (S) will be stored separately from data measured using approved medical measurement devices and data measured using non-medical measurement devices such as smartwatches. Measurement results from medical measurement devices will be used in online medical consultations, and health advice will be provided in virtual consultations by weighting the reliability of AI analysis between the measurement results from medical measurement devices and non-medical measurement results.

[0061] The common dietary data (E) can be obtained using a publicly known nutrition calculation app that calculates nutritional value by performing AI image analysis on dietary images (see FIG. 11) of each dish taken with a food image capturing camera (see FIG. 10). Specifically, the nutritional components include at least the numerical data for "protein," "fat," "carbohydrate," "sugar," "dietary fiber," and "salt," as shown in FIG. 12, but other nutritional components may also be included.

[0062] The activity status rule group 83 creates rules based on the reference value data for at least the subject's body temperature data, blood pressure data, weight data, and blood glucose level data. Blood pressure data is determined based on the reference value for home blood pressure. Normal values ​​are 125 / 75 mmHg or less, the reference value for low blood pressure is a systolic blood pressure (maximum blood pressure) of less than 100 mmHg, and the reference value for high blood pressure is 135 / 85 mmHg or more. Blood glucose level data is determined as approximately 70 to 100 mg / dL when fasting, and less than 140 mg / dL after eating. Blood glucose levels can be determined more accurately because the date and time of meals and meal data are stored in the database. Weight data is determined based on the standard weight for each height. For example, if a person is 170 cm tall, their weight would be 60.36 kg. This standard value is determined within a ±10% range. Body temperature data is determined based on a healthy body temperature of 35.5 to 37.5°C.

[0063] The exercise state rule group 85 is based on target exercise value data. For simple discrimination, the rule group 85 is based on criteria such as a pedometer value of 5000 steps, the number of strength training sessions, and the duration of aerobic exercise.

[0064] The sleep state rule group 87 distinguishes between measurement data obtained by medical measurement devices and measurement data obtained by non-medical measurement devices, accumulates trend data, and makes judgments based on rules.

[0065] The dietary state rule group 89 is made into rules based on the calorie calculation values ​​of the nutritional components as shown in Fig. 12. The judgment is made based on the standard daily intake of the nutritional components.

[0066] (L+T+S+E) Evaluation data 90 calculates the evaluation value of each common data to obtain evaluation data.

[0067] The comprehensive evaluation rule group 91 creates a rule group by multiplying the evaluation value data by the degree of risk of the disease.

[0068] The AI ​​analysis means 92 applies each rule group 83, 85, 87, 89, and 91 based on each common data 82, 84, 86, and 88, analyzes each evaluation data, uses data from the trend database 93, applies the comprehensive evaluation rule group, and performs a comprehensive evaluation.

[0069] FIG. 14 is a diagram illustrating an example of the hardware configuration of a portable communication terminal as a subject terminal.

[0070] As shown in FIG. 14, the mobile communication terminal has a CPU 131, a ROM 132, a RAM 133, a display 134, a biometric authentication memory unit 135, an ID / password authentication unit 136, a touch input unit 137, a communication interface 138, a camera 139, a speaker 140, and a microphone 141.

[0071] The mobile communication terminal is connected to a cloud server via cloud 5. In such a communication system, the health monitoring system may be realized by software, and a health monitoring application 152 (similar to that disclosed in Japanese Patent No. 7614679) and an online linking application 153 (the present invention) may be installed on a mobile communication terminal 150 as shown in FIG. 15, and online linking may be performed by executing the online linking application 153. The example in FIG. 15 shows a state in which icons for the health monitoring application 152 and the online linking application 153 are displayed on a screen 151. Alternatively, a general-purpose personal computer may be used.

[0072] Hereinafter, the technology of Japanese Patent No. 7614679 will be used for the visualization display by health monitoring, and therefore a description thereof will be omitted in this specification.

[0073] FIG. 16 is a functional block diagram of an online cooperative application according to an embodiment of the present invention. As shown in FIG. 16 , the health management application 153 in this embodiment has a daily activity recording unit 154, an exercise recording unit 155, a sleep recording unit 156, a meal image recording unit 157, a nutrition calculation unit 158, a nutrition recording unit 159, a pre-illness visualization unit 160, a virtual doctor collaboration unit 161, a real doctor collaboration unit 162, and an online collaboration dashboard unit 163.

[0074] The online linking app 153 is installed in advance on a mobile communication terminal such as a smartphone, a tablet PC, a portable laptop, a desktop computer, or the like, and then used. For example, it may be installed in advance on a mobile communication terminal, the online linking app 153 is started, and measurement data from a blood pressure monitor, a weight scale, and a thermometer may be automatically recorded using a short-range wireless communication function. Alternatively, the measured data may be manually entered by the subject. When using the online linking app 153, height data, age, and gender (male or female; gender entry may be omitted if not desired) are required. The subject's name (nickname is acceptable) is also entered, as this is the minimum information required for effective online linking.

[0075] The daily activity recording unit 154 records various daily activity data (blood pressure, weight, body temperature, blood sugar level). If the blood pressure monitor can measure the pulse rate, pulse data may be recorded at the same time.

[0076] The exercise recording unit 155 is linked to a pedometer and allows automatic input or manual input by the subject. Automatic input is more preferable to reduce the burden of recording on the subject. Alternatively, the exercise recording unit 155 allows manual input of whether or not muscle training or resistance exercise was performed.

[0077] The sleep recording unit 156 records sleep-related data from an electroencephalogram measuring device disclosed in Japanese Patent No. 7614679. Alternatively, the sleep state is recorded using a non-medical measurement device such as a smart watch.

[0078] The meal image recording unit 157 takes meal images with a camera and records the meal date and time data and the food image for each dish.

[0079] The nutrition calculation unit 158 ​​performs AI image analysis on the food images recorded in the meal image recording unit 157 and estimates and calculates nutrition.

[0080] The nutrition recording unit 159 records the nutrition (nutritional components: see FIG. 12) calculated by the nutrition calculation unit 158.

[0081] The pre-disease visualization unit 160 displays a pre-disease avatar. Various other data may also be visualized.

[0082] The virtual doctor collaboration unit 161 performs the virtual doctor collaboration described above.

[0083] The real doctor collaboration unit 162 performs the real doctor collaboration as described above.

[0084] The online collaboration dashboard unit 163 controls the entire online collaboration. It may also control the entire health condition data, such as trend display of health management data and various chart displays. It may also output reports of health management data.

[0085] The health management application 152 described above records the subject's subjective symptoms. For example, sleepiness upon waking up is evaluated on a three-point scale and recorded. This is to obtain data from the subject that cannot be obtained from measurement data and to use it to determine whether the subject is healthy or not. For example, sleepiness may be evaluated as "refreshed," "normal," or "sleepy." Other subjective symptom parameters may also be set as subjective symptom parameters.

[0086] In this specification, "pre-illness" refers to "a state in which there are no subjective symptoms but tests show abnormalities" and "a state in which there are subjective symptoms but tests show no abnormalities." "Health" refers to a state in which there are no subjective symptoms but tests show no abnormalities. "Illness" refers to a crossover state in which there are subjective symptoms but tests show abnormalities. In addition, in this specification, online medical consultations are also considered to include nutritional consultations by registered dietitians.

[0087] Figure 17 shows an example of a display pattern displayed in the in vivo cybernetic avatar display area. The in vivo CA measurement unit associates the type of in vivo CA measured with the associated body part to create an avatar, which shows the healthy and pre-illness states. Since the subjects are not experts, simply displaying the measurement information numerically encourages behavioral change in the subjects. When an in vivo sensor is placed in the brain, the displayed image of the brain avatar changes depending on the healthy, pre-illness (1) and pre-illness (2) states. For a healthy state, an image of a smiling brain is displayed; for pre-illness (1), an image of the brain appearing slightly distressed is displayed; and for pre-illness (2), an image of the brain appearing distressed is displayed.

[0088] Similarly, if an in-vivo sensor is placed in the heart, the avatar image of the heart is changed and displayed as healthy, pre-disease (1), or pre-disease (2). If an in-vivo sensor is placed in the stomach, the avatar image of the stomach is changed and displayed as healthy, pre-disease (1), or pre-disease (2). Furthermore, if an in-vivo sensor is placed in the intestine, the avatar image of the intestine is changed and displayed as healthy, pre-disease (1), or pre-disease (2). The types of in-vivo CA may be configured to create and display avatar images for each organ.

[0089] In this way, displaying an in vivo cybernetic avatar appeals intuitively to the subject's visual sense, and is expected to promote behavioral change.

[0090] Figure 18 shows example display patterns for avatars of the three major diseases. Examples of the diseases shown are cancer, cerebral infarction, and heart disease. For cancer, the healthy state shows an avatar image attacking cancer cells, the pre-disease (1) state shows an avatar image in which the cancer cells are slightly gaining momentum, and the pre-disease (2) state shows an avatar image in which the cancer cells are also gaining momentum. For cerebral infarction, the healthy state shows an avatar image in which red blood cells are swimming freely through blood vessels, the pre-disease (1) state shows an avatar image in which the blood vessels have slightly developed arteriosclerosis, making it difficult for the red blood cells to swim, and the pre-disease (2) state shows an avatar image in which the blood vessels have developed arteriosclerosis, making it difficult for the red blood cells to swim. For heart disease, the healthy state shows an avatar image in which the heart's blood vessels are slightly clogged, the pre-disease (1) state shows an avatar image in which the heart's blood vessels are slightly clogged, causing some pain to the heart, and the pre-disease (2) state shows an avatar image in which the heart's blood vessels are clogged and enlarged, causing pain to the heart. Here, the display of avatar images for the three major diseases has been described, but similar avatar images may also be displayed for hypertension, dementia, diabetes, hyperlipidemia, hyperuricemia, pulmonary hypertension, and the like.

[0091] Figure 19 shows a list of recommendations in the 2023 Physical Activity and Exercise Guide for Health Promotion. The list of recommendations in Figure 19 is used as the standard for the amount of physical activity displayed in the physical activity display area. In Figure 19, the subjects are divided into elderly people, adults, and children, and physical activity (including daily activities and exercise) and sedentary behavior. For elderly people, daily activities are walking or physical activity of an equivalent intensity (3 METs or more) for 40 minutes or more per day (6,000 steps or more per day), and exercise is multi-component exercise such as aerobic exercise, strength training, balance exercise, and flexibility exercise for 3 days or more per week, and strength training for 2-3 days per week.

[0092] For example, if the subject is elderly and 67 years old, the recommended daily step count is 6,000 steps or more. Therefore, the physical activity display area displays 7,500 steps for daily activities, exceeding the recommended value and indicating OK. Exercise options are displayed, such as one aerobic exercise (OK), three strength training sessions (OK), and two balance exercise sessions (OK). The exercise display is calculated by converting the exercise into METs and exercises. METs are a unit of measurement that expresses the intensity of physical activity as a multiple of resting activity. For example, sitting and resting corresponds to 1 MET, while normal walking corresponds to 3 METs. Exercise is a unit of measurement that expresses the amount of physical activity. It is calculated by multiplying the intensity of physical activity (METs) by the duration of physical activity (hours). The more intense the physical activity, the shorter the time required for one exercise. Energy expenditure (kcal) is calculated by 1.05 x exercise (METs x hours) x body weight (kg).

[0093] FIG. 20 is a flowchart illustrating a health monitoring method according to an embodiment of the present invention.

[0094] In the health monitoring method in the online collaboration system of this embodiment, the medical measurement device includes at least a blood pressure monitor, a thermometer, and a weight scale, and the medical measurement device includes a biological information acquisition step 801 of acquiring biological information of the subject using at least the blood pressure monitor, the thermometer, and the weight scale; system The device includes a body-wearable device and comprises a behavioral information acquisition step 802 for acquiring behavioral information, which is physical activity information, from the body-wearable device; a dietary data acquisition step 803 for acquiring dietary data obtained by calculating nutritional information from dietary image data of the subject captured by a dietary image sensor; a health checkup result information acquisition step 804 for acquiring health checkup result information including at least brain images and in vivo endoscopic images; an analysis step 805 for analyzing the acquired biometric information, behavioral information, and health checkup result information of the subject; and a health condition estimation step 806 for estimating the future health condition of the subject using the results of the above analysis and the biometric information, behavioral information, and health checkup result information of the subject.

[0095] At least blood pressure data, body temperature data, and weight data are acquired in the biological information acquisition step 801. Other biological information that may be acquired includes blood glucose level data.

[0096] In a behavioral information acquisition step 802, physical activity information (step count data, muscle training data, etc.) is acquired.

[0097] In the dietary data acquisition step 803, dietary data is acquired by calculating nutritional information from dietary image data of the subject captured by a dietary image sensor. By acquiring actual dietary data in addition to the subject's (patient's) biological information, doctors can use the dietary data to diagnose lifestyle-related diseases such as hypertension and diabetes, and can explain the patient's health condition while checking it with a 3D avatar during online medical consultations (remote diagnosis), thereby improving the accuracy of online medical consultations.

[0098] In the health check result information acquisition step 804, in-vivo information of the subject's brain image and in-vivo endoscopic image is acquired and used in the next analysis step 805.

[0099] In the analysis step 805, the acquired biological information, behavioral information, and health check result information of the subject are analyzed. Genetic testing and other information may also be added to the analysis data. Creating a health status model using health check big data and utilizing AI analysis technology can improve prediction accuracy. Furthermore, adding blood glucose level data monitored for up to 24 hours a day can further improve the prediction accuracy of AI analysis. When blood glucose levels rise, the avatar's blood vessels turn black, encouraging improvement of lifestyle-related diseases and allowing users to intuitively understand their health condition.

[0100] In the health condition estimation step 806, the results of the above analysis and the subject's biological information, behavioral information, and health check result information are used to estimate the subject's future health condition, predict whether the subject is healthy or not, and predict areas that will deteriorate in the future.

[0101] FIG. 21 is a flowchart illustrating a health monitoring method according to another embodiment of the present invention.

[0102] A health monitoring method according to another embodiment includes an electroencephalogram (EEG) measuring step 901, a biological information acquiring step 902, a behavioral information acquiring step 903, a dietary data acquiring step 904, a health checkup result information acquiring step 905, an analysis step 906, and a health condition estimating step 907. Here, the biological information acquiring step 902, the behavioral information acquiring step 903, the health checkup result information acquiring step 904, the dietary data acquiring step 905, the analysis step 906, and the health condition estimating step 907 are similar to the biological information acquiring step 901, the behavioral information acquiring step 902, the health checkup result information acquiring step 903, the analysis step 904, and the health condition estimating step 905, respectively, and therefore their explanations will be omitted and only the electroencephalogram measuring step 901 will be described.

[0103] In the EEG measurement step 901, EEG measurements of the subject's sleep state are performed at home. In the EEG measurement step 901, a sleep test equivalent to a polysomnogram is performed. The EEG measurement device used is the one shown in Figure 5 of Patent Publication No. 7614679. By measuring sleep, the influence of the subject's sleep on the progression to lifestyle-related diseases can be taken into account, improving the accuracy of predicting the subject's health condition. Furthermore, because actual dietary data is acquired, the AI ​​can predict future risks and reflect them in the appearance of the avatar. If the risk of obesity increases, the avatar will make changes such as gaining weight, thereby notifying the patient.

[0104] In this way, according to an embodiment of the present invention, it is possible to link with a health monitoring system that uses an in vivo cybernetic avatar, and to realize an online linkage system that improves the accuracy of online medical consultations.

[0105] Thus, according to another embodiment of the present invention, a health monitoring method can be realized that can predict the current health state or the location where a future illness will worsen. [Explanation of symbols]

[0106] 1, 2, 3,... Subject terminal 4. Administrator terminal 5. Cloud 6 Management Server 61 Virtual Doctor Club 62 Bio-Cybernetic Avatar Department 7 Online Collaboration Server 71 Virtual Doctor Collaboration Department 72 Real Doctor Collaboration Department 73 databases 8 Virtual Doctor Terminal 9 Real Doctor Terminal 9a Hospital terminal 9b Family doctor 10, 11, 12 Common data conversion unit

Claims

1. In an online collaboration system in which a subject terminal for a subject receiving online medical care, a virtual doctor terminal for at least a virtual doctor who is a virtual doctor, a real doctor terminal for specialists including at least a doctor, an online collaboration server, and a management server having a virtual doctor unit and an in-vivo cybernetic avatar unit are connected via a cloud, and the subject terminal is linked to the virtual doctor terminal by the online collaboration server or to the real doctor terminal by the selection of the subject, and online medical care is provided, The online collaboration server a database storing subject information and in-vivo cybernetic avatar information; The online collaboration server When a virtual doctor selection request is received from the subject terminal, the subject information and in-vivo cybernetic avatar information stored in the database are transferred to the virtual doctor terminal, and a virtual online medical examination is performed. When a real doctor selection request is received from the subject terminal, the subject information and in-vivo cybernetic avatar information stored in the database are transferred to the real doctor terminal, and online medical treatment is performed. An online collaboration system characterized by:

2. The online collaboration system described in claim 1, characterized in that the subject information includes the subject's body temperature data, blood pressure data, weight data, and blood glucose data measured by medical measurement system devices, and the subject's exercise record data, sleep status data, and dietary data measured by non-medical measurement system devices.

3. The online collaboration system according to claim 1 or 2, characterized in that the in vivo cybernetic avatar information is spatiotemporal internal body environment information measured by a measurement means for measuring spatiotemporal internal body environment information using an in vivo cybernetic avatar.

4. The online collaboration system described in claim 2, characterized in that the database stores measurement data from the medical measurement system device and measurement data from the non-medical measurement system device after converting them into common data with the same data structure.

5. 3. A health monitoring method in an online collaboration system according to claim 2, comprising: the medical measurement system device includes at least a blood pressure monitor, a thermometer, and a weight scale, and a biological information acquiring step of acquiring biological information of the subject using the blood pressure monitor, the thermometer, and the weight scale; the non-medical measurement system device includes a body-wearable device, and a behavioral information acquisition step of acquiring behavioral information, which is physical activity information, from the body-wearable device; a meal data acquisition step of acquiring meal data obtained by calculating nutritional information from meal image data of the subject captured by a meal image sensor; a health checkup result information acquisition step of acquiring health checkup result information including at least a brain image and an in-vivo endoscope image; an analysis step of analyzing the acquired biological information, behavioral information, dietary data, and health check result information of the subject; an estimation step of estimating the subject's future health condition using the results of the analysis, the subject's biological information, behavioral information, dietary data, and health check result information; A health monitoring method in an online collaboration system, comprising:

6. The health monitoring method in an online collaborative system according to claim 5, further comprising an electroencephalogram (EEG) measurement step of measuring the subject's sleep state at home, wherein the electroencephalogram (EEG) measurement step performs a sleep test equivalent to a polysomnogram test.

7. 6. A health monitoring method in an online collaborative system according to claim 5, wherein the medical measurement system device further comprises a blood glucose meter, a blood glucose data acquisition step of acquiring blood glucose data of the subject using the blood glucose meter, and an estimation step of using the blood glucose data acquired in the blood glucose data acquisition step in the analysis, and estimating the future health condition of the subject using the results of the analysis, biological information including the blood glucose data of the subject, behavioral information, dietary data, and health check result information.

8. 6. The health monitoring method in an online collaborative system according to claim 5, wherein the virtual doctor is a doctor's avatar, and the avatar can be changed to a family member, friend, or doctor-like avatar.

9. The health monitoring method in an online collaborative system according to claim 8, wherein the avatar is configured to be able to converse by voice using multimodal generation AI.

Images