System

Abstract

To provide a system that administers an appropriate amount of insulin on the basis of an amount of carbohydrates in a meal by allowing diabetic patient to accurately grasp the amount of carbohydrates.SOLUTION: A system according to an embodiment includes a reception part, and a measurement part, and a system. The reception part receives image information showing an image of a meal of a patient. The measurement part analyzes the image information received by the reception part by using a generative AI and measures an amount of carbohydrates. The system calculates an amount of insulin corresponding to the patient on the basis of the amount of carbohydrates measured by the measurement part and a blood glucose level of the patient obtained from a continuous blood glucose meter.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional techniques have the problem that it is difficult for diabetic patients to accurately grasp the carbohydrate content of their meals and administer an appropriate amount of insulin based on that information.

[0005] The system according to the embodiment aims to enable a diabetic patient to accurately grasp the amount of carbohydrates in a meal and administer an appropriate amount of insulin based on that information. [Means for solving the problem]

[0006] The system according to the embodiment includes a reception unit, a measurement unit, and a system. The reception unit receives image information showing an image of a patient's meal. The measurement unit analyzes the image information received by the reception unit using a generation AI to measure the amount of carbohydrates. The system calculates and administers an amount of insulin appropriate for the patient based on the amount of carbohydrates measured by the measurement unit and the patient's blood glucose level obtained from a continuous glucose monitor. Effect of the Invention

[0007] The system according to the embodiment enables a diabetic patient to accurately grasp the amount of carbohydrates in a meal and administer an appropriate amount of insulin based on that information. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The automatic insulin administration system according to the embodiment of the present invention is a system for diabetes patients. In this system, a patient takes a photo of a meal with a smartphone, and the image information is received by the reception unit. Next, the image information is analyzed using a generation AI to measure the amount of carbohydrates. Based on the measured amount of carbohydrates and the patient's blood glucose level obtained from a continuous glucose monitor, the insulin pump system calculates and administers an amount of insulin appropriate for the patient. Also included is a system that obtains information on the patient's exercise schedule, predicts hyperglycemia or hypoglycemia based on the information, and notifies the patient. For example, a patient takes a photo of a meal with a smartphone. For example, a photo of a meal such as bread for breakfast or pasta for lunch is taken. This image information is received by the reception unit. Next, the generation AI analyzes the image information received by the reception unit. The generation AI recognizes the contents of the meal from the image information and measures the amount of carbohydrates. For example, the amount of carbohydrates is measured from a photo of bread, and the amount of carbohydrates is similarly measured from a photo of pasta. Based on the measured amount of carbohydrates and the patient's blood glucose level obtained from a continuous glucose monitor, the insulin pump system calculates an amount of insulin appropriate for the patient. For example, if the amount of carbohydrates is high, the amount of insulin is increased, and if the amount of carbohydrates is low, the amount of insulin is decreased. Based on the calculation results, the insulin pump system automatically administers insulin. In addition, information on the patient's exercise schedule is acquired. For example, information on the exercise schedule (such as jogging or walking) entered by the patient in advance is acquired. Based on this information, the generation AI predicts hyperglycemia or hypoglycemia. For example, a high amount of exercise increases the risk of hypoglycemia, and a low amount of exercise increases the risk of hyperglycemia. By notifying the patient of the prediction results, the patient can take measures in advance. This system allows diabetic patients to receive appropriate insulin administration according to their diet and exercise, thereby reducing the risk of hyperglycemia and hypoglycemia. In addition, patients can easily use the system using their smartphones, making it easy to manage in daily life. As a result, the automatic insulin administration system allows diabetic patients to receive appropriate insulin administration according to their diet and exercise, thereby reducing the risk of hyperglycemia and hypoglycemia.

[0029] The automatic insulin administration system according to the embodiment includes a reception unit, a measurement unit, and an insulin pump system. The reception unit receives image information showing an image of a patient's meal. The image information includes, for example, a photo, a video, a scanned image, and the like, but is not limited to such examples. The reception unit receives, for example, a photo of a meal taken with a smartphone. The reception unit can also receive a video of a meal taken in a video format. Furthermore, the reception unit can also receive a scanned image of a meal. The measurement unit uses a generation AI to analyze the image information received by the reception unit and measure the amount of carbohydrates. The generation AI analyzes the image information using, for example, techniques such as deep learning and neural networks. For example, the generation AI uses a deep learning model to measure the amount of carbohydrates from an image of a meal. The generation AI can also measure the amount of carbohydrates from an image of a meal using a neural network. Furthermore, the generation AI can execute an algorithm for measuring the amount of carbohydrates based on image information. For example, the generation AI measures the amount of carbohydrates using a deep learning model that inputs an image of a meal and outputs the amount of carbohydrates. The insulin pump system calculates and administers an amount of insulin appropriate for the patient based on the amount of carbohydrates measured by the measuring unit and the patient's blood glucose level obtained from the continuous blood glucose meter. The amount of insulin is calculated based on, for example, body weight, meal contents, blood glucose level, etc. The insulin pump system increases the amount of insulin when the amount of carbohydrates is high, and decreases the amount of insulin when the amount of carbohydrates is low. The insulin pump system can also adjust the amount of insulin based on the blood glucose level obtained from the continuous blood glucose meter. Furthermore, the insulin pump system can also calculate the amount of insulin based on the body weight and meal contents of the patient. For example, the insulin pump system calculates the amount of insulin using an algorithm that inputs the amount of carbohydrates and the blood glucose level and outputs the amount of insulin. As a result, the automatic insulin administration system according to the embodiment can automatically calculate and administer an appropriate amount of insulin based on the patient's meal information.

[0030] The reception unit receives image information showing an image of the patient's meal. The image information includes, for example, a photograph, a video, a scanned image, and the like, but is not limited to such examples. The reception unit receives, for example, a photograph of the meal taken with a smartphone. The reception unit can also receive a video of the meal. Furthermore, the reception unit can also receive a scanned image of the meal. The reception unit can provide a dedicated application or a web interface to receive these image information. For example, the patient can take a photo of the meal using an app on a smartphone and send the photo to the reception unit through the app. Similarly, a video of the meal can be taken using an app and sent to the reception unit. As for the scanned image, for example, an image of the meal can be scanned using a home scanner or a scanning function of a smartphone and the image can be sent to the reception unit. The reception unit can centrally manage the received image information and store it in a database for sending to the measurement unit. This allows the reception unit to efficiently receive and manage various forms of image information from the patient. Furthermore, the reception unit can have a function of checking the quality of the received image information and filtering out unclear or inappropriate images. For example, the image resolution and brightness can be automatically evaluated, and if an image does not meet the criteria, a message can be sent to the patient urging them to resubmit it. This allows the reception unit to provide high-quality image information for the measurement unit to perform accurate analysis.

[0031] The measurement unit uses the generation AI to analyze the image information accepted by the reception unit and measure the amount of carbohydrates. The generation AI analyzes the image information using technologies such as deep learning and neural networks. Specifically, the generation AI uses a deep learning model to measure the amount of carbohydrates from the image of the meal. The deep learning model is pre-trained using a large dataset of meal images and their corresponding carbohydrate amounts, and can estimate the amount of carbohydrates with high accuracy even for new images. For example, the generation AI takes an image of a meal as input, identifies the type and amount of each food in the image, and calculates the amount of carbohydrates contained in each food. The neural network can have a layer for extracting image features and classifying the type of food, and a layer for estimating the amount of carbohydrates. In addition, the generation AI can also execute an algorithm for measuring the amount of carbohydrates based on the image information. For example, the generation AI measures the amount of carbohydrates using a deep learning model that takes an image of a meal as input and outputs the amount of carbohydrates. The measurement unit stores the amount of carbohydrates measured by the generation AI in a database and transmits it to the insulin pump system. This allows the measurement unit to quickly and accurately analyze the accepted image information and measure the amount of carbohydrates. Furthermore, the measurement unit can be equipped with a function to verify the analysis results of the generation AI and manually correct them if necessary. For example, if the generation AI misidentifies a particular food, an operator of the measurement unit can manually correct it and measure the accurate carbohydrate amount. This allows the measurement unit to combine the high analytical capabilities of the generation AI with human judgment to achieve more accurate carbohydrate amount measurements.

[0032] The insulin pump system calculates and administers an amount of insulin appropriate for the patient based on the amount of carbohydrates measured by the measuring unit and the patient's blood glucose level obtained from the continuous blood glucose monitor. The amount of insulin is calculated based on, for example, body weight, meal contents, blood glucose level, etc. The insulin pump system increases the amount of insulin when the amount of carbohydrates is high, and decreases the amount of insulin when the amount of carbohydrates is low. The insulin pump system can also adjust the amount of insulin based on the blood glucose level obtained from the continuous blood glucose monitor. Furthermore, the insulin pump system can calculate the amount of insulin based on the body weight and meal contents of the patient. For example, the insulin pump system calculates the amount of insulin using an algorithm that inputs the amount of carbohydrates and the blood glucose level and outputs the amount of insulin. The insulin pump system includes a mechanism for automatically administering the calculated amount of insulin to the patient. Specifically, the insulin pump system administers insulin through a cannula inserted subcutaneously. The cannula is fixed to the patient's skin and administers insulin at an appropriate time based on instructions from the insulin pump system. The insulin pump system can monitor the patient's blood glucose level in real time and adjust the amount of insulin as necessary. For example, if the blood glucose level rises suddenly, the insulin pump system stabilizes the blood glucose level by administering additional insulin. In addition, if blood glucose levels drop, insulin administration can be temporarily stopped. This allows the insulin pump system to keep the patient's blood glucose levels in an optimal range at all times. Furthermore, the insulin pump system can customize the insulin administration schedule according to the patient's lifestyle and activity level. For example, it can reduce the amount of insulin on days when the patient exercises a lot and increase the amount of insulin on days when the patient eats a lot. This allows the insulin pump system to provide optimal insulin administration according to the individual needs of the patient.

[0033] The automatic insulin administration system includes an acquisition unit that acquires information on the patient's exercise schedule. The acquisition unit acquires information on the patient's exercise schedule. The exercise schedule includes, for example, the type of exercise, time, and frequency, but is not limited to such examples. The acquisition unit acquires, for example, information on the exercise schedule input by the patient in advance. The acquisition unit can also acquire information on the exercise schedule input by the patient using a smartphone or a wearable device. Furthermore, the acquisition unit can also be equipped with a sensor for automatically acquiring the patient's exercise schedule. For example, the acquisition unit acquires information on the exercise schedule from a smartphone or a wearable device when the patient performs exercise such as jogging or walking. This makes it possible to adjust insulin administration based on the patient's exercise schedule. A part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input information on the patient's exercise schedule to AI and cause AI to perform analysis of the exercise schedule.

[0034] The automatic insulin administration system includes a prediction unit that predicts fluctuations in the patient's blood glucose level based on the information of the exercise schedule acquired by the acquisition unit. The prediction unit predicts fluctuations in the patient's blood glucose level based on the information of the exercise schedule acquired by the acquisition unit. The fluctuations in the blood glucose level are predicted based on, for example, past data, exercise intensity, meal contents, and the like. The prediction unit predicts fluctuations in the blood glucose level based on, for example, past data. The prediction unit can also predict fluctuations in the blood glucose level based on exercise intensity. Furthermore, the prediction unit can predict fluctuations in the blood glucose level based on meal contents. For example, the prediction unit predicts fluctuations in the blood glucose level using an algorithm that inputs past data and outputs fluctuations in the blood glucose level. This makes it possible to predict hyperglycemia or hypoglycemia based on the exercise schedule. A part or all of the above-mentioned processing in the prediction unit may be performed using, for example, AI, or may be performed without using AI. For example, the prediction unit can input information about the exercise schedule to AI and cause AI to execute prediction of fluctuations in the blood glucose level.

[0035] The automatic insulin administration system includes a notification unit that notifies the patient of the prediction result by the prediction unit. The notification unit notifies the patient of the prediction result by the prediction unit. The prediction result includes, for example, a numerical value, a graph, an alert, and the like, but is not limited to such examples. The notification unit notifies the patient of the prediction result, for example, as a numerical value. The notification unit can also notify the patient of the prediction result as a graph. Furthermore, the notification unit can also notify the patient of the prediction result as an alert. For example, the notification unit displays the prediction result as a numerical value and notifies the patient. In this way, by notifying the patient of the prediction result, measures can be taken in advance. A part or all of the above-mentioned processing in the notification unit may be performed, for example, using AI, or may be performed without using AI. For example, the notification unit can input the prediction result to AI and cause the AI ​​to execute the timing and method of notification.

[0036] The reception unit analyzes the patient's past meal history and selects the optimal image reception method. The reception unit, for example, proposes optimal shooting angles and lighting conditions based on images of meals taken by the patient in the past. The reception unit can also detect a specific meal pattern from the patient's past meal history and adjust the image reception method based on the pattern. The reception unit can also analyze the quality of images taken by the patient in the past and propose optimal camera settings. For example, the reception unit analyzes images of meals taken by the patient in the past and proposes optimal shooting angles and lighting conditions. This allows for more accurate measurement of carbohydrate amounts by selecting the optimal image reception method based on the past meal history. Part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input past meal history data to AI and cause AI to select the optimal image reception method.

[0037] The reception unit performs filtering based on the patient's current health condition and the type of meal when receiving image information. For example, when the patient's current blood glucose level is high, the reception unit preferentially receives images of meals with a high carbohydrate content. The reception unit can also filter images of a specific meal category (e.g., low-carbohydrate meals) based on the patient's current health condition. Furthermore, the reception unit can filter images of meals containing specific ingredients based on the patient's allergy information. For example, when the patient's current blood glucose level is high, the reception unit preferentially receives images of meals with a high carbohydrate content. This makes it possible to receive more appropriate image information by filtering based on the patient's health condition and the type of meal. Part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the patient's health condition data to AI and leave the execution of filtering to AI.

[0038] The measurement unit adjusts the level of detail of the measurement based on the type and amount of the meal during measurement. For example, in the case of a high-carbohydrate meal, the measurement unit performs detailed measurement. In addition, in the case of a low-carbohydrate meal, the measurement unit can also perform simplified measurement. Furthermore, in the case of a large amount of meal, the measurement unit can perform detailed measurement. For example, the measurement unit adjusts the level of detail of the measurement based on the type and amount of meal. This allows for more accurate measurement of the amount of carbohydrates by adjusting the level of detail of the measurement based on the type and amount of meal. Part or all of the above-mentioned processing in the measurement unit may be performed, for example, using AI, or may be performed without using AI. For example, the measurement unit can input data on the type and amount of meal to AI and cause AI to adjust the level of detail of the measurement.

[0039] The measurement unit applies different measurement algorithms depending on the category of the meal when measuring. For example, in the case of bread, the measurement unit applies a specific algorithm to measure. In addition, in the case of pasta, the measurement unit can also apply a different algorithm to measure. Furthermore, in the case of desserts, the measurement unit can also apply yet another algorithm to measure. For example, the measurement unit applies different measurement algorithms depending on the category of the meal. This makes it possible to measure the amount of carbohydrates more accurately by applying different measurement algorithms depending on the category of the meal. Some or all of the above-mentioned processing in the measurement unit may be performed, for example, using AI, or may be performed without using AI. For example, the measurement unit can input meal category data to AI and cause AI to apply the measurement algorithm.

[0040] When administering insulin, the insulin pump system calculates the optimal dosage by referring to the patient's past blood glucose data. The insulin pump system calculates the optimal insulin dosage based on, for example, the patient's blood glucose data from the past week. The insulin pump system can also adjust the insulin dosage in consideration of long-term trends based on the patient's blood glucose data from the past month. Furthermore, the insulin pump system can calculate the optimal insulin dosage by combining the patient's past meal pattern and blood glucose data. For example, the insulin pump system calculates the optimal insulin dosage based on the patient's blood glucose data from the past week. This allows for more accurate insulin administration by calculating the optimal insulin dosage based on the past blood glucose data. Some or all of the above-mentioned processing in the insulin pump system may be performed, for example, using AI, or may be performed without using AI. For example, the insulin pump system can input past blood glucose data into AI and have AI execute calculation of the optimal dosage.

[0041] The insulin pump system customizes the administration method based on the patient's current health condition when administering insulin. For example, when the patient's current blood glucose level is high, the insulin pump system selects a method of administering insulin quickly. In addition, when the patient's current health condition is stable, the insulin pump system can also apply a normal insulin administration method. Furthermore, when the patient is in poor health, the insulin pump system can select a method of reducing the administration amount or temporarily suspending administration. For example, when the patient's current blood glucose level is high, the insulin pump system selects a method of administering insulin quickly. This allows for more appropriate insulin administration by customizing the administration method based on the current health condition. Some or all of the above-mentioned processing in the insulin pump system may be performed, for example, using AI, or may be performed without using AI. For example, the insulin pump system can input current health condition data into AI and have AI execute customization of the administration method.

[0042] The acquisition unit analyzes the patient's past exercise history and selects the optimal acquisition method. The acquisition unit proposes an optimal exercise schedule based on, for example, the type and frequency of exercise performed by the patient in the past. The acquisition unit can also detect a specific exercise pattern from the patient's past exercise history and adjust the exercise schedule based on the pattern. The acquisition unit can also analyze the effect of the patient's past exercise and propose an optimal exercise schedule. For example, the acquisition unit proposes an optimal exercise schedule based on the type and frequency of exercise performed by the patient in the past. This makes it possible to acquire a more accurate exercise schedule by selecting an optimal acquisition method based on the past exercise history. Part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, AI, or may be performed without using AI. For example, the acquisition unit can input past exercise history data to AI and cause AI to select an optimal acquisition method.

[0043] The prediction unit optimizes the prediction algorithm by referring to the patient's past blood glucose data at the time of prediction. The prediction unit optimizes the prediction algorithm based on, for example, the patient's blood glucose data from the past week. The prediction unit can also adjust the prediction algorithm in consideration of long-term trends based on the patient's blood glucose data from the past month. Furthermore, the prediction unit can optimize the prediction algorithm by combining the patient's past meal patterns and blood glucose data. For example, the prediction unit optimizes the prediction algorithm based on the patient's blood glucose data from the past week. As a result, by optimizing the prediction algorithm based on the past blood glucose data, more accurate prediction of hyperglycemia or hypoglycemia is possible. A part or all of the above-mentioned processing in the prediction unit may be performed, for example, using AI, or may be performed without using AI. For example, the prediction unit can input past blood glucose data to AI and cause AI to execute optimization of the prediction algorithm.

[0044] At the time of notification, the notification unit selects the optimal notification method by referring to the patient's past notification history. The notification unit, for example, selects the most effective notification method from the patient's past notification history. The notification unit can also analyze the patient's past notification history and select an effective notification method for a specific time period. Furthermore, the notification unit can determine the optimal notification frequency based on the patient's past notification history. For example, the notification unit selects the most effective notification method from the patient's past notification history. As a result, more effective notification is possible by selecting the optimal notification method based on the past notification history. A part or all of the above-mentioned processing in the notification unit may be performed, for example, using AI, or may be performed without using AI. For example, the notification unit can input past notification history data to AI and cause AI to select the optimal notification method.

[0045] The notification unit selects the optimal notification method in consideration of the device information of the patient at the time of notification. For example, when the patient is using a smartphone, the notification unit performs a push notification. In addition, when the patient is using a tablet, the notification unit can also provide a notification method optimized for a large screen. Furthermore, when the patient is using a smartwatch, the notification unit can also provide a simple and highly visible notification method. For example, when the patient is using a smartphone, the notification unit performs a push notification. This enables more effective notification by selecting the optimal notification method in consideration of the device information of the patient. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using AI, or may be performed without using AI. For example, the notification unit can input device information to AI and cause AI to select the optimal notification method.

[0046] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0047] The automatic insulin administration system may further include a sleep monitoring unit that monitors the patient's sleep patterns. The sleep monitoring unit records the quality and duration of the patient's sleep and adjusts the insulin dosage based on the quality and duration of the patient's sleep. For example, if the patient is not getting enough sleep, the insulin dosage may be increased since insulin sensitivity may be reduced. Conversely, if the patient is getting enough sleep, the insulin dosage may be decreased. Furthermore, the sleep monitoring unit may analyze the patient's sleep patterns over the long term and provide advice to improve the quality of sleep. This may improve the patient's overall health.

[0048] The automatic insulin administration system may further include a meal recorder that records the patient's meal contents in detail. The meal recorder records the food ingredients and their amounts ingested by the patient in detail, and adjusts the insulin dosage based on this. For example, if the patient eats a high-fat meal, the insulin dosage can be increased because insulin sensitivity may be reduced. The meal recorder can also suggest a balanced meal to the patient and support the improvement of the meal contents. This can strengthen the patient's dietary management and optimize the effect of insulin administration.

[0049] The automatic insulin administration system may further include an exercise performance monitoring unit that monitors the exercise performance of the patient. The exercise performance monitoring unit records the exercise intensity and duration of the patient, and adjusts the insulin dosage based on the recorded exercise intensity and duration. For example, if the patient performs high-intensity exercise, the insulin dosage can be reduced because insulin sensitivity may be improved. The exercise performance monitoring unit may also suggest an appropriate exercise plan to the patient to maximize the effect of exercise. This can strengthen the patient's exercise management and optimize the effect of insulin administration.

[0050] The automated insulin administration system may further include a social activity monitoring unit that monitors the patient's social activities. The social activity monitoring unit records the frequency of the patient's social events and outings, and adjusts the insulin administration amount based on the record. For example, if the patient frequently goes out, the timing and content of meals may change, so the insulin administration amount can be adjusted. The social activity monitoring unit may also suggest appropriate social activities to the patient to support social health. This allows the patient's social activities to be managed and the effect of insulin administration to be optimized.

[0051] The process flow of the first embodiment will be briefly described below.

[0052] Step 1: The reception unit receives image information showing an image of the patient's meal. The image information includes, for example, a photo, a video, a scanned image, etc. The reception unit can receive a photo of the meal taken with a smartphone, a video of the meal, or a scanned image of the meal. Step 2: The measurement unit uses the generation AI to analyze the image information received by the reception unit and measure the amount of carbohydrates. The generation AI analyzes the image information using technologies such as deep learning and neural networks, and measures the amount of carbohydrates from the image of the meal. Step 3: The insulin pump system calculates and administers the appropriate amount of insulin for the patient based on the amount of carbohydrates measured by the measuring unit and the patient's blood glucose level obtained from the continuous glucose monitor. The amount of insulin is calculated based on weight, meal contents, blood glucose level, etc. For example, if the amount of carbohydrates is high, the amount of insulin is increased, and if the amount of carbohydrates is low, the amount of insulin is decreased. The amount of insulin can also be adjusted based on the blood glucose level obtained from the continuous glucose monitor.

[0053] (Example 2) The automatic insulin administration system according to the embodiment of the present invention is a system for diabetes patients. In this system, a patient takes a photo of a meal with a smartphone, and the image information is received by the reception unit. Next, the image information is analyzed using a generation AI to measure the amount of carbohydrates. Based on the measured amount of carbohydrates and the patient's blood glucose level obtained from a continuous glucose monitor, the insulin pump system calculates and administers an amount of insulin appropriate for the patient. Also included is a system that obtains information on the patient's exercise schedule, predicts hyperglycemia or hypoglycemia based on the information, and notifies the patient. For example, a patient takes a photo of a meal with a smartphone. For example, a photo of a meal such as bread for breakfast or pasta for lunch is taken. This image information is received by the reception unit. Next, the generation AI analyzes the image information received by the reception unit. The generation AI recognizes the contents of the meal from the image information and measures the amount of carbohydrates. For example, the amount of carbohydrates is measured from a photo of bread, and the amount of carbohydrates is similarly measured from a photo of pasta. Based on the measured amount of carbohydrates and the patient's blood glucose level obtained from a continuous glucose monitor, the insulin pump system calculates an amount of insulin appropriate for the patient. For example, if the amount of carbohydrates is high, the amount of insulin is increased, and if the amount of carbohydrates is low, the amount of insulin is decreased. Based on the calculation results, the insulin pump system automatically administers insulin. In addition, information on the patient's exercise schedule is acquired. For example, information on the exercise schedule (such as jogging or walking) entered by the patient in advance is acquired. Based on this information, the generation AI predicts hyperglycemia or hypoglycemia. For example, a high amount of exercise increases the risk of hypoglycemia, and a low amount of exercise increases the risk of hyperglycemia. By notifying the patient of the prediction results, the patient can take measures in advance. This system allows diabetic patients to receive appropriate insulin administration according to their diet and exercise, thereby reducing the risk of hyperglycemia and hypoglycemia. In addition, patients can easily use the system using their smartphones, making it easy to manage in daily life. As a result, the automatic insulin administration system allows diabetic patients to receive appropriate insulin administration according to their diet and exercise, thereby reducing the risk of hyperglycemia and hypoglycemia.

[0054] The automatic insulin administration system according to the embodiment includes a reception unit, a measurement unit, and an insulin pump system. The reception unit receives image information showing an image of a patient's meal. The image information includes, for example, a photo, a video, a scanned image, and the like, but is not limited to such examples. The reception unit receives, for example, a photo of a meal taken with a smartphone. The reception unit can also receive a video of a meal taken in a video format. Furthermore, the reception unit can also receive a scanned image of a meal. The measurement unit uses a generation AI to analyze the image information received by the reception unit and measure the amount of carbohydrates. The generation AI analyzes the image information using, for example, techniques such as deep learning and neural networks. For example, the generation AI uses a deep learning model to measure the amount of carbohydrates from an image of a meal. The generation AI can also measure the amount of carbohydrates from an image of a meal using a neural network. Furthermore, the generation AI can execute an algorithm for measuring the amount of carbohydrates based on image information. For example, the generation AI measures the amount of carbohydrates using a deep learning model that inputs an image of a meal and outputs the amount of carbohydrates. The insulin pump system calculates and administers an amount of insulin appropriate for the patient based on the amount of carbohydrates measured by the measuring unit and the patient's blood glucose level obtained from the continuous blood glucose meter. The amount of insulin is calculated based on, for example, body weight, meal contents, blood glucose level, etc. The insulin pump system increases the amount of insulin when the amount of carbohydrates is high, and decreases the amount of insulin when the amount of carbohydrates is low. The insulin pump system can also adjust the amount of insulin based on the blood glucose level obtained from the continuous blood glucose meter. Furthermore, the insulin pump system can also calculate the amount of insulin based on the body weight and meal contents of the patient. For example, the insulin pump system calculates the amount of insulin using an algorithm that inputs the amount of carbohydrates and the blood glucose level and outputs the amount of insulin. As a result, the automatic insulin administration system according to the embodiment can automatically calculate and administer an appropriate amount of insulin based on the patient's meal information.

[0055] The reception unit receives image information showing an image of the patient's meal. The image information includes, for example, a photograph, a video, a scanned image, and the like, but is not limited to such examples. The reception unit receives, for example, a photograph of the meal taken with a smartphone. The reception unit can also receive a video of the meal. Furthermore, the reception unit can also receive a scanned image of the meal. The reception unit can provide a dedicated application or a web interface to receive these image information. For example, the patient can take a photo of the meal using an app on the smartphone and send the photo to the reception unit through the app. Similarly, a video of the meal can be taken using an app and sent to the reception unit. As for the scanned image, for example, an image of the meal can be scanned using a home scanner or a scanning function of a smartphone and the image can be sent to the reception unit. The reception unit can centrally manage the received image information and store it in a database for sending to the measurement unit. This allows the reception unit to efficiently receive and manage various forms of image information from the patient. Furthermore, the reception unit can have a function of checking the quality of the received image information and filtering out unclear or inappropriate images. For example, the image resolution and brightness can be automatically evaluated, and if an image does not meet the criteria, a message can be sent to the patient urging them to resubmit it. This allows the reception unit to provide high-quality image information for the measurement unit to perform accurate analysis.

[0056] The measurement unit uses the generation AI to analyze the image information accepted by the reception unit and measure the amount of carbohydrates. The generation AI analyzes the image information using technologies such as deep learning and neural networks. Specifically, the generation AI uses a deep learning model to measure the amount of carbohydrates from the image of the meal. The deep learning model is pre-trained using a large dataset of meal images and their corresponding carbohydrate amounts, and can estimate the amount of carbohydrates with high accuracy even for new images. For example, the generation AI takes an image of a meal as input, identifies the type and amount of each food in the image, and calculates the amount of carbohydrates contained in each food. The neural network can have a layer for extracting image features and classifying the type of food, and a layer for estimating the amount of carbohydrates. In addition, the generation AI can also execute an algorithm for measuring the amount of carbohydrates based on the image information. For example, the generation AI measures the amount of carbohydrates using a deep learning model that takes an image of a meal as input and outputs the amount of carbohydrates. The measurement unit stores the amount of carbohydrates measured by the generation AI in a database and transmits it to the insulin pump system. This allows the measurement unit to quickly and accurately analyze the accepted image information and measure the amount of carbohydrates. Furthermore, the measurement unit can be equipped with a function to verify the analysis results of the generation AI and manually correct them if necessary. For example, if the generation AI misidentifies a particular food, an operator of the measurement unit can manually correct it and measure the accurate carbohydrate amount. This allows the measurement unit to combine the high analytical capabilities of the generation AI with human judgment to achieve more accurate carbohydrate amount measurements.

[0057] The insulin pump system calculates and administers an amount of insulin appropriate for the patient based on the amount of carbohydrates measured by the measuring unit and the patient's blood glucose level obtained from the continuous blood glucose monitor. The amount of insulin is calculated based on, for example, body weight, meal contents, blood glucose level, etc. The insulin pump system increases the amount of insulin when the amount of carbohydrates is high, and decreases the amount of insulin when the amount of carbohydrates is low. The insulin pump system can also adjust the amount of insulin based on the blood glucose level obtained from the continuous blood glucose monitor. Furthermore, the insulin pump system can calculate the amount of insulin based on the body weight and meal contents of the patient. For example, the insulin pump system calculates the amount of insulin using an algorithm that inputs the amount of carbohydrates and the blood glucose level and outputs the amount of insulin. The insulin pump system includes a mechanism for automatically administering the calculated amount of insulin to the patient. Specifically, the insulin pump system administers insulin through a cannula inserted subcutaneously. The cannula is fixed to the patient's skin and administers insulin at an appropriate time based on instructions from the insulin pump system. The insulin pump system can monitor the patient's blood glucose level in real time and adjust the amount of insulin as necessary. For example, if the blood glucose level rises suddenly, the insulin pump system stabilizes the blood glucose level by administering additional insulin. In addition, if blood glucose levels drop, insulin administration can be temporarily stopped. This allows the insulin pump system to keep the patient's blood glucose levels in an optimal range at all times. Furthermore, the insulin pump system can customize the insulin administration schedule according to the patient's lifestyle and activity level. For example, it can reduce the amount of insulin on days when the patient exercises a lot and increase the amount of insulin on days when the patient eats a lot. This allows the insulin pump system to provide optimal insulin administration according to the individual needs of the patient.

[0058] The automatic insulin administration system includes an acquisition unit that acquires information on the patient's exercise schedule. The acquisition unit acquires information on the patient's exercise schedule. The exercise schedule includes, for example, the type of exercise, time, and frequency, but is not limited to such examples. The acquisition unit acquires, for example, information on the exercise schedule input by the patient in advance. The acquisition unit can also acquire information on the exercise schedule input by the patient using a smartphone or a wearable device. Furthermore, the acquisition unit can also be equipped with a sensor for automatically acquiring the patient's exercise schedule. For example, the acquisition unit acquires information on the exercise schedule from a smartphone or a wearable device when the patient performs exercise such as jogging or walking. This makes it possible to adjust insulin administration based on the patient's exercise schedule. A part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input information on the patient's exercise schedule to AI and cause AI to perform analysis of the exercise schedule.

[0059] The automatic insulin administration system includes a prediction unit that predicts fluctuations in the patient's blood glucose level based on the information of the exercise schedule acquired by the acquisition unit. The prediction unit predicts fluctuations in the patient's blood glucose level based on the information of the exercise schedule acquired by the acquisition unit. The fluctuations in the blood glucose level are predicted based on, for example, past data, exercise intensity, meal contents, and the like. The prediction unit predicts fluctuations in the blood glucose level based on, for example, past data. The prediction unit can also predict fluctuations in the blood glucose level based on exercise intensity. Furthermore, the prediction unit can predict fluctuations in the blood glucose level based on meal contents. For example, the prediction unit predicts fluctuations in the blood glucose level using an algorithm that inputs past data and outputs fluctuations in the blood glucose level. This makes it possible to predict hyperglycemia or hypoglycemia based on the exercise schedule. A part or all of the above-mentioned processing in the prediction unit may be performed using, for example, AI, or may be performed without using AI. For example, the prediction unit can input information about the exercise schedule to AI and cause AI to execute prediction of fluctuations in the blood glucose level.

[0060] The automatic insulin administration system includes a notification unit that notifies the patient of the prediction result by the prediction unit. The notification unit notifies the patient of the prediction result by the prediction unit. The prediction result includes, for example, a numerical value, a graph, an alert, and the like, but is not limited to such examples. The notification unit notifies the patient of the prediction result, for example, as a numerical value. The notification unit can also notify the patient of the prediction result as a graph. Furthermore, the notification unit can also notify the patient of the prediction result as an alert. For example, the notification unit displays the prediction result as a numerical value and notifies the patient. In this way, by notifying the patient of the prediction result, measures can be taken in advance. A part or all of the above-mentioned processing in the notification unit may be performed, for example, using AI, or may be performed without using AI. For example, the notification unit can input the prediction result to AI and cause the AI ​​to execute the timing and method of notification.

[0061] The reception unit estimates the patient's emotion and adjusts the timing of receiving the image information based on the estimated emotion of the patient. For example, when the patient is stressed, the reception unit receives the image information during a time period when the patient can relax. Furthermore, when the patient is relaxed, the reception unit can also receive the image information immediately after a meal. Furthermore, when the patient is in a hurry, the reception unit can also receive the image information before a meal. For example, the reception unit estimates the patient's emotion and adjusts the timing of receiving the image information based on the estimated emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. Thereby, by adjusting the timing of receiving the image information according to the emotion of the patient, it is possible to receive the image information at a more appropriate timing. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the emotion data of the patient to the generation AI and cause the generation AI to execute the emotion estimation.

[0062] The reception unit analyzes the patient's past meal history and selects the optimal image reception method. The reception unit, for example, proposes optimal shooting angles and lighting conditions based on images of meals taken by the patient in the past. The reception unit can also detect a specific meal pattern from the patient's past meal history and adjust the image reception method based on the pattern. The reception unit can also analyze the quality of images taken by the patient in the past and propose optimal camera settings. For example, the reception unit analyzes images of meals taken by the patient in the past and proposes optimal shooting angles and lighting conditions. This allows for more accurate measurement of carbohydrate amounts by selecting the optimal image reception method based on the past meal history. Part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input past meal history data to AI and cause AI to select the optimal image reception method.

[0063] The reception unit performs filtering based on the patient's current health condition and the type of meal when receiving image information. For example, when the patient's current blood glucose level is high, the reception unit preferentially receives images of meals with a high carbohydrate content. The reception unit can also filter images of a specific meal category (e.g., low-carbohydrate meals) based on the patient's current health condition. Furthermore, the reception unit can filter images of meals containing specific ingredients based on the patient's allergy information. For example, when the patient's current blood glucose level is high, the reception unit preferentially receives images of meals with a high carbohydrate content. This makes it possible to receive more appropriate image information by filtering based on the patient's health condition and the type of meal. Part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the patient's health condition data to AI and leave the execution of filtering to AI.

[0064] The measurement unit estimates the patient's emotion and adjusts the measurement method of the carbohydrate amount based on the estimated emotion of the patient. For example, when the patient is stressed, the measurement unit provides a simplified measurement method. In addition, when the patient is relaxed, the measurement unit can also provide a detailed measurement method. Furthermore, when the patient is in a hurry, the measurement unit can also provide a method that allows quick measurement. For example, the measurement unit estimates the patient's emotion and adjusts the measurement method of the carbohydrate amount based on the estimated emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. This allows more appropriate measurement by adjusting the measurement method of the carbohydrate amount according to the patient's emotion. A part or all of the above-mentioned processing in the measurement unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the measurement unit can input the patient's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0065] The measurement unit adjusts the level of detail of the measurement based on the type and amount of the meal during measurement. For example, in the case of a high-carbohydrate meal, the measurement unit performs detailed measurement. In addition, in the case of a low-carbohydrate meal, the measurement unit can also perform simplified measurement. Furthermore, in the case of a large amount of meal, the measurement unit can perform detailed measurement. For example, the measurement unit adjusts the level of detail of the measurement based on the type and amount of meal. This allows for more accurate measurement of the amount of carbohydrates by adjusting the level of detail of the measurement based on the type and amount of meal. Part or all of the above-mentioned processing in the measurement unit may be performed, for example, using AI, or may be performed without using AI. For example, the measurement unit can input data on the type and amount of meal to AI and cause AI to adjust the level of detail of the measurement.

[0066] The measurement unit applies different measurement algorithms depending on the category of the meal when measuring. For example, in the case of bread, the measurement unit applies a specific algorithm to measure. In addition, in the case of pasta, the measurement unit can also apply a different algorithm to measure. Furthermore, in the case of desserts, the measurement unit can also apply yet another algorithm to measure. For example, the measurement unit applies different measurement algorithms depending on the category of the meal. This makes it possible to measure the amount of carbohydrates more accurately by applying different measurement algorithms depending on the category of the meal. Some or all of the above-mentioned processing in the measurement unit may be performed, for example, using AI, or may be performed without using AI. For example, the measurement unit can input meal category data to AI and cause AI to apply the measurement algorithm.

[0067] The insulin pump system estimates the patient's emotion and adjusts the timing of insulin administration based on the estimated emotion of the patient. For example, when the patient is stressed, the insulin pump system administers insulin during a time period when the patient can relax. Also, when the patient is relaxed, the insulin pump system can also administer insulin immediately after a meal. Furthermore, when the patient is in a hurry, the insulin pump system can also administer insulin before a meal. For example, the insulin pump system estimates the patient's emotion and adjusts the timing of insulin administration based on the estimated emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. This allows more appropriate insulin administration by adjusting the timing of insulin administration according to the patient's emotion. A part or all of the above-mentioned processing in the insulin pump system may be performed using, for example, an AI, or may be performed without using an AI. For example, the insulin pump system can input the patient's emotion data into the generation AI and cause the generation AI to execute emotion estimation.

[0068] When administering insulin, the insulin pump system calculates the optimal dosage by referring to the patient's past blood glucose data. The insulin pump system calculates the optimal insulin dosage based on, for example, the patient's blood glucose data from the past week. The insulin pump system can also adjust the insulin dosage in consideration of long-term trends based on the patient's blood glucose data from the past month. Furthermore, the insulin pump system can calculate the optimal insulin dosage by combining the patient's past meal pattern and blood glucose data. For example, the insulin pump system calculates the optimal insulin dosage based on the patient's blood glucose data from the past week. This allows for more accurate insulin administration by calculating the optimal insulin dosage based on the past blood glucose data. Some or all of the above-mentioned processing in the insulin pump system may be performed, for example, using AI, or may be performed without using AI. For example, the insulin pump system can input past blood glucose data into AI and have AI execute calculation of the optimal dosage.

[0069] The insulin pump system customizes the administration method based on the patient's current health condition when administering insulin. For example, when the patient's current blood glucose level is high, the insulin pump system selects a method of administering insulin quickly. In addition, when the patient's current health condition is stable, the insulin pump system can also apply a normal insulin administration method. Furthermore, when the patient is in poor health, the insulin pump system can select a method of reducing the administration amount or temporarily suspending administration. For example, when the patient's current blood glucose level is high, the insulin pump system selects a method of administering insulin quickly. This allows for more appropriate insulin administration by customizing the administration method based on the current health condition. Some or all of the above-mentioned processing in the insulin pump system may be performed, for example, using AI, or may be performed without using AI. For example, the insulin pump system can input current health condition data into AI and have AI execute customization of the administration method.

[0070] The acquisition unit estimates the patient's emotion and adjusts the timing of acquiring the exercise schedule based on the estimated emotion of the patient. For example, when the patient is feeling stressed, the acquisition unit acquires the exercise schedule during a time period when the patient can relax. Furthermore, when the patient is relaxed, the acquisition unit can also acquire the exercise schedule immediately before exercise. Furthermore, when the patient is in a hurry, the acquisition unit can also acquire the exercise schedule after exercise. For example, the acquisition unit estimates the patient's emotion and adjusts the timing of acquiring the exercise schedule based on the estimated emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. As a result, by adjusting the timing of acquiring the exercise schedule according to the emotion of the patient, it becomes possible to acquire a more appropriate exercise schedule. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the emotion data of the patient to the generation AI and cause the generation AI to execute the emotion estimation.

[0071] The acquisition unit analyzes the patient's past exercise history and selects the optimal acquisition method. The acquisition unit proposes an optimal exercise schedule based on, for example, the type and frequency of exercise performed by the patient in the past. The acquisition unit can also detect a specific exercise pattern from the patient's past exercise history and adjust the exercise schedule based on the pattern. The acquisition unit can also analyze the effect of the patient's past exercise and propose an optimal exercise schedule. For example, the acquisition unit proposes an optimal exercise schedule based on the type and frequency of exercise performed by the patient in the past. This makes it possible to acquire a more accurate exercise schedule by selecting an optimal acquisition method based on the past exercise history. Part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, AI, or may be performed without using AI. For example, the acquisition unit can input past exercise history data to AI and cause AI to select an optimal acquisition method.

[0072] The prediction unit estimates the patient's emotion and adjusts the method of predicting hyperglycemia or hypoglycemia based on the estimated emotion of the patient. For example, when the patient is stressed, the prediction unit predicts hyperglycemia or hypoglycemia taking into account the influence of stress. In addition, when the patient is relaxed, the prediction unit can also apply a normal prediction method. Furthermore, when the patient is in a hurry, the prediction unit can also apply a method of making a quick prediction. For example, the prediction unit estimates the patient's emotion and adjusts the method of predicting hyperglycemia or hypoglycemia based on the estimated emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. As a result, by adjusting the prediction method according to the patient's emotion, more accurate prediction of hyperglycemia or hypoglycemia is possible. A part or all of the above-mentioned processing in the prediction unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the prediction unit can input the patient's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0073] The prediction unit optimizes the prediction algorithm by referring to the patient's past blood glucose data at the time of prediction. The prediction unit optimizes the prediction algorithm based on, for example, the patient's blood glucose data from the past week. The prediction unit can also adjust the prediction algorithm in consideration of long-term trends based on the patient's blood glucose data from the past month. Furthermore, the prediction unit can optimize the prediction algorithm by combining the patient's past meal patterns and blood glucose data. For example, the prediction unit optimizes the prediction algorithm based on the patient's blood glucose data from the past week. As a result, by optimizing the prediction algorithm based on the past blood glucose data, more accurate prediction of hyperglycemia or hypoglycemia is possible. A part or all of the above-mentioned processing in the prediction unit may be performed, for example, using AI, or may be performed without using AI. For example, the prediction unit can input past blood glucose data to AI and cause AI to execute optimization of the prediction algorithm.

[0074] The notification unit estimates the patient's emotion and adjusts the timing of the notification based on the estimated emotion of the patient. For example, if the patient is feeling stressed, the notification unit issues a notification during a time period when the patient can relax. Also, if the patient is relaxed, the notification unit can also issue a notification immediately after a meal. Furthermore, if the patient is in a hurry, the notification unit can also issue a notification before a meal. For example, the notification unit estimates the patient's emotion and adjusts the timing of the notification based on the estimated emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. Thereby, by adjusting the timing of the notification according to the emotion of the patient, it is possible to issue a notification at a more appropriate timing. A part or all of the above-mentioned processing in the notification unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the notification unit can input the emotion data of the patient to the generation AI and cause the generation AI to execute the emotion estimation.

[0075] At the time of notification, the notification unit selects the optimal notification method by referring to the patient's past notification history. The notification unit, for example, selects the most effective notification method from the patient's past notification history. The notification unit can also analyze the patient's past notification history and select an effective notification method for a specific time period. Furthermore, the notification unit can determine the optimal notification frequency based on the patient's past notification history. For example, the notification unit selects the most effective notification method from the patient's past notification history. As a result, more effective notification is possible by selecting the optimal notification method based on the past notification history. A part or all of the above-mentioned processing in the notification unit may be performed, for example, using AI, or may be performed without using AI. For example, the notification unit can input past notification history data to AI and cause AI to select the optimal notification method.

[0076] The notification unit estimates the patient's emotion and determines the priority of notifications based on the estimated emotion of the patient. For example, when the patient is feeling stressed, the notification unit prioritizes notifications of content that can relax the patient. In addition, when the patient is relaxed, the notification unit can also prioritize notifications including detailed information. Furthermore, when the patient is in a hurry, the notification unit can also prioritize notifications that focus on the main points. For example, the notification unit estimates the patient's emotion and determines the priority of notifications based on the estimated emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. This enables more appropriate notification by determining the priority of notifications according to the patient's emotion. A part or all of the above-mentioned processing in the notification unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the notification unit can input the patient's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0077] The notification unit selects the optimal notification method in consideration of the device information of the patient at the time of notification. For example, when the patient is using a smartphone, the notification unit performs a push notification. In addition, when the patient is using a tablet, the notification unit can also provide a notification method optimized for a large screen. Furthermore, when the patient is using a smartwatch, the notification unit can also provide a simple and highly visible notification method. For example, when the patient is using a smartphone, the notification unit performs a push notification. This enables more effective notification by selecting the optimal notification method in consideration of the device information of the patient. Part or all of the above-mentioned processing in the notification unit may be performed, for example, using AI, or may be performed without using AI. For example, the notification unit can input device information to AI and cause AI to select the optimal notification method.

[0078] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0079] The automatic insulin administration system may further include a sleep monitoring unit that monitors the patient's sleep patterns. The sleep monitoring unit records the quality and duration of the patient's sleep and adjusts the insulin dosage based on the quality and duration of the patient's sleep. For example, if the patient is not getting enough sleep, the insulin dosage may be increased since insulin sensitivity may be reduced. Conversely, if the patient is getting enough sleep, the insulin dosage may be decreased. Furthermore, the sleep monitoring unit may analyze the patient's sleep patterns over the long term and provide advice to improve the quality of sleep. This may improve the patient's overall health.

[0080] The automatic insulin administration system may further include a stress monitoring unit that monitors the patient's stress level. The stress monitoring unit measures biosignals such as the patient's heart rate and electrodermal activity to evaluate the stress level. For example, if the patient is in a high stress state, insulin sensitivity may be reduced, so the insulin administration amount may be adjusted. The stress monitoring unit may also suggest relaxation advice and stress reduction exercises to the patient. This allows the patient's stress level to be managed and the effect of insulin administration to be optimized.

[0081] The automatic insulin administration system may further include a meal recorder that records the patient's meal contents in detail. The meal recorder records the food ingredients and their amounts ingested by the patient in detail, and adjusts the insulin dosage based on this. For example, if the patient eats a high-fat meal, the insulin dosage can be increased because insulin sensitivity may be reduced. The meal recorder can also suggest a balanced meal to the patient and support the improvement of the meal contents. This can strengthen the patient's dietary management and optimize the effect of insulin administration.

[0082] The automatic insulin administration system may further include an exercise performance monitoring unit that monitors the exercise performance of the patient. The exercise performance monitoring unit records the exercise intensity and duration of the patient, and adjusts the insulin dosage based on the recorded exercise intensity and duration. For example, if the patient performs high-intensity exercise, the insulin dosage can be reduced because insulin sensitivity may be improved. The exercise performance monitoring unit may also suggest an appropriate exercise plan to the patient to maximize the effect of exercise. This can strengthen the patient's exercise management and optimize the effect of insulin administration.

[0083] The automated insulin administration system may further include a social activity monitoring unit that monitors the patient's social activities. The social activity monitoring unit records the frequency of the patient's social events and outings, and adjusts the insulin administration amount based on the record. For example, if the patient frequently goes out, the timing and content of meals may change, so the insulin administration amount can be adjusted. The social activity monitoring unit may also suggest appropriate social activities to the patient to support social health. This allows the patient's social activities to be managed and the effect of insulin administration to be optimized.

[0084] The automatic insulin administration system may further include an acquisition unit that estimates the patient's emotions and adjusts the timing of acquiring the exercise schedule based on the estimated emotions. For example, if the patient is feeling stressed, the acquisition unit acquires the exercise schedule during a time period when the patient can relax. Also, if the patient is relaxed, the acquisition unit may acquire the exercise schedule immediately before exercise. Furthermore, if the patient is in a hurry, the acquisition unit may acquire the exercise schedule after exercise. This makes it possible to acquire a more appropriate exercise schedule by adjusting the timing of acquiring the exercise schedule according to the patient's emotions.

[0085] The automatic insulin administration system may further include a prediction unit that estimates the patient's emotions and adjusts a method of predicting hyperglycemia or hypoglycemia based on the estimated emotions. For example, when the patient is stressed, the prediction unit predicts hyperglycemia or hypoglycemia taking into account the influence of stress. The prediction unit may also apply a normal prediction method when the patient is relaxed. Furthermore, the prediction unit may also apply a method for making a quick prediction when the patient is in a hurry. This allows for more accurate prediction of hyperglycemia or hypoglycemia by adjusting the prediction method according to the patient's emotions.

[0086] The automatic insulin administration system may further include a notification unit that estimates the patient's emotions and adjusts the timing of notification based on the estimated emotions. For example, if the patient is feeling stressed, the notification unit may notify the patient during a time period when the patient is able to relax. Also, if the patient is relaxed, the notification unit may notify the patient immediately after a meal. Furthermore, if the patient is in a hurry, the notification unit may notify the patient before a meal. In this way, the timing of notification may be adjusted according to the patient's emotions, making it possible to notify at a more appropriate time.

[0087] The automatic insulin administration system may further include a notification unit that estimates the patient's emotions and determines the priority of notifications based on the estimated emotions. For example, if the patient is feeling stressed, the notification unit prioritizes notifications that will relax the patient. Also, if the patient is relaxed, the notification unit may prioritize notifications that include detailed information. Furthermore, if the patient is in a hurry, the notification unit may prioritize notifications that focus on the main points. This allows for more appropriate notifications by determining the priority of notifications according to the patient's emotions.

[0088] The automatic insulin administration system may further include an insulin pump system that estimates the patient's emotions and adjusts the timing of insulin administration based on the estimated emotions. For example, if the patient is stressed, the insulin pump system administers insulin at a time when the patient is relaxed. Also, if the patient is relaxed, the insulin pump system can administer insulin immediately after a meal. Furthermore, if the patient is in a hurry, the insulin pump system can administer insulin before a meal. This allows for more appropriate insulin administration by adjusting the timing of insulin administration according to the patient's emotions.

[0089] The process flow of the second embodiment will be briefly described below.

[0090] Step 1: The reception unit receives image information showing an image of the patient's meal. The image information includes, for example, a photo, a video, a scanned image, etc. The reception unit can receive a photo of the meal taken with a smartphone, a video of the meal, or a scanned image of the meal. Step 2: The measurement unit uses the generation AI to analyze the image information received by the reception unit and measure the amount of carbohydrates. The generation AI analyzes the image information using technologies such as deep learning and neural networks, and measures the amount of carbohydrates from the image of the meal. Step 3: The insulin pump system calculates and administers the appropriate amount of insulin for the patient based on the amount of carbohydrates measured by the measuring unit and the patient's blood glucose level obtained from the continuous glucose monitor. The amount of insulin is calculated based on weight, meal contents, blood glucose level, etc. For example, if the amount of carbohydrates is high, the amount of insulin is increased, and if the amount of carbohydrates is low, the amount of insulin is decreased. The amount of insulin can also be adjusted based on the blood glucose level obtained from the continuous glucose monitor.

[0091] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0092] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0093] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0094] For example, the reception unit is realized by the reception device 38 of the smart device 14. For example, the measurement unit is realized by the specific processing unit 290 of the data processing device 12. For example, the insulin pump system is realized by the specific processing unit 290 of the data processing device 12. For example, the acquisition unit is realized by the camera 42 and the communication I / F 44 of the smart device 14. For example, the prediction unit is realized by the specific processing unit 290 of the data processing device 12. For example, the notification unit is realized by the output device 40 of the smart device 14. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0095] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0096] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0097] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0098] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0099] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0100] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0101] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0102] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0103] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0104] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0105] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0106] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0107] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0108] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0109] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0110] For example, the reception unit is realized by the microphone 238 of the smart glasses 214. For example, the measurement unit is realized by the specific processing unit 290 of the data processing device 12. For example, the insulin pump system is realized by the specific processing unit 290 of the data processing device 12. For example, the acquisition unit is realized by the camera 42 and the communication I / F 44 of the smart glasses 214. For example, the prediction unit is realized by the specific processing unit 290 of the data processing device 12. For example, the notification unit is realized by the speaker 240 of the smart glasses 214. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0111] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0112] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0113] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0114] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0115] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0116] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0117] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0118] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0119] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0120] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0121] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0122] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0123] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0124] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0125] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0126] For example, the reception unit is realized by the microphone 238 of the headset type terminal 314. For example, the measurement unit is realized by the specific processing unit 290 of the data processing device 12. For example, the insulin pump system is realized by the specific processing unit 290 of the data processing device 12. For example, the acquisition unit is realized by the camera 42 and the communication I / F 44 of the headset type terminal 314. For example, the prediction unit is realized by the specific processing unit 290 of the data processing device 12. For example, the notification unit is realized by the display 343 of the headset type terminal 314. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0127] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0128] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0129] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0130] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0131] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0132] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0133] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0134] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0135] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0136] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0137] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0138] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0139] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0140] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0141] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0142] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0143] For example, the reception unit is realized by the microphone 238 of the robot 414. For example, the measurement unit is realized by the specific processing unit 290 of the data processing device 12. For example, the insulin pump system is realized by the specific processing unit 290 of the data processing device 12. For example, the acquisition unit is realized by the camera 42 and the communication I / F 44 of the robot 414. For example, the prediction unit is realized by the specific processing unit 290 of the data processing device 12. For example, the notification unit is realized by the speaker 240 of the robot 414. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0144] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0145] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0146] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0147] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0148] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0149] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0150] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0151] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0152] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0153] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0154] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0155] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0156] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0157] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0158] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0159] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0160] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0161] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0162] (Appendix 1) A reception unit that receives image information showing an image of a patient's meal; a measuring unit that analyzes the image information received by the receiving unit using a generation AI and measures the amount of carbohydrates; and a system for calculating and administering an amount of insulin appropriate for the patient based on the amount of carbohydrates measured by the measuring unit and the blood glucose level of the patient obtained from a continuous blood glucose monitor. A system characterized by: (Appendix 2) an acquisition unit for acquiring information on the patient's exercise schedule; 2. The system of claim 1, (Appendix 3) a prediction unit that predicts a change in the blood glucose level of the patient based on the information of the exercise schedule acquired by the acquisition unit; 3. The system of claim 2, (Appendix 4) a notification unit that notifies the patient of the prediction result by the prediction unit; 4. The system of claim 3. (Appendix 5) The reception unit is A system that estimates a patient's emotions and adjusts the timing of receiving image information based on the estimated patient's emotions. (Appendix 6) The reception unit is Analyze the patient's past dietary history and select the optimal image reception method 2. The system of claim 1, (Appendix 7) The reception unit is Filtering incoming image information based on the patient's current health status and dietary habits 2. The system of claim 1, (Appendix 8) The measurement unit is Inferring a patient's emotions and adjusting the method of measuring carbohydrate amounts based on the estimated patient's emotions 2. The system of claim 1, (Appendix 9) The measurement unit is When measuring, adjust the measurement detail based on the type and amount of food 2. The system of claim 1, (Appendix 10) The measurement unit is When measuring, different measurement algorithms are applied depending on the food category. 2. The system of claim 1, (Appendix 11) The insulin pump system comprises: Inferring a patient's emotions and adjusting the timing of insulin administration based on the inferred emotions 2. The system of claim 1, (Appendix 12) The insulin pump system comprises: When administering insulin, the optimal dose is calculated by looking at the patient's past blood glucose data. 2. The system of claim 1, (Appendix 13) The insulin pump system comprises: Customize insulin delivery based on the patient's current health status 2. The system of claim 1, (Appendix 14) The acquisition unit is To estimate a patient's emotion and adjust the timing of acquiring an exercise schedule based on the estimated emotion of the patient. 3. The system of claim 2, (Appendix 15) The acquisition unit is Analyze the patient's past exercise history and select the most appropriate acquisition method 3. The system of claim 2, (Appendix 16) The prediction unit is To estimate a patient's emotion and adjust a method for predicting hyperglycemia or hypoglycemia based on the estimated emotion of the patient. 4. The system of claim 3. (Appendix 17) The prediction unit is When making predictions, the prediction algorithm is optimized by referencing the patient's past blood glucose data. 4. The system of claim 3. (Appendix 18) The notification unit is Estimate the patient's emotions and adjust the timing of notifications based on the estimated patient emotions 5. The system of claim 4. (Appendix 19) The notification unit is When notifying patients, the system selects the optimal notification method by referring to the patient's past notification history. 5. The system of claim 4. (Appendix 20) The notification unit is Estimate the patient's emotions and prioritize notifications based on the estimated patient emotions 5. The system of claim 4. (Appendix 21) The notification unit is When notifying, the optimal notification method is selected taking into account the patient's device information 5. The system of claim 4. [Explanation of symbols]

[0163] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A reception unit that receives image information showing an image of a patient's meal; A measuring unit that analyzes the image information received by the receiving unit using a generating AI and measures the amount of carbohydrates; and a system for calculating and administering an amount of insulin appropriate for the patient based on the amount of carbohydrates measured by the measuring unit and the blood glucose level of the patient obtained from a continuous blood glucose monitor. A system characterized by:

2. an acquisition unit for acquiring information on the patient's exercise schedule; 2. The system of claim 1 .

3. a prediction unit that predicts a change in the blood glucose level of the patient based on the information of the exercise schedule acquired by the acquisition unit; 3. The system of claim 2.

4. a notification unit that notifies the patient of the prediction result by the prediction unit; 4. The system of claim 3.

5. The reception unit is The patient's emotion is estimated, and the timing of receiving the image information is adjusted based on the estimated emotion of the patient.

4. The system of claim 3.

6. The reception unit is Analyze the patient's past dietary history and select the optimal image reception method.

2. The system of claim 1 .

7. The reception unit is When receiving the image information, filtering is performed based on the patient's current health condition and dietary type.

2. The system of claim 1 .

8. The measurement unit is Estimating the patient's emotion and adjusting the carbohydrate amount measurement method based on the estimated patient's emotion.

2. The system of claim 1 .

9. The measurement unit is When measuring, adjust the measurement detail based on the type and amount of food 2. The system of claim 1 .

10. The measurement unit is When measuring, different measurement algorithms are applied depending on the food category.

2. The system of claim 1 .

Images