Full-closed-loop artificial pancreas control method and system integrating carbon-water quantification, terminal equipment and storage medium

Abstract

The invention discloses a full-closed-loop artificial pancreas control method and system integrating carbon and water quantification, terminal equipment and a storage medium. The method comprises the following steps: firstly, acquiring an image frame containing meals, and preprocessing the image frame; selecting a deep learning instance segmentation model to carry out segmentation processing on the image frame, and obtaining a segmentation mask and segmentation confidence of each meal; based on the segmentation processing result, three-dimensional reconstruction is carried out in combination with image geometric information, and meal volume information is obtained; calculating the total carbon-water content according to the unit carbon-water content of different meals, obtaining an estimated value of the total carbohydrate intake, and calculating and outputting an uncertainty index of the estimated value; the artificial pancreas system dynamically adjusts an infusion strategy based on the indexes and allows a user to confirm; and performing subsequent model optimization based on a multi-layer closed-loop feedback mechanism, setting a calibration factor and performing iterative optimization, and calibrating the total carbohydrate intake to the accuracy meeting the requirement.

Description

Technical Field

[0001] This invention belongs to the field of artificial pancreas technology, and specifically relates to a fully closed-loop artificial pancreas control method, system, terminal equipment and storage medium with integrated carbohydrate quantification. Background Technology

[0002] The fully closed-loop artificial pancreas system (APS) is a revolutionary technology in diabetes management, designed to automatically regulate blood glucose levels by integrating continuous glucose monitoring (CGM), an insulin pump, and intelligent control algorithms to mimic the physiological functions of a healthy pancreas. While existing APS systems have made significant progress in basal insulin infusion and can effectively manage daily blood glucose fluctuations, their ability to handle the sharp rise in postprandial blood glucose caused by eating—the "postprandial blood glucose spike"—remains significantly limited.

[0003] Current mainstream APS systems generally rely on users manually inputting the amount of carbohydrates they intend to consume before eating. The system then calculates and administers the corresponding large dose of insulin. This "manual estimation + manual input" mode suffers from problems in practical applications, including difficulty in user estimation, low accuracy, and inability to respond to unexpected food intake. Some image recognition-based food identification applications have emerged in existing technologies. However, most of these technologies can only achieve rough identification of food types, generally lacking the ability to accurately quantify the physical weight / volume of food, and even less able to accurately correlate and calculate visual information with detailed nutritional databases (especially carbohydrate content). Furthermore, most existing methods rely on single two-dimensional image features or single three-dimensional reconstruction strategies (such as using only reference objects or only depth estimation), failing to adaptively select or integrate multiple reconstruction schemes according to the scene. This results in inherent and significant biases in volume estimation for irregularly shaped foods, scenes without reference objects, or mixed dishes. Therefore, the output results cannot meet the stringent standards required for a fully closed-loop APS as a medical-grade device in terms of accuracy, reliability, and real-time performance, making it difficult to directly integrate for automated insulin dosing decisions. Developing a new technology that can automatically, accurately, and safely estimate the carbohydrate content of food and seamlessly integrate with APS has become a key challenge in promoting the development of refined diabetes management. Summary of the Invention

[0004] Purpose of the invention: To address the problems existing in the above-mentioned background technology, the present invention provides a fully closed-loop artificial pancreas system and control method with integrated carbohydrate quantification. Based on deep learning computer vision technology, it realizes end-to-end automatic calculation from food images to carbohydrate content, and seamlessly integrates the results into the APS.

[0005] Technical solution: A fully closed-loop artificial pancreas control method integrating carbohydrate quantification, comprising the following steps: Step S1: Acquire image frames containing food using an image acquisition device and perform preprocessing; Step S2: Select a deep learning instance segmentation model and train it to segment the image frame; identify and segment the types of food contained in the image, and output the segmentation mask and segmentation confidence of each type of food. Step S3: Based on the segmentation processing results, perform three-dimensional reconstruction by combining image geometric information to obtain food volume information; Step S4: Convert the food volume information into weight information, and calculate the total carbohydrate content based on the unit carbohydrate content of different types of food to obtain the estimated total carbohydrate intake Total_Carbs_estimated; at the same time, calculate and output the uncertainty index U_total of the estimated value. Step S5: The artificial pancreas system dynamically adjusts the insulin infusion strategy based on Total_Carbs_estimated and U_total; Step S6: The artificial pancreas system displays the type, weight, Carbs_i, and Total_Carbs_estimated of each meal instance to the user, and provides the infusion strategy according to step S5 for the user to confirm; the user's modification behavior will be recorded by the system and used as high-precision, high-weight label data, which will be given priority for subsequent model optimization. Step S7: Optimize the model based on the multi-layer closed-loop feedback mechanism, set the calibration factor and iterate to calibrate Total_Carbs_estimated to the required accuracy.

[0006] Furthermore, in step S1, the image acquisition device includes a smartphone, smartwatch, or smart glasses; the acquired image results include a single color image, multi-angle photos, and video streams; brightness equalization, color correction, and distortion correction are performed on the extracted multi-frame images; when acquiring multi-angle photos or video streams, the best image frame or multi-frame fusion analysis is selected.

[0007] Furthermore, the three-dimensional reconstruction strategy in step S3 includes: (1) When a standard reference is preset in step S1, the physical size of the food is calculated based on the image ratio relationship using the standard reference. (2) When a single color image is acquired in step S1 and there is no standard reference, a monocular depth estimation model is used to predict the depth map and restore the three-dimensional volume of the food. (3) When collecting multi-angle images or video streams in step S1, obtain accurate food volume information by generating a 3D point cloud model.

[0008] Furthermore, the method for calculating the uncertainty index U_total specifically includes: Calculate the local uncertainty U_i for each meal instance i, where each U_i includes visual perception uncertainty U_vision_i and physical quantification uncertainty U_volume_i; U_vision_i includes identification uncertainty and segmentation uncertainty; where identification uncertainty U_ident_i=1-P_i, P_i represents the segmentation confidence obtained in step S2; segmentation uncertainty U_seg_i is the average entropy value of the segmentation mask boundary; U_vision_i is the weighted sum of U_ident_i and U_seg_i; The physical quantization uncertainty U_volume_i is set based on the 3D reconstruction strategy selected in step S3; when strategy (1) is adopted, U_volume_i is set as a function positively correlated with distance, that is, the farther the food is from the camera, the larger U_volume_i is; when strategy (2) is adopted, the uncertainty map output by the depth estimation model is directly used, and the average uncertainty in the food area is calculated as U_volume_i; when strategy (3) is adopted, U_volume_i is set based on the sparsity of the 3D point cloud; finally, the following is calculated: U_total=Σ((Carbs_i / Total_Carbs_estimated)*(α*U_vision_i+β*U_volume_i)) Where Carbs_i represents the carbohydrate contribution of meal instance i, and α and β are weighting coefficients; after normalizing U_total, the final range is [0, 1].

[0009] Furthermore, the dynamic adjustment of the insulin infusion strategy in step S5 specifically includes: When U_total≤t1, the uncertainty is low, and the full dose is infused based on Total_Carbs_estimated. When t1<U_total≤t2, the uncertainty is medium, and the infusion dose is calculated based on m*Total_Carbs_estimated, or the infusion interval window is extended, where m∈[0,1]. When U_total>t2, the uncertainty is high, and automatic infusion is stopped, and the user is immediately prompted to manually confirm or enter the infusion dose. Here, t1 and t2 are preset thresholds.

[0010] Furthermore, the subsequent model optimization method in step S7 includes: Step S7.1: Completely record the Total_Carbs_estimated for this meal, the insulin dose injected, and subsequent continuous blood glucose monitoring data; Step S7.2, Real-time alarm based on short-term feedback: Real-time monitoring of the initial rise slope of postprandial blood glucose. When the actual slope exceeds a certain range of the model slope predicted based on the infusion dose and estimated carbohydrate content, the system issues an early warning and, under strict safety rules, suggests or executes a small amount of corrective dose. Step S7.3: Model calibration based on long-term feedback: When the user meets the conditions of stable blood glucose before eating, no exercise after eating, and no drastic emotional changes, model calibration is initiated; the actual blood glucose curve and the predicted blood glucose curve under Total_Carbs_estimated are compared, the ratio of the area under the curve (AUC) is calculated to obtain the systematic bias, and a calibration factor u is set to calibrate Total_Carbs_estimated to the actual total carbohydrate intake; wherein the calibration factor u is set within the preset safety boundary range [a1,a2], and the calibrated Total_Carbs = Total_Carbs_estimated * u; Step S7.4: After multiple iterations, the system's estimation accuracy of Total_Carbs_estimated gradually matches the user's actual metabolic response and dietary habits, ultimately achieving highly personalized and precise management.

[0011] A fully closed-loop artificial pancreas system integrates an image processing module and a carbohydrate quantification module for performing any of the methods described above; the two modules exist in the form of software or hardware in a mobile computing device, a cloud server, or an APS main controller; the image processing module is used to perform the segmentation processing step in step S2, and the carbohydrate quantification module is used to perform steps S3-S4.

[0012] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the program, it implements the steps of any of the methods described above.

[0013] A computer-readable storage medium having a computer program stored thereon; said computer program, when executed by a processor, implements the steps of any of the methods described above.

[0014] Compared with existing technical solutions, the technical solution adopted in this invention has the following advantages: (1) This invention provides a high-precision, high-robust, and high-security adaptive image recognition food detection and carbohydrate quantification method, which improves the reliability of visual perception, provides a robust and accurate physical quantification method, and integrates the quantified carbohydrate and the corresponding uncertainty assessment into the APS for dynamic adjustment of the infusion strategy, thereby achieving a key balance between automation and safety.

[0015] (2) This invention constructs an intelligent carbohydrate quantification method with a complete closed-loop capability of "perception-decision-execution-learning". Its core is the introduction of a safe and continuous personalized optimization mechanism based on physiological signal feedback. This invention establishes a complete closed-loop process from perception and execution, detection and feedback to learning and optimization. By actively detecting postprandial blood glucose, the error is derived in reverse using blood glucose feedback data, and the core model is further calibrated.

[0016] (3) This invention combines long-term and short-term feedback to enable the system to cope with errors at different time scales, making it more sensitive and accurate. At the same time, it clearly sets the upper and lower limits of the optimization parameters to prevent the optimization process from "running away" due to a single data anomaly, thus ensuring user safety. Attached Figure Description

[0017] Figure 1 The schematic diagram shows the control method of the integrated carbon quantification fully closed-loop artificial pancreas system provided by the present invention. Detailed Implementation

[0018] This invention provides a fully closed-loop artificial pancreas control method integrating carbohydrate quantification. Addressing the problem in existing technologies that primarily rely on manual estimation of meal carbohydrate content and input into the APS (Advanced Persistent Spectrum) to adjust insulin infusion levels, which suffers from inaccurate estimations and leads to significant discrepancies between the insulin infusion strategy and the ideal state, this invention utilizes an improved deep learning model based on image recognition and uncertainty assessment to generate more reasonable carbohydrate quantification results and uncertainty weights. The APS control algorithm uses these results as key inputs to dynamically adjust the insulin infusion strategy. Furthermore, it provides corresponding model optimization methods to effectively improve model accuracy. The fully closed-loop artificial pancreas control method provided by this invention is described in detail below with reference to the accompanying drawings.

[0019] like Figure 1 As shown, the fully closed-loop artificial pancreas control method provided by the present invention includes the following steps: Step S1: Acquire image frames containing food using an image acquisition device and perform preprocessing.

[0020] Users take photos of their meals using smart mobile devices, including smartphones, smartwatches, and smart glasses. The results include single color images, multi-angle photos, and video streams. For multi-angle photos and video streams, the best image frame is selected or multiple frames are fused for analysis. The extracted image frames undergo brightness equalization, color correction, and distortion correction for subsequent image recognition processing.

[0021] In this embodiment, the user's plate is used as the standard reference during the food photography process. The plate size is preset as the size benchmark, which can achieve seamless calibration.

[0022] Step S2: Select a deep learning instance segmentation model and train it to segment the image frames. Identify and segment the types of food in the image, and output the segmentation mask and segmentation confidence score for each type of food.

[0023] Step S3: Based on the segmentation results, perform 3D reconstruction using image geometric information to obtain food volume information. The 3D reconstruction strategy includes: (1) When a standard reference is preset in step S1, the physical size of the food is calculated based on the image ratio relationship using the standard reference.

[0024] (2) When a single color image is acquired in step S1 and there is no standard reference, a monocular depth estimation model is used to predict the depth map and restore the three-dimensional volume of the meal.

[0025] (3) When collecting multi-angle images or video streams in step S1, obtain accurate food volume information by generating a 3D point cloud model.

[0026] Step S4: Convert the food volume information into weight information, calculate the total carbohydrate content based on the unit carbohydrate content of different types of food, and finally add them together to obtain the estimated total carbohydrate intake value Total_Carbs_estimated; at the same time, calculate the uncertainty index of this estimate.

[0027] The specific calculation method for the uncertainty index is as follows: For each food instance i, calculate the local uncertainty U_i, and finally summarize it into the overall uncertainty index U_total. Each U_i includes visual perception uncertainty U_vision_i and physical quantification uncertainty U_volume_i.

[0028] U_vision_i depends on the instance segmentation result in step S2, including identification uncertainty and segmentation uncertainty. The identification uncertainty U_ident_i = 1 - P_i, where P_i represents the segmentation confidence in step S2. The segmentation uncertainty U_seg_i is the average entropy value of the segmentation mask boundary. Finally, U_vision_i is the weighted sum of U_ident_i and U_seg_i.

[0029] U_volume_i depends on the 3D reconstruction strategy selected in step S3. When strategy (1) is adopted, U_volume_i is set as a function positively correlated with distance, that is, the farther the food is from the camera, the larger U_volume_i is. When strategy (2) is adopted, the uncertainty map output by the depth estimation model is used directly, and the average uncertainty in the food area is calculated as U_volume_i. When strategy (3) is adopted, U_volume_i is set based on the sparsity of the 3D point cloud.

[0030] The final calculation yields: U_total=Σ((Carbs_i / Total_Carbs_estimated)*(α*U_vision_i+β*U_volume_i)) Where Carbs_i represents the carbohydrate contribution of meal instance i, and α and β are weighting coefficients. After normalizing U_total, the final range is [0, 1], where 0 represents complete certainty and 1 represents complete uncertainty.

[0031] Step S5: The artificial pancreas system dynamically adjusts the insulin infusion strategy based on Total_Carbs_estimated and U_total obtained in step S4; When U_total≤t1, the uncertainty is low, and the full dose is infused based on Total_Carbs_estimated.

[0032] When t1 < U_total ≤ t2, it falls under the category of medium uncertainty. In this case, the infusion dose should be calculated based on m*Total_Carbs_estimated, or the infusion interval window should be extended to smooth out blood glucose fluctuations.

[0033] When U_total > t2, it is considered a high uncertainty. At this point, automatic infusion should be stopped, and the user should be immediately prompted to manually confirm or enter the infusion dose.

[0034] t1 and t2 are preset thresholds, and m∈[0,1].

[0035] Step S6: The artificial pancreas system displays the type, weight, Carbs_i, and Total_Carbs_estimated for each meal example to the user, and provides the infusion strategy provided in step S5 for user confirmation. The user can increase or decrease the insulin infusion volume independently.

[0036] Any modifications made by the user are recorded by the system and used as high-precision, high-weight label data, prioritized for subsequent model optimization. Once confirmed by the user or deemed safe by the system, Total_Carbs_estimated is automatically transmitted to the APS main controller via a secure communication protocol (such as Bluetooth). The APS then combines real-time blood glucose and active insulin information to calculate and execute the infusion of large doses of insulin during meals.

[0037] Step S7: Continuously optimize the model based on a multi-layer closed-loop feedback mechanism. Specifically, Step S7.1: Completely record the Total_Carbs_estimated for this meal, the insulin dose injected, and the subsequent continuous glucose monitoring (CGM) data.

[0038] Step S7.2, Real-time Alarm Based on Short-Term Feedback: Real-time monitoring of the initial rise slope of postprandial blood glucose. When the actual slope exceeds a certain range of the model slope predicted based on the infusion dose and estimated carbohydrate intake, the system issues an early warning and, under strict safety rules, suggests or executes a small correction dose, achieving "feedforward + feedback".

[0039] Step S7.3: Model calibration based on long-term feedback: When the user meets the criteria of stable pre-meal blood glucose, no post-meal exercise, and no drastic emotional changes, model calibration is initiated. The actual blood glucose curve is compared with the predicted blood glucose curve under Total_Carbs_estimated. The ratio of the area under the curve (AUC) is calculated to obtain systematic bias, and a calibration factor u is set to calibrate Total_Carbs_estimated to the actual total carbohydrate intake. The calibration factor u is set within the preset safety boundary range [a1, a2]. The calibrated Total_Carbs = Total_Carbs_estimated * u.

[0040] Step S7.4: After multiple iterations, the system's estimation accuracy of Total_Carbs_estimated becomes increasingly closer to the user's actual metabolic response and dietary habits, ultimately achieving highly personalized and precise management.

[0041] Based on the above-described fully closed-loop artificial pancreas control method, this invention also designs a corresponding fully closed-loop artificial pancreas system, which integrates an image processing module and a carbohydrate quantification module on the basis of existing fully closed-loop artificial pancreas systems. The image processing module is used to execute the segmentation processing step in step S2, and the carbohydrate quantification module is used to execute steps S3-S4. These two modules exist in software or hardware form in a mobile computing device, a cloud server, or an APS main controller.

[0042] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An integrated carbohydrate quantification full closed loop artificial pancreas control method, characterized by, The method comprises the following steps: Step S1, obtaining an image frame containing food based on an image acquisition device and performing preprocessing; Step S2, selecting a deep learning instance segmentation model and training the same to perform segmentation processing on the image frame; identifying and segmenting the food types contained in the image, and outputting the segmentation mask and segmentation confidence of each food type; Step S3, based on the segmentation processing result, combining image geometric information to perform three-dimensional reconstruction to obtain food volume information; Step S4, converting the food volume information into weight information, and calculating the total carbohydrate content according to the unit carbohydrate content of different types of food to obtain an estimated value of total carbohydrate intake Total_Carbs_estimated; meanwhile, the uncertainty index U_total of the estimated value is calculated and outputted; Step S5, the artificial pancreas system dynamically adjusts the insulin infusion strategy based on the Total_Carbs_estimated and U_total; Step S6, the artificial pancreas system shows the user the food instance types, weight, Carbs_i and Total_Carbs_estimated, and provides the user with the infusion strategy provided in step S5 for confirmation; the user's modification behavior will be recorded by the system and used as high-precision and high-weight label data, which is preferentially used for subsequent model optimization; Step S7, subsequent model optimization is performed based on a multi-layer closed-loop feedback mechanism, a calibration factor is set and iteration is performed, and the Total_Carbs_estimated is calibrated to a required precision.

2. The integrated carbohydrate quantification full closed loop artificial pancreas control method according to claim 1, characterized in that, The image acquisition device in step S1 includes a smartphone, a smartwatch, and smart glasses; the image acquisition result includes a single color picture, multi-angle photos, and a video stream; the extracted multiple image frames are subjected to brightness equalization, color correction, and distortion correction processing; When multi-angle photos or a video stream are collected, the best image frame or multiple frame fusion analysis is selected.

3. The integrated carbohydrate quantification full closed loop artificial pancreas control method according to claim 1, wherein, The three-dimensional reconstruction strategy in step S3 includes: (1) when a standard reference object is pre-set in step S1, the food physical size is calculated based on the standard reference object through image proportion relationship; (2) when a single color picture is collected in step S1 without a standard reference object, a monocular depth estimation model is used to perform depth map prediction to restore the three-dimensional volume of the food; (3) when multi-angle pictures or a video stream are collected in step S1, a 3D point cloud model is generated to obtain accurate food volume information.

4. The integrated carbohydrate quantification full closed loop artificial pancreas control method according to claim 3, wherein, The uncertainty index U_total calculation method specifically includes: Calculate the local uncertainty U_i for each food instance i, and each U_i includes visual perception uncertainty U_vision_i and physical quantization uncertainty U_volume_i; U_vision_i includes recognition uncertainty and segmentation uncertainty; wherein the recognition uncertainty U_ident_i = 1 - P_i, P_i represents the segmentation confidence obtained in step S2; the segmentation uncertainty U_seg_i is the average entropy value of the segmentation mask boundary; U_vision_i is the weighted sum of U_ident_i and U_seg_i; Set the physical quantitative uncertainty U_volume_i based on the three-dimensional reconstruction strategy selected in step S3; when strategy (1) is adopted, U_volume_i is set as a function positively correlated with the distance, that is, the farther the meal is from the camera, the greater U_volume_i is; when strategy (2) is adopted, the uncertainty map output by the depth estimation model is directly used, and the average uncertainty in the food region is calculated as U_volume_i; when strategy (3) is adopted, U_volume_i is set based on the sparsity of the 3D point cloud; finally, the following is calculated: U_total=Σ((Carbs_i / Total_Carbs_estimated)*(α*U_vision_i+β*U_volume_i)) where Carbs_i represents the carbohydrate contribution of meal instance i, and α and β are weight coefficients; U_total is normalized to finally range from 0 to 1.

5. The integrated carbohydrate quantification full closed loop artificial pancreas control method according to claim 1, wherein, The dynamic adjustment of the insulin infusion strategy in step S5 specifically includes: When U_total≤t1, it belongs to low uncertainty, at this time, full-dose infusion is performed based on Total_Carbs_estimated; when t1 6. The integrated carbohydrate quantification full closed loop artificial pancreas control method of claim 1, wherein, The subsequent model optimization method in step S7 includes: Step S7.1, record Total_Carbs_estimated, injected insulin dose, and subsequent continuous blood glucose monitoring data of this meal; Step S7.2, real-time alarm based on short-term feedback: real-time monitoring of the initial rising slope of postprandial blood glucose, when the actual slope exceeds the model slope predicted based on the infusion dose and the estimated carbohydrate amount within a certain range, the system gives an early warning, and under strict safety rules, it suggests or executes a small amount of correction dose; Step S7.3, model calibration based on long-term feedback: when the user meets the conditions of stable pre-meal blood glucose, no postprandial exercise, and no emotional changes, start model calibration; compare the actual blood glucose curve and the predicted blood glucose curve under Total_Carbs_estimated, calculate the ratio of the area under the curve AUC to obtain the systematic bias, and set a calibration factor u to calibrate Total_Carbs_estimated to the actual total carbohydrate intake; wherein the calibration factor u is set within the preset safety boundary range [a1, a2], and the calibrated Total_Carbs=Total_Carbs_estimated*u; Step S7.4, through multiple iterations, the system gradually fits the accuracy of the estimation of Total_Carbs_estimated to the user's true metabolic response and dietary habits, and finally achieves highly personalized precision management.

7. A fully closed loop artificial pancreas system, characterized by, integrated with image processing module and carbohydrate quantification module for performing the method as claimed in any one of claims 1-6; the two modules exist in the form of software or hardware in mobile computing device, cloud server or APS main controller; the image processing module is used for performing the segmentation processing step in step S2, and the carbohydrate quantification module is used for performing steps S3-S4.

8. A terminal device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method as claimed in any one of claims 1 to 6 when executing the program.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the steps of the method as claimed in any one of claims 1 to 6 when executed by the processor.

Images