Dialysate temperature safety monitoring system based on deep reinforcement learning

Abstract

The invention provides a dialysate temperature safety monitoring system based on deep reinforcement learning, and belongs to the technical field of safety monitoring systems, and the system comprises a temperature measurement device which is used for monitoring the temperature data of dialysate in real time; the control execution device is used for controlling the dialysate heating or cooling device to ensure that the temperature of the dialysate is maintained in a safe interval; and the temperature monitoring system is used for acquiring real-time temperature data acquired by the temperature measuring devices on different hemodialysis machines in a wireless or wired manner based on the hyper-converged architecture. The method has the beneficial effects that the captured temperature monitoring information is subjected to data analysis through the deep reinforcement learning model, so that the abnormal data information in the temperature monitoring information is identified, and the control execution device of the hemodialysis machine corresponding to the source of the abnormal data information sends the output control signal; and online remote intelligent control on the control execution device can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of security monitoring systems, and relates to a dialysis fluid temperature security monitoring system based on deep reinforcement learning. Background Art

[0002] Hemodialysis is one of the important means for treating renal failure, and the temperature control of dialysis fluid is crucial for the health of patients. Existing dialysis fluid temperature control systems usually rely on simple temperature sensors and manual adjustment mechanisms, lacking the capabilities of intelligence and remote monitoring, resulting in slow response speed in case of abnormal situations and increasing the risks for patients.

[0003] For example, a patent for invention with the application number CN202310886427.5 provides a dialysis fluid temperature monitoring platform, including a temperature measuring device arranged on a hemodialysis machine and a temperature monitoring system for information interaction with the device; the temperature monitoring system obtains real-time temperature data collected on different hemodialysis machines in a wireless or wired manner based on a hyper-converged architecture to achieve remote real-time monitoring by medical staff.

[0004] In summary, although some existing technical solutions can achieve remote real-time monitoring by medical staff, when neither medical staff nor patient guardians pay timely attention to abnormal monitoring results, once there is a control failure inside the hemodialysis instrument, accidents may occur, and there is a large room for improvement. Summary of the Invention

[0005] The object of the present invention is to address the above problems existing in the prior art and propose a dialysis fluid temperature security monitoring system based on deep reinforcement learning.

[0006] The object of the present invention can be achieved by the following technical solutions: A dialysis fluid temperature security monitoring system based on deep reinforcement learning includes:

[0007] A temperature measuring device, which is arranged on a hemodialysis machine, and the temperature measuring device is used for real-time monitoring of the temperature data of the dialysis fluid;

[0008] A control execution device, which is arranged on a hemodialysis machine, and the control execution device is used for controlling a heating or cooling device of the dialysis fluid to ensure that the temperature of the dialysis fluid is maintained within a safe range;

[0009] A temperature monitoring system, which conducts information interaction with the temperature measuring device, and the temperature monitoring system obtains in a wireless or wired manner the real-time temperature data collected by the temperature measuring device on different hemodialysis machines based on a hyper-converged architecture, and transmits control signals to the control execution devices on each hemodialysis machine to achieve remote real-time monitoring by medical staff.

[0010] In the above-mentioned dialysis fluid temperature safety monitoring system based on deep reinforcement learning, the temperature monitoring system includes:

[0011] A data acquisition module that grabs dialysis fluid temperature monitoring information in different medical institutions according to a preset time;

[0012] A cloud platform-based data center that is provided with a deep reinforcement learning model. The deep reinforcement learning model performs data analysis on the grabbed temperature monitoring information to identify abnormal data information therein, and sends an output control signal to the control execution device of the hemodialysis machine corresponding to the source of the abnormal data information.

[0013] In the above-mentioned dialysis fluid temperature safety monitoring system based on deep reinforcement learning, the temperature monitoring system further includes:

[0014] A first safety alarm module that is used to emit sound and light to remind medical staff. When the deep reinforcement learning model identifies that the abnormal data information exceeds the preset safety range, it sends an alarm signal to the first safety alarm module to control the first safety alarm module to emit sound and light to remind medical staff.

[0015] In the above-mentioned dialysis fluid temperature safety monitoring system based on deep reinforcement learning, the temperature monitoring system further includes:

[0016] A remote terminal that receives the temperature monitoring information and output control signal records processed by the cloud platform-based data center, and displays the patient's temperature monitoring information and output control signal records in a visual form.

[0017] In the above-mentioned dialysis fluid temperature safety monitoring system based on deep reinforcement learning, the remote terminal is client software that can receive external control instructions and intervene in the operating temperature of the dialysis fluid.

[0018] In the above-mentioned dialysis fluid temperature safety monitoring system based on deep reinforcement learning, the remote terminal includes a second safety alarm module that is used to emit sound and light to remind users. When the deep reinforcement learning model identifies that the abnormal data information exceeds the preset safety range, it sends an alarm signal to the second safety alarm module to control the second safety alarm module to emit sound and light to remind users.

[0019] In the above-mentioned dialysis fluid temperature safety monitoring system based on deep reinforcement learning, the deep reinforcement learning model includes:

[0020] A convolutional neural network (CNN) part for processing local features in the input time series data;

[0021] The long short-term memory network (LSTM) part is used to capture long-term dependencies in time series data;

[0022] The experience replay buffer is used to store the historical experience of the agent, and the historical experience includes state, action, reward, and next state;

[0023] The target network is used to provide stable target Q values, and the parameters of the target network are updated every predetermined period;

[0024] The main network is used to learn the optimal policy and regularly copy its parameters to the target network.

[0025] In the above-mentioned hemodialysis fluid temperature safety monitoring system based on deep reinforcement learning, the data center based on the cloud platform is also provided with a feedback evaluation module, which includes:

[0026] The effect evaluation unit is used to compare the changes in the main physiological indicators of the patient before and after adjustment;

[0027] The model update unit is used to dynamically adjust the parameters of the deep reinforcement learning model according to the results of the effect evaluation unit.

[0028] In the above-mentioned hemodialysis fluid temperature safety monitoring system based on deep reinforcement learning, the number of temperature measurement devices is several, and each of the temperature measurement devices measures the liquid at different positions in the hemodialysis machine pipeline in a non-contact manner.

[0029] In the above-mentioned hemodialysis fluid temperature safety monitoring system based on deep reinforcement learning, the temperature measurement device is an infrared temperature sensor.

[0030] Compared with the prior art, the beneficial effects of the present invention are:

[0031] 1. Through the deep reinforcement learning model, data analysis is carried out on the captured temperature monitoring information, so as to identify abnormal data information therein, and an output control signal is sent to the control execution device of the hemodialysis machine corresponding to the source of the abnormal data information, which can realize remote intelligent control of the control execution device online.

[0032] 2. The first safety alarm module can timely give out a sound and light alarm when detecting abnormal temperatures exceeding the preset safety range, reminding medical staff to take measures, thereby improving the response speed and reducing potential risks.

[0033] 3. The remote terminal can receive and visually display the processed temperature monitoring information, enabling medical staff to monitor the patient's status anytime and anywhere, improving the convenience and response efficiency of medical services.

[0034] 4. It allows medical staff to directly intervene in the operating temperature of the dialysis fluid through a remote terminal, providing a more flexible operation method and further ensuring the safety during the treatment process.

[0035] 5. When the second safety alarm module detects an abnormal situation, it will also issue an alarm, but it acts on the remote terminal, adding a redundant alarm mechanism and further enhancing the safety guarantee level. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is the overall system architecture diagram of the present invention.

[0037] Figure 2 It is the internal structure diagram of the temperature monitoring system of the present invention.

[0038] Figure 3 It is the model structure diagram of the deep reinforcement learning of the present invention.

[0039] Figure 4 It is the layout diagram of the temperature measuring device of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0040] The following are specific embodiments of the present invention and in combination with the accompanying drawings, the technical solutions of the present invention are further described, but the present invention is not limited to these embodiments.

[0041] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If this specific posture changes, then the directional indications will also change accordingly.

[0042] In addition, in the present invention, descriptions such as "first", "second", "one" are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "a plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0043] In the present invention, unless otherwise clearly defined and limited, terms such as "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two components or the interaction relationship between two components, unless otherwise clearly limited. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0044] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0045] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Those skilled in the art to which the present invention pertains can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

[0046] As Figures 1-4 shown, a dialysate temperature safety monitoring system based on deep reinforcement learning includes: a temperature measurement device, a control execution device, and a temperature monitoring system.

[0047] Among them, the temperature measurement device is arranged on the hemodialysis machine, and the temperature measurement device is used to monitor the temperature data of the dialysate in real time.

[0048] Among them, the control execution device is arranged on the hemodialysis machine, and the control execution device is used to control the heating or cooling device of the dialysate to ensure that the temperature of the dialysate is maintained within a safe range.

[0049] Among them, the temperature monitoring system interacts with the temperature measurement device. The temperature monitoring system obtains the real-time temperature data collected by the temperature measurement device on different hemodialysis machines in a wireless or wired manner based on a hyper-converged architecture, and transmits control signals to the control execution device on each hemodialysis machine to achieve remote real-time monitoring by medical staff.

[0050] In this embodiment, by integrating the temperature measurement device, the control execution device, and the temperature monitoring system, not only the real-time monitoring of the dialysate temperature is realized, but also the intelligent control of the dialysate temperature is realized, ensuring the safety of the dialysate temperature during hemodialysis and reducing the risks caused by temperature runaway.

[0051] As Figures 1-4 shown, on the basis of the above embodiment, the temperature monitoring system includes: a data acquisition module and a data center based on a cloud platform.

[0052] Among them, the data acquisition module grabs the dialysate temperature monitoring information in different medical institutions according to a preset time.

[0053] Among them, the data center based on the cloud platform is provided with a deep reinforcement learning model. The deep reinforcement learning model analyzes the captured temperature monitoring information to identify abnormal data information therein, and sends an output control signal to the control execution device of the hemodialysis machine corresponding to the source of the abnormal data information.

[0054] It should be noted here that by using the data acquisition module and the data center based on the cloud platform, cross-medical institution data sharing and remote management can be realized, enhancing the scalability and flexibility of the system, and improving the efficiency and accuracy of data analysis.

[0055] In this embodiment, the deep reinforcement learning model analyzes the captured temperature monitoring information to identify abnormal data information therein, and sends an output control signal to the control execution device of the hemodialysis machine corresponding to the source of the abnormal data information, enabling remote intelligent control of the control execution device online.

[0056] As Figures 1-4 shown, on the basis of the above embodiment, the temperature monitoring system further includes:

[0057] A first safety alarm module, which is used to emit sound and light to remind medical staff. When the deep reinforcement learning model identifies that the abnormal data information exceeds the preset safety range, it sends an alarm signal to the first safety alarm module to control the first safety alarm module to emit sound and light to remind medical staff.

[0058] It should be noted here that the first safety alarm module can emit sound and light alarms in time when detecting abnormal temperatures exceeding the preset safety range, reminding medical staff to take measures, thereby improving the response speed and reducing potential risks.

[0059] In this embodiment, the deep reinforcement learning model can not only realize remote intelligent control of the control execution device online, but also send an alarm signal to the first safety alarm module in case of emergency to control the first safety alarm module to emit sound and light alarms to remind medical staff to pay attention to this situation in time for manual intervention.

[0060] As Figures 1-4 shown, on the basis of the above embodiment, the temperature monitoring system further includes:

[0061] A remote terminal, which receives the temperature monitoring information and the output control signal record processed by the data center based on the cloud platform, and displays the temperature monitoring information and the output control signal record of the patient in a visual form.

[0062] In this embodiment, the remote terminal can receive and visually display the processed temperature monitoring information, enabling medical staff to monitor the patient's condition anytime and anywhere, thus improving the convenience and response efficiency of medical services.

[0063] As Figures 1-4 shown, based on the above embodiment, the remote terminal is client software, which can receive external control instructions to intervene in the operating temperature of the dialysate.

[0064] In this embodiment, medical staff are allowed to directly intervene in the operating temperature of the dialysate through the remote terminal, providing a more flexible operation method and further ensuring the safety during the treatment process.

[0065] As Figures 1-4 shown, based on the above embodiment, the remote terminal includes a second safety alarm module, which is used to emit sound and light to remind the user. When the deep reinforcement learning model identifies that the abnormal data information exceeds the preset safety range, it sends an alarm signal to the second safety alarm module to control the second safety alarm module to emit sound and light to remind the user.

[0066] It should be noted here that the second safety alarm module will also emit an alarm when detecting an abnormal situation, but it acts on the remote terminal, adding a redundant alarm mechanism and further improving the safety guarantee level.

[0067] In this embodiment, the deep reinforcement learning model can not only remotely and intelligently control the control execution device online, but also send an alarm signal to the second safety alarm module in case of an emergency to control the second safety alarm module to emit a sound and light alarm to remind the user (patient or guardian) to also notice this situation in time for manual intervention.

[0068] As Figures 1-4 shown, based on the above embodiment, the deep reinforcement learning model includes: a convolutional neural network (CNN) part, a long short-term memory network (LSTM) part, an experience replay buffer, a target network, and a main network.

[0069] Among them, the convolutional neural network (CNN) part is used to process the local features in the input time series data.

[0070] Among them, the long short-term memory network (LSTM) part is used to capture the long-term dependencies in the time series data.

[0071] Among them, the experience replay buffer is used to store the historical experience of the agent, and the historical experience includes state, action, reward, and next state.

[0072] Among them, the target network is used to provide a stable target Q value, and the parameters of the target network are updated every predetermined period.

[0073] Among them, the main network is used to learn the optimal policy and regularly copy its parameters to the target network.

[0074] In this embodiment, technical components such as convolutional neural networks and long short-term memory networks are adopted, combined with an experience replay buffer and a target network, to improve the learning ability and prediction accuracy of the model, making the temperature monitoring more accurate and effective.

[0075] As Figures 1-4 shown, on the basis of the above embodiment, the data center based on the cloud platform is further provided with a feedback evaluation module, which includes: an effect evaluation unit and a model update unit.

[0076] Among them, the effect evaluation unit is used to compare the changes in the main physiological indicators of the patient before and after adjustment.

[0077] Among them, the model update unit is used to dynamically adjust the parameters of the deep reinforcement learning model according to the results of the effect evaluation unit.

[0078] In this embodiment, through the effect evaluation unit and the model update unit, the parameters of the deep reinforcement learning model are dynamically adjusted to optimize the model performance and ensure long-term stable monitoring effects.

[0079] As Figures 1-4 shown, on the basis of the above embodiment, the number of temperature measurement devices is several, and each of the temperature measurement devices measures the liquid at different positions in the hemodialysis machine pipeline in a non-contact manner.

[0080] In this embodiment, multiple temperature measurement devices can more comprehensively cover different positions of the hemodialysis machine, provide more accurate temperature monitoring, and enhance the reliability and monitoring accuracy of the system.

[0081] As Figures 1-4 shown, on the basis of the above embodiment, the temperature measurement device is an infrared temperature sensor.

[0082] In this embodiment, a non-contact infrared temperature sensor is used for measurement, avoiding the risks of pollution or damage that may be brought by physical contact, and at the same time improving the measurement speed and accuracy.

[0083] Preferably, the infrared temperature sensor can be calibrated according to the following calibration steps:

[0084] S1. Build a temperature measurement environment to make the liquid in the dialysis fluid bag flow through the pipeline;

[0085] S2. Calibrate the infrared temperature sensor for at least one temperature value;

[0086] S3. Detect the temperature value of the liquid flowing through the pipeline by the infrared temperature sensor;

[0087] S4. Collect the water temperature of the actual liquid in the pipeline with a data recorder;

[0088] S5. Compare the temperature value measured by the infrared temperature sensor with the true water temperature obtained by the data recorder, so as to obtain the error value of the infrared temperature sensor, and adjust the calibration coefficient of the infrared temperature sensor with this error value.

Claims

1. A dialysate temperature safety monitoring system based on deep reinforcement learning, characterized in that: include: A temperature measuring device, which is arranged on the hemodialysis machine and is used to monitor the temperature data of the dialysate in real time; A control execution device, which is arranged on the hemodialysis machine, and is used to control the dialysate heating or cooling device to ensure that the temperature of the dialysate is maintained within a safe range; A temperature monitoring system exchanges information with the temperature measuring device. The temperature monitoring system obtains real-time temperature data collected by the temperature measuring devices on different hemodialysis machines in a wireless or wired manner based on a hyper-converged architecture, and transmits control signals to the control execution devices on each hemodialysis machine to achieve remote real-time monitoring of medical care.

2. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 1, characterized in that: The temperature monitoring system comprises: A data acquisition module that captures dialysate temperature monitoring information in different medical institutions at preset times; The data center based on the cloud platform is provided with a deep reinforcement learning model, which performs data analysis on the captured temperature monitoring information to identify abnormal data information therein, and sends an output control signal to the control execution device of the hemodialysis machine corresponding to the source of the abnormal data information.

3. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 2, characterized in that: The temperature monitoring system also includes: The first safety alarm module is used to emit sound and light to alert medical staff. When the deep reinforcement learning model recognizes that abnormal data information exceeds a preset safety range, it sends an alarm signal to the first safety alarm module to control the first safety alarm module to emit sound and light to alert medical staff.

4. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 2, characterized in that: The temperature monitoring system also includes: A remote terminal receives the temperature monitoring information and output control signal records processed by the cloud platform-based data center, and displays the patient's temperature monitoring information and output control signal records in a visual form.

5. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 4, characterized in that: The remote terminal is client software, which can receive external control instructions and intervene in the operating temperature of the dialysate.

6. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 5, characterized in that: The remote terminal includes a second security alarm module, which is used to emit sound and light to alert the user. When the deep reinforcement learning model recognizes that the abnormal data information exceeds a preset safety range, it sends an alarm signal to the second security alarm module to control the second security alarm module to emit sound and light to alert the user.

7. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 2, characterized in that: Deep reinforcement learning models include: The convolutional neural network (CNN) part is used to process local features in the input time series data; The Long Short-Term Memory (LSTM) network is used to capture long-term dependencies in time series data. An experience replay buffer is used to store the agent's historical experience, including state, action, reward, and next state; A target network, used to provide a stable target Q value, and update the parameters of the target network every predetermined period; The master network is used to learn the optimal policy and periodically copies its parameters to the target network.

8. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 7, characterized in that: The cloud platform-based data center is also provided with a feedback evaluation module, which includes: Effect evaluation unit, used to compare the changes in patients' main physiological indicators before and after adjustment; A model updating unit is used to dynamically adjust the parameters of the deep reinforcement learning model according to the result of the effect evaluation unit.

9. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 1, characterized in that: There are several temperature measuring devices, and each of the temperature measuring devices measures the liquid at different positions in the pipeline of the hemodialysis machine in a non-contact manner.

10. A dialysate temperature safety monitoring system based on deep reinforcement learning according to claim 9, characterized in that: The temperature measuring device is an infrared temperature sensor.

Images