Medical auxiliary monitoring method and device based on monitor screen waveform camera
Recording the monitor screen through the mobile terminal camera, extracting the invasive arterial waveform and calculating the cardiac output and stroke output, solving the problem that minimally invasive hemodynamic monitoring cannot be implemented in underdeveloped areas, and achieving advanced hemodynamic parameters continuous monitoring is suitable for areas with limited resources.
Patent Information
- Application Number
- CN202411077738.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-07
AI Technical Summary
The existing minimally invasive hemodynamic monitoring equipment is difficult to implement in underdeveloped areas, resulting in the inability to achieve precise patient management and lack of non-invasive, economical and continuous cardiovascular function monitoring methods.
Record the monitor screen in real time through the camera on the mobile terminal, obtain invasive arterial waveform video, and calculate the cardiac output and stroke-per-stroke output through tilt correction, scale detection, red blood pressure waveform extraction and deep learning algorithms to achieve advanced blood Continuous monitoring of flow dynamics parameters.
Continuous, automatic and prospective monitoring of advanced hemodynamic parameters is achieved on mobile terminals, which can be consistent with the results of VIJI flow measurements, and is suitable for areas with limited resources, providing assistance to anesthesiologist patient management and remote guidance.
Smart Images

Figure CN118592919B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of monitor data processing, and in particular relates to a medical auxiliary monitoring method and device based on monitor screen waveform photography. Background Art
[0002] Appropriate fluid management during anesthesia is beneficial to the patient's prognosis. The main purpose of fluid infusion is to increase cardiac output and thus improve tissue oxygen delivery. Hypovolemia can lead to poor peripheral vascular perfusion, organ dysfunction and increased mortality, while hypervolemia is associated with poor prognosis, such as increased hospital stay, complications such as pulmonary edema, and increased mortality. Monitoring of advanced hemodynamic parameters such as cardiac output (CO) and stroke volume (SV) is an important indicator for assessing cardiovascular function and guiding individualized fluid therapy strategies to avoid unnecessary fluid infusion. Ideally, advanced hemodynamic parameter monitoring should be non-invasive, cost-effective, continuous and reliable. To date, there is no ideal hemodynamic monitoring method that meets the above standards. Compared with invasive monitoring such as pulmonary thermodilution, minimally invasive hemodynamic monitoring equipment has attracted more and more attention. Among them, the Vigileo FloTrac technology Edwards Lifesciences LLC, Irvine, CA, USA has been verified and improved worldwide and applied in most hospitals and institutions. This minimally invasive monitoring through radial artery puncture can provide advanced hemodynamic parameters such as CO and SV. However, minimally invasive hemodynamic monitoring equipment requires specific and expensive monitors and sensors, and precise patient management cannot be implemented in underdeveloped areas. Summary of the invention
[0003] The purpose of the present invention is to provide a medical auxiliary monitoring method and device based on monitor screen waveform photography in view of the deficiencies in the prior art.
[0004] According to a first aspect of the present invention, there is provided a medical auxiliary monitoring method based on monitor screen waveform photography, the method comprising:
[0005] Step S1, recording the monitor screen in real time through the camera on the mobile terminal to obtain the invasive arterial waveform video of the monitor;
[0006] Step S2, sampling the invasive arterial waveform video of the monitor obtained in step S1 in real time to obtain an invasive arterial waveform sampling image;
[0007] Step S3, performing tilt correction processing on the invasive arterial waveform sampling image obtained in step S2 to obtain an invasive arterial waveform tilt-corrected image;
[0008] Step S4, performing scale line detection processing on the invasive arterial waveform tilt-corrected image obtained in step S3 to obtain a scale value for reference of the blood pressure value;
[0009] Step S5, purifying the red blood pressure waveform portion of the invasive arterial waveform tilt-corrected image obtained in step S3 to obtain a red blood pressure waveform image;
[0010] Step S6, converting the red blood pressure waveform image obtained in step S5 from a pixel block into a blood pressure waveform relative value array, and converting the blood pressure waveform relative value array into a blood pressure waveform true value array based on the scale value of the blood pressure value reference obtained in step S4;
[0011] Step S7, when an accident occurs during the execution of any one of the steps S3 to S5, the invasive arterial waveform sampling image obtained in the step S2 is input into the universal visual segmentation model, and the image segmentation conditions are input into the universal visual segmentation model, and a red blood pressure waveform image is output, and the red blood pressure waveform image is converted into a blood pressure waveform true value array.
[0012] Furthermore, the step S1 is specifically as follows:
[0013] Fix the camera of the mobile terminal in front of the monitor to collect the invasive arterial waveform video of the monitor;
[0014] The camera shake is compensated by physically moving the camera components and combined with an anti-shake algorithm to capture two frames of video, estimate the motion between the frames, and correct the camera shake.
[0015] Furthermore, the step S4 is specifically as follows:
[0016] The Canny edge detection algorithm is used to extract all horizontal dotted lines in the invasive arterial waveform tilt-corrected image in step S3. Different dotted lines in the invasive arterial waveform tilt-corrected image represent different scale values of blood pressure, thereby determining the scale value of the blood pressure value reference in the invasive arterial waveform tilt-corrected image.
[0017] Furthermore, the step S5 includes the following sub-steps:
[0018] Step S51, taking into account the difference in hue range, a dynamic threshold technology is used to extract the red blood pressure waveform portion, wherein the difference in hue range is manifested in the difference in brightness and color display of the monitor screen;
[0019] Step S52, identifying and removing non-waveform pixel blocks in the red blood pressure waveform part extracted in step S51 by connected domain analysis technology, thereby retaining only the largest connected domain as the true waveform area to obtain a red blood pressure waveform image.
[0020] Furthermore, the step S7 is specifically as follows:
[0021] The API interface is used to call the general visual segmentation model, and the image set previously converted from the video stream is cyclically accessed to process the images in batches. In the general visual segmentation model, the SAM segmentation function is called for each image to be processed to segment the red blood pressure waveform part from the image.
[0022] When segmenting the first image, adjust the segmentation parameters of the general visual segmentation model to optimize the segmentation effect; if the segmentation result does not meet the requirements, re-iterate and adjust the segmentation parameters or make manual corrections to ensure that the segmentation parameters remain consistent during batch processing;
[0023] After segmenting each image, the segmentation results are saved in a specified folder. After completing batch segmentation, the output needs to be post-processed, and the post-processing includes removing noise, smoothing edges, or performing further morphological operations to obtain a red blood pressure waveform image; the red blood pressure waveform image is converted through a data conversion method to obtain a blood pressure waveform true value array.
[0024] Furthermore, the medical auxiliary monitoring method based on monitor screen waveform photography also includes the following steps:
[0025] Inputting the blood pressure waveform true value array into a hemodynamic analysis system in real time;
[0026] The hemodynamic analysis system includes a hemodynamic host computer software, wherein the hemodynamic host computer software is embedded with a blood pressure waveform true value array feature point recognition algorithm for hemodynamic parameter calculation and a hemodynamic parameter calculation model;
[0027] The blood pressure waveform true value array feature point recognition algorithm is used to identify feature points in the blood pressure waveform true value array data, and the feature points include systolic pressure point, diastolic pressure point and mid-fall point;
[0028] The hemodynamic parameter calculation model uses a deep learning algorithm to train patient data collected over a long period of time to learn a cardiac output prediction model and a stroke volume prediction model; the blood pressure waveform true value array feature point recognition algorithm identifies feature points in the blood pressure waveform true value array data as input to the hemodynamic parameter calculation model, and outputs advanced hemodynamic parameters, including cardiac output CO and stroke volume SV.
[0029] Furthermore, the recognition process of the blood pressure waveform true value array feature point recognition algorithm includes:
[0030] By performing a differential operation on the collected true value array data of the blood pressure waveform, the zero-crossing point is found in the first-order differential signal, thereby identifying the systolic pressure point; the same differential operation is performed after inverting the data to identify the diastolic pressure point; by calculating the curvature of the data between the systolic pressure point and the diastolic pressure point, the point with the maximum curvature is identified as the descending gorge point.
[0031] Furthermore, the calculation formula of the cardiac output CO is: , where R represents peripheral resistance, T represents the time of a cardiac cycle, t represents the time, P represents blood pressure, and HR represents heart rate; the calculation formula of the stroke volume SV is: ,in represents the time interval of the ejection phase, represents the amount of blood pumped by the heart each time, and C represents the compliance of the artery.
[0032] Furthermore, the medical auxiliary monitoring method based on monitor screen waveform photography also includes the following steps:
[0033] Taking the advanced hemodynamic parameters and the blood pressure waveform true value array as input;
[0034] Collect the true value array of the blood pressure waveform of the patient during the operation, use the mean arterial pressure in the Vijet flow record as the label, find the true value array of the blood pressure waveform and the advanced hemodynamic parameters 5 minutes, 15 minutes and 30 minutes before the mean arterial pressure corresponding time to form three different time window data sets, and train the prediction model for adverse blood pressure conditions;
[0035] The adverse blood pressure prediction model adopts the ResNet50 network model; the adverse blood pressure prediction model is trained on data sets of three different time windows respectively, and the network model is optimized through back propagation after each iteration, and the cross entropy loss function is selected.
[0036] According to a second aspect of the present invention, there is provided a medical auxiliary monitoring device based on waveform photography of a monitor screen, comprising a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement the above-mentioned medical auxiliary monitoring method based on waveform photography of a monitor screen.
[0037] The beneficial effects of the present invention are as follows: the present invention uses the camera on the mobile terminal to record the monitor screen in real time, identify and extract invasive arterial waveforms to estimate advanced hemodynamic parameters, thereby realizing continuous, automatic, and prospective monitoring of advanced hemodynamic parameters by the mobile terminal, and being able to estimate advanced hemodynamic parameters based on the patient monitor screen image, and derive whether there is consistency and interchangeability between the cardiac output and stroke volume measured by Vijet flow. The present invention innovatively proposes to use video waveform recognition monitoring as an aid for anesthesiologists to manage patients and provide remote guidance, and can be applied to areas with limited resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0039] Figure 1 A flow chart of a medical auxiliary monitoring method based on monitor screen waveform photography provided by an exemplary embodiment;
[0040] Figure 2 A comparison diagram of a blood pressure waveform image on a monitor screen provided by an exemplary embodiment and an array of true values of the blood pressure waveform obtained by a data conversion method;
[0041] Figure 3 A comparison diagram of a blood pressure waveform image on a monitor screen provided by an exemplary embodiment and a blood pressure waveform segmented and extracted based on a general visual segmentation model;
[0042] Figure 4 An overview diagram of a single blood pressure waveform true value array analysis process provided by an exemplary embodiment;
[0043] Figure 5 A schematic diagram of a blood pressure waveform in a Vijet stream and extraction screen provided for an exemplary embodiment;
[0044] Figure 6 A comparison chart of cardiac output measured by Vijetflow and cardiac output calculated by a monitor screen recorded in real time by a camera is provided for an exemplary embodiment;
[0045] Figure 7 A comparison chart of stroke volume measured by Vijetflow and stroke volume calculated by a monitor screen recorded in real time by a camera is provided for an exemplary embodiment;
[0046] Figure 8 A structural diagram of a medical auxiliary monitoring device based on monitor screen waveform photography provided as an exemplary embodiment. DETAILED DESCRIPTION
[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0049] like Figure 1 As shown, the present invention provides a medical auxiliary monitoring method based on monitor screen waveform photography, the method comprising the following steps:
[0050] Step S1, recording the monitor screen in real time through the camera on the mobile terminal to obtain the invasive arterial waveform video of the monitor;
[0051] In some embodiments, step S1 specifically includes the following contents:
[0052] The camera of the mobile terminal is fixed in front of the monitor to collect the invasive arterial waveform video of the monitor; wherein the mobile terminal includes a mobile phone, a tablet, etc.
[0053] In order to maintain the stability of the acquired invasive arterial waveform video of the monitor, the camera shake is compensated by physically moving the camera assembly and combined with the anti-shake algorithm to capture two frames of the video, estimate the motion between the frames, and correct the camera shake.
[0054] Step S2, sampling the invasive arterial waveform video of the monitor obtained in step S1 in real time to obtain an invasive arterial waveform sampling image;
[0055] In some embodiments, step S2 specifically includes the following contents:
[0056] The invasive arterial waveform video of the monitor obtained in step S1 is sampled in real time, and the video stream is converted into an image for processing. In order to reduce the amount of data processing while ensuring the real-time nature of the data, the sampling frequency is dynamically adjusted, and the interval time of each sampling is automatically adjusted according to the dynamic change of the monitor refresh frequency, thereby optimizing the sampling efficiency. In this embodiment, for the American GE B650 monitor, the sampling interval is set to 8 seconds. Too short or too long a time will result in the loss of blood pressure data. In order for the subsequent segmentation tools to work properly, ensure that all images to be processed are of uniform format and size, perform preprocessing, resize, crop or convert the image format, and finally obtain the invasive arterial waveform sampling image.
[0057] Step S3, performing tilt correction processing on the invasive arterial waveform sampling image obtained in step S2 to obtain an invasive arterial waveform tilt-corrected image;
[0058] In some embodiments, step S3 specifically includes the following contents:
[0059] The invasive arterial waveform sampling image obtained in step S2 is subjected to tilt correction processing, and the dotted lines in the monitor screen are detected using the Hough transform. Here, the Hough transform can identify various shapes from the image, especially straight lines, and can be efficiently identified even in a noisy background. The straight line data extracted from the Hough transform result is used to calculate the actual tilt of the image. By measuring the angles between these straight lines and the ideal horizontal line and calculating the average value of these angles, an estimate of the overall image tilt is obtained; the image is then rotated accordingly to complete the image tilt correction, and then the random sample consensus algorithm (RANSAC) is applied to further filter out abnormal detection results and determine the most reliable tilt angle, thereby achieving accurate tilt correction, so that the waveform on the screen is displayed horizontally, and an invasive arterial waveform tilt correction image is obtained. This can ensure the accuracy and reliability of subsequent waveform data analysis.
[0060] Step S4, performing scale line detection processing on the invasive arterial waveform tilt-corrected image obtained in step S3 to obtain a scale value for reference of the blood pressure value;
[0061] In some embodiments, step S4 specifically includes the following contents:
[0062] The Canny edge detection algorithm is used to extract all horizontal dotted lines in the invasive arterial waveform tilt correction image in step S3. The Canny edge detection algorithm first uses a pair of one-dimensional Gaussian filters to smooth the image, and then calculates the gradient strength and direction of each pixel in the image. Non-maximum suppression: After obtaining the gradient strength, the Canny edge detection algorithm will perform non-maximum suppression, remove non-edge pixels, and only retain some thin lines as potential edges. Double threshold detection: Use two thresholds to determine the true edge. Strong edges are directly determined as edges, and weak edges are retained when connected to strong edges, otherwise discarded. This method is used to determine the scale value of the blood pressure value in the image. In this embodiment, the five dotted lines in the invasive arterial waveform tilt correction image represent 0, 50, 100, 150, and 200 values, respectively. The positions of the five dotted lines in the image are identified respectively, and the one-to-one mapping relationship between the scale value and the position in the image is obtained. When the arterial blood pressure value is extracted later, the true blood pressure value can be calculated according to the mapping relationship.
[0063] Step S5, purifying the red blood pressure waveform portion of the invasive arterial waveform tilt-corrected image obtained in step S3 to obtain a red blood pressure waveform image;
[0064] In some embodiments, step S5 specifically includes the following contents:
[0065] Step S51, converting the invasive arterial waveform tilt-corrected image from RGB format into the HSV color space. The HSV color space represents colors in terms of hue, saturation, and brightness components, and is more suitable for dealing with color-related issues. Set upper and lower limits in the HSV color space, analyze the hue (H) distribution of all pixels in the image, and find the main hue range of the red area. Since red may span the starting point of the HSV color wheel (i.e., close to 0° and 360°), special attention needs to be paid to the processing of this area. In order to extract the red blood pressure waveform portion, a logical mask is created based on the set dynamic threshold, which only retains pixels whose hue, saturation, and brightness are within the set threshold range. Since there are differences in the hue range in the red blood pressure waveform portion of the invasive arterial waveform tilt-corrected image, a dynamic thresholding technique is used instead of a fixed threshold to extract the red blood pressure waveform portion, wherein the difference in hue range is reflected in the brightness and color display differences on the monitor screen;
[0066] Step S52, using the connected domain analysis technology to identify and remove the non-waveform pixel blocks in the red blood pressure waveform part extracted in step S51, so as to retain only the largest connected domain as the true waveform area to obtain a red blood pressure waveform image.
[0067] Step S6, converting the red blood pressure waveform image obtained in step S5 from a pixel block into a blood pressure waveform relative value array, and converting the blood pressure waveform relative value array into a blood pressure waveform true value array based on the scale value of the blood pressure value reference obtained in step S4;
[0068] Specifically, using the one-to-one mapping relationship between the scale value and the position in the image obtained in step S4, the value of the relative position of each pixel in the image is converted into the true blood pressure value according to the one-to-one mapping relationship, and a series of xy arrays are obtained, where x corresponds to the time series and y corresponds to the true value of the arterial blood pressure, and saved as a blood pressure waveform true value array for subsequent calculations, such as Figure 2 As shown, (a) is the captured monitor screen image, (b) is the blood pressure waveform true value array obtained by the red blood pressure waveform image through the data conversion method, and (c) is the comparison diagram of the monitor image and the blood pressure waveform true value array.
[0069] Step S7, when an accident occurs during the execution of any one of steps S3 to S5, the invasive arterial waveform sampling image obtained in step S2 is input into the universal visual segmentation model, and the image segmentation conditions are input into the universal visual segmentation model, and the red blood pressure waveform image is output, and the red blood pressure waveform image is converted into a blood pressure waveform true value array.
[0070] In some embodiments, step S7 specifically includes the following contents:
[0071] The API interface is used to call the general visual segmentation model, and the image set previously converted from the video stream is cyclically accessed to process the images in batches. In the general visual segmentation model, the SAM segmentation function is called for each image to be processed to segment the red blood pressure waveform part from the image.
[0072] When segmenting the first image, adjust the segmentation parameters of the general visual segmentation model to optimize the segmentation effect; if the segmentation result does not meet the requirements, re-iterate and adjust the segmentation parameters or make manual corrections to ensure that the segmentation parameters remain consistent during batch processing;
[0073] After segmenting each image, the segmentation results are saved to a specified folder. After completing batch segmentation, the output needs to be post-processed, which includes removing noise, smoothing edges, or performing further morphological operations to obtain a red blood pressure waveform image; Figure 3 As shown, (a) is the image of the monitor screen, and (b) is the result of segmentation based on the general visual segmentation model; the red blood pressure waveform image is converted into a blood pressure waveform true value array by a data conversion method;
[0074] The obtained array of true values of the blood pressure waveform is subjected to denoising filtering. In addition to median filtering, wavelet transform denoising can more effectively retain the characteristic points of the waveform and remove non-characteristic noise. First, a median filter is applied to the arterial blood pressure data. The median filter can effectively remove prominent outliers such as spike noise by replacing the value of each sample point with the median of its neighborhood (usually the neighboring points with a window size of 3 to 5) without significantly changing the edges and sharp features of the signal. The wavelet transform is used to perform multi-level decomposition of the data that has been processed by the median filter. A data-based method can be used to dynamically select the threshold to adapt to different signal and noise conditions. The quality of the signal before and after filtering is evaluated, including the improvement of the signal-to-noise ratio, the retention of characteristic points, and any possible signal distortion. Finally, the true value array of the denoised blood pressure waveform is obtained.
[0075] The medical auxiliary monitoring method based on monitor screen waveform photography also includes the following steps:
[0076] The array of true values of blood pressure waveform obtained in the medical auxiliary monitoring method based on the waveform camera of the monitor screen is input into the hemodynamic analysis system in real time to output advanced hemodynamic parameters.
[0077] Here, a compression algorithm is used to reduce the amount of data transmission, and a secure encryption protocol is used to ensure the security of data transmission. The hemodynamic analysis system includes a hemodynamic host computer software, which embeds a blood pressure waveform true value array feature point recognition algorithm for hemodynamic parameter calculation and a hemodynamic parameter calculation model. The host computer software uses a PC as the operating platform, which can receive the pressure waveform true value array in real time and realize the calculation, display and storage of hemodynamic parameters; it realizes the data playback function for later data management.
[0078] Inputting the true value array of the blood pressure waveform into the hemodynamic host computer software in real time and using the feature point recognition algorithm of the true value array of the blood pressure waveform calculated by the hemodynamic parameters to accurately identify the feature points in the true value array of the blood pressure waveform, such as the systolic pressure point, diastolic pressure point and mid-isthmus point, is crucial for accurately calculating the hemodynamic parameters. Figure 4 As shown, first, by performing a differential operation on the collected blood pressure waveform real value array data, the zero-crossing point is found in the first-order differential signal, thereby identifying the systolic pressure point; then, the same differential operation is performed on the data after inverting it to identify the diastolic pressure point; finally, by calculating the curvature of the data between the systolic pressure point and the diastolic pressure point, the point with the maximum curvature is identified as the descending gorge point.
[0079] After obtaining the systolic pressure point, diastolic pressure point and descending isthmus point of a single waveform, a more complex hemodynamic parameter calculation model, such as an individualized model based on human physiological characteristics, is used in combination with a machine learning algorithm to optimize the calculation of cardiac output CO to adapt to the specific conditions of different patients and improve the accuracy and applicability of the calculation. For example, a deep learning algorithm can be used to train patient data collected over a long period of time to learn a more accurate cardiac output prediction model and a stroke volume prediction model, thereby providing each patient with a more personalized and accurate hemodynamic analysis. The characteristic points in the true value array data of the blood pressure waveform are used as the input of the hemodynamic parameter calculation model, and high-level hemodynamic parameters are output, including cardiac output CO and stroke volume SV; among them, the most basic calculation formula for cardiac output CO is as follows:
[0080]
[0081] Among them, R represents peripheral resistance, T represents the time of a cardiac cycle, t represents the time, P represents blood pressure, and HR represents heart rate;
[0082] The calculation formula of stroke volume SV is as follows:
[0083]
[0084] in, represents the time interval of the ejection phase, represents the amount of blood pumped by the heart each time, and C represents the compliance of the artery.
[0085] like Figure 5 As shown, a three-way valve is used to connect the radial artery catheter to a common pressure sensor and a Flotrac sensor, and the Flotrac sensor is connected to a multi-parameter monitor and a Vijet flow monitor, respectively. Multi-parameter optimization and time series analysis are used to conduct in-depth analysis of the waveform, so that more hemodynamic parameters related to the blood circulation state can be extracted, including but not limited to indicators such as vascular compliance and blood viscosity. The data obtained by the Vijet flow monitor is compared with the hemodynamic parameters calculated by extracting invasive blood pressure data from the screen image of the multi-parameter monitor using a mobile terminal camera and extracting the blood pressure waveform using image processing technology, as shown in the figure. Figure 6 and Figure 7 As shown, it is verified that the hemodynamic parameters calculated from the invasive arterial waveform data obtained from the monitor device screen are consistent with the values and change trends measured by the current standard measuring instrument, the Vijet flow monitor.
[0086] The medical auxiliary monitoring method based on monitor screen waveform photography also includes the following steps:
[0087] Advanced hemodynamic parameters obtained from a medical-assisted monitoring method based on waveform photography of the monitor screen are used as input.
[0088] The method in steps S1-S7 is used to collect the true value array of the blood pressure waveform of the patient during the operation. The mean arterial pressure in the Weijie flow record is used as the label. The true value array of the blood pressure waveform 5 minutes, 15 minutes and 30 minutes before the corresponding time of the mean arterial pressure and the advanced hemodynamic parameters obtained in the medical auxiliary monitoring method based on the monitor screen waveform camera constitute three data sets of different time windows to train the prediction model of poor blood pressure. The prediction model of poor blood pressure adopts the ResNet50 network model as the basic framework. The network model is mainly composed of a Stem block and four Stage blocks. The Stem block is mainly composed of a sequentially connected convolutional layer, a batch normalization layer BatchNorm, a ReLU activation function layer and a maximum pooling layer; each Stage block is composed of a Bottleneck nest; Bottleneck is mainly composed of a convolutional block Convolutional Block and an identity block Identity Block, and the number of ConvolutionalBlock and Identity Block in different Stage blocks is different.
[0089] The data sets of three different time windows were trained respectively. After each iteration, the network model needed to undergo back propagation to further optimize the parameters. The cross entropy loss function was selected and the AdamW optimizer was used for optimization. After 100 iterative trainings, the three blood pressure adverse condition prediction models corresponding to the data sets of three different time windows had an accuracy rate of more than 80% in the validation set. Among them, the prediction results in the first 5 minutes were better than those in the first 15 minutes and the first 30 minutes, with an accuracy rate of 88%. This indicates that the prediction in the first 5 minutes of the hypotension attack time window is the most accurate.
[0090] According to the prediction results, it is found that the onset of blood pressure events itself is an adverse event caused by multiple factors, and the results of different factors will eventually be coupled to the blood pressure waveform. However, for factors with fast onset and acute onset (such as diuretics), the prediction accuracy of the short-term time window is better than that of the long-term time window. Deep learning combined with machine learning models can predict 5, 15 and 30 minutes of blood pressure events based on invasive arterial pressure and non-invasive monitoring signals (blood oxygen, respiratory rate, etc.). As expected, the 5-minute prediction model performs better than the 15-minute and 30-minute prediction models, using data fusion technology and intelligent decision support systems to provide doctors with a comprehensive disease analysis and prediction platform. This system can use artificial intelligence algorithms, such as decision trees, random forests or neural networks, to evaluate and predict the patient's immediate and future health status, supporting and assisting doctors to make more accurate and timely treatment decisions.
[0091] Corresponding to the aforementioned embodiment of the medical auxiliary monitoring method based on waveform photography of the monitor screen, the present invention also provides an embodiment of a medical auxiliary monitoring device based on waveform photography of the monitor screen.
[0092] See also Figure 8 The medical auxiliary monitoring device based on monitor screen waveform photography provided in an embodiment of the present invention includes a memory and one or more processors. The memory stores executable code. When the processor executes the executable code, it is used to implement the medical auxiliary monitoring method based on monitor screen waveform photography in the above embodiment.
[0093] The embodiment of the medical auxiliary monitoring device based on the monitor screen waveform camera of the present invention can be applied to any device with data processing capability, and the any device with data processing capability can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capability in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 8 As shown, it is a hardware structure diagram of any device with data processing capability where the medical auxiliary monitoring device based on monitor screen waveform photography of the present invention is located, except Figure 8 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.
[0094] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.
[0095] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.
[0096] An embodiment of the present invention further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the medical auxiliary monitoring method based on monitor screen waveform photography in the above embodiment is implemented.
[0097] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capability. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.
[0098] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.
Claims
1. A medical auxiliary monitoring method based on monitor screen waveform photography, It is characterized in that The following steps are involved: Step S1, recording the monitor screen in real time through the camera on the mobile terminal to obtain the invasive arterial waveform video of the monitor; Step S2, sampling the invasive arterial waveform video of the monitor obtained in step S1 in real time to obtain an invasive arterial waveform sampling image; Step S3, performing tilt correction processing on the invasive arterial waveform sampling image obtained in step S2 to obtain an invasive arterial waveform tilt-corrected image; Step S4, performing scale line detection processing on the invasive arterial waveform tilt-corrected image obtained in step S3 to obtain a scale value for reference of the blood pressure value; Step S5, purifying the red blood pressure waveform portion of the invasive arterial waveform tilt-corrected image obtained in step S3 to obtain a red blood pressure waveform image; Step S6, converting the red blood pressure waveform image obtained in step S5 from a pixel block into a blood pressure waveform relative value array, and converting the blood pressure waveform relative value array into a blood pressure waveform true value array based on the scale value of the blood pressure value reference obtained in step S4; Step S7, when an accident occurs during the execution of any one of the steps S3 to S5, the invasive arterial waveform sampling image obtained in the step S2 is input into the universal visual segmentation large model, and the image segmentation conditions are input into the universal visual segmentation large model, a red blood pressure waveform image is output, and the red blood pressure waveform image is converted into a blood pressure waveform true value array; Inputting the blood pressure waveform true value array into a hemodynamic analysis system in real time; The hemodynamic analysis system includes a hemodynamic host computer software, wherein the hemodynamic host computer software is embedded with a blood pressure waveform true value array feature point recognition algorithm for hemodynamic parameter calculation and a hemodynamic parameter calculation model; The blood pressure waveform true value array feature point recognition algorithm is used to identify feature points in the blood pressure waveform true value array data, and the feature points include systolic pressure point, diastolic pressure point and mid-fall point; The hemodynamic parameter calculation model uses a deep learning algorithm to train patient data collected over a long period of time to learn a cardiac output prediction model and a stroke volume prediction model; the blood pressure waveform true value array feature point recognition algorithm identifies feature points in the blood pressure waveform true value array data as input to the hemodynamic parameter calculation model, and outputs advanced hemodynamic parameters, including cardiac output CO and stroke volume SV.
2. The medical auxiliary monitoring method based on monitor screen waveform photography according to claim 1, It is characterized in that The step S1 is specifically as follows: Fix the camera of the mobile terminal in front of the monitor to collect the invasive arterial waveform video of the monitor; The camera shake is compensated by physically moving the camera components and combined with an anti-shake algorithm to capture two frames of video, estimate the motion between the frames, and correct the camera shake.
3. The medical auxiliary monitoring method based on monitor screen waveform photography according to claim 1, It is characterized in that The step S4 is specifically as follows: The Canny edge detection algorithm is used to extract all horizontal dotted lines in the invasive arterial waveform tilt-corrected image in step S3. Different dotted lines in the invasive arterial waveform tilt-corrected image represent different scale values of blood pressure, thereby determining the scale value of the blood pressure value reference in the invasive arterial waveform tilt-corrected image.
4. The medical auxiliary monitoring method based on monitor screen waveform photography according to claim 1, It is characterized in that The step S5 comprises the following sub-steps: Step S51, taking into account the difference in hue range, a dynamic threshold technology is used to extract the red blood pressure waveform portion, wherein the difference in hue range is manifested in the difference in brightness and color display of the monitor screen; Step S52, identifying and removing non-waveform pixel blocks in the red blood pressure waveform part extracted in step S51 by connected domain analysis technology, thereby retaining only the largest connected domain as the true waveform area to obtain a red blood pressure waveform image.
5. The medical auxiliary monitoring method based on monitor screen waveform photography according to claim 1, It is characterized in that The step S7 is specifically as follows: The API interface is used to call the general visual segmentation model, and the image set previously converted from the video stream is cyclically accessed to process the images in batches. In the general visual segmentation model, the SAM segmentation function is called for each image to be processed to segment the red blood pressure waveform part from the image. When segmenting the first image, adjust the segmentation parameters of the general visual segmentation model to optimize the segmentation effect; if the segmentation result does not meet the requirements, re-iterate and adjust the segmentation parameters or make manual corrections to ensure that the segmentation parameters remain consistent during batch processing; After segmenting each image, the segmentation results are saved in a specified folder. After completing batch segmentation, the output needs to be post-processed, and the post-processing includes removing noise, smoothing edges, or performing further morphological operations to obtain a red blood pressure waveform image; the red blood pressure waveform image is converted through a data conversion method to obtain a blood pressure waveform true value array.
6. The medical auxiliary monitoring method based on monitor screen waveform photography according to claim 1, It is characterized in that The recognition process of the blood pressure waveform true value array feature point recognition algorithm includes: By performing a differential operation on the collected true value array data of the blood pressure waveform, the zero-crossing point is found in the first-order differential signal, thereby identifying the systolic pressure point; the same differential operation is performed after inverting the data to identify the diastolic pressure point; by calculating the curvature of the data between the systolic pressure point and the diastolic pressure point, the point with the maximum curvature is identified as the descending gorge point.
7. The medical auxiliary monitoring method based on monitor screen waveform photography according to claim 1, It is characterized in that The calculation formula of the cardiac output CO is: Wherein R represents peripheral resistance, T represents the time of a cardiac cycle, t represents the time, P represents blood pressure, and HR represents heart rate; the calculation formula of the stroke volume SV is: Where T s represents the time interval of the ejection phase, q in (t) represents the amount of blood pumped by the heart each time, and C represents the compliance of the artery.
8. The medical auxiliary monitoring method based on monitor screen waveform photography according to claim 1, It is characterized in that The following steps are also included: Taking the advanced hemodynamic parameters and the blood pressure waveform true value array as input; Collect the true value array of the blood pressure waveform of the patient during the operation, use the mean arterial pressure in the Vijet flow record as the label, find the true value array of the blood pressure waveform and the advanced hemodynamic parameters 5 minutes, 15 minutes and 30 minutes before the mean arterial pressure corresponding time to form three different time window data sets, and train the prediction model for adverse blood pressure conditions; The adverse blood pressure prediction model adopts the ResNet50 network model; the adverse blood pressure prediction model is trained on data sets of three different time windows respectively, and the network model is optimized through back propagation after each iteration, and the cross entropy loss function is selected.
9. A medical auxiliary monitoring device based on monitor screen waveform photography, comprising a memory and one or more processors, wherein the memory stores executable code, It is characterized in that When the processor executes the executable code, it is used to implement the medical auxiliary monitoring method based on monitor screen waveform photography as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Universal method and device for collecting custody data of custody instruments automatically
CN103678916A
Automatic blood pressure measuring system
CN103908236A
Measuring method for physiological parameter
CN105266772A
Hemodynamics parameter obtaining method and device
CN107296593A
Blood pressure detection method and terminal for extracting pulse wave based on video
CN113288092A
