A method for real-time monitoring of cardiac status based on ultrasound images

By constructing and labeling ultrasound datasets, using deep convolution model and Monte Carlo algorithm, real-time heart status monitoring of ultrasound equipment is achieved, solving the problem that existing equipment cannot effectively analyze weak heartbeat conditions, and providing diastolic for heart contraction Real-time evaluation and audio alarms of intensity have improved monitoring efficiency.

CN117122350BActive Publication Date: 2024-11-19SICHUAN WESIMO MEDICAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311113581.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-08-31
Publication Date
2024-11-19
Estimated Expiration
2043-08-31

AI Technical Summary

Technical Problem

Existing ultrasound equipment and electrocardiogram monitoring equipment cannot effectively provide accurate information on the heart condition when analyzing weak heartbeat conditions, and cannot judge the strength of heart contraction and diastolicity, resulting in doctors needing to observe the cardiac display device for a long time.

Method used

By constructing and annotating clinical ultrasound data sets, using deep convolutional models for training, combining Monte Carlo algorithm and variance method, real-time calculation of cardiac chamber area and audio alarms are achieved, avoiding doctors from observing monitoring equipment for a long time.

Benefits of technology

Real-time monitoring of heart state is realized, the chamber area and volume is calculated through image segmentation data, high volume or bottom volume is output, cardiac audio output is provided, and audio alarm is issued when the frequency or beating threshold is exceeded, improving the real-time heart condition Evaluate and monitor efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117122350B_ABST
    Figure CN117122350B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for real-time monitoring of cardiac status based on ultrasonic images, S1: constructing a clinical ultrasonic data set and annotating it; S2: changing the annotated data into a picture; S3: importing deep convolution model reasoning training; S4: obtaining a model parameter file through training, and identifying and running it in an Android system; S5: performing model segmentation reasoning through the Android system to obtain image segmentation data, and performing Monte Carlo algorithm to calculate the area; S6: obtaining cardiac status through variance method, beat threshold and beat frequency evaluation, and starting an audio alarm when the frequency threshold or beat threshold is exceeded. After ultrasonic data is annotated and trained, it is transmitted to the Android system for display, image segmentation data is obtained through segmentation reasoning, the cavity area is calculated in real time, the cavity volume is displayed, a high volume is output when the cavity maximum value is obtained, and a low sound is output when the cavity contraction minimum value is obtained, so as to realize cardiac audio output, thereby avoiding doctors from observing the monitoring equipment for a long time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of cardiac monitoring, and in particular to a method for real-time monitoring of cardiac status based on ultrasonic images. Background Art

[0002] In some cases, if the heart beats very weakly, the electrocardiogram and some traditional monitoring equipment cannot effectively analyze the heart's condition, and cannot provide doctors with the most accurate heart condition.

[0003] With the progress of perioperative medicine, advanced computer and electronic information technologies, deep learning has begun to be widely used in image segmentation processing, and medical image segmentation is an important branch of deep learning image processing. Through the combination of deep learning image segmentation and ultrasound, real-time tracking of left atrial contraction and all-weather detection and tracking of heart conditions can be achieved on wearable ultrasound instruments. Existing ultrasound equipment can only display images of heart movement in real time, and doctors need to observe the electrocardiogram display equipment for a long time. The heart monitoring in the monitor can only reflect changes in heart rhythm, and cannot judge the strength of heart contraction and relaxation. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for real-time monitoring of cardiac status based on ultrasound images.

[0005] The purpose of the present invention is achieved through the following technical solution: A method for real-time monitoring of heart status based on ultrasound images, comprising the following steps:

[0006] S1: Build and annotate clinical ultrasound datasets;

[0007] S2: According to the annotated original data, the annotated data is changed into images, and the data is integrated into the train, test, and mask folders to make the training images correspond to the mask files one by one;

[0008] S3: Import deep convolutional model inference training;

[0009] S4: Get the model parameter file through training, use tensorflow to convert it into the unified model format of the middleware, and recognize and run it on the Android system;

[0010] S5: Perform model segmentation reasoning through the Android system to obtain image segmentation data, and perform a Monte Carlo algorithm to calculate the area of ​​each image segmentation data to achieve a zoom display of a heart chamber;

[0011] S6: The heart condition is obtained through the variance method, beat threshold and beat frequency evaluation. When the frequency threshold or beat threshold is exceeded, the system starts an audio alarm.

[0012] Preferably, step S1 further includes the following steps:

[0013] S11: Collect cardiac short-axis section data video;

[0014] S12: Use the ffmpeg open source package to split the data video into pictures with a frequency of 10 Hz;

[0015] S13: label the short axis area on the image using the third-party segmentation and labeling program labelme;

[0016] S14: The ultrasound short-axis section on the image is segmented through the image convolution segmentation model.

[0017] Preferably, step S3 further includes the following steps:

[0018] S31: data processing, format normalization, obtaining image data and label algorithm display definition;

[0019] S32: By defining and calling the deep convolution model, the input interface and output interface are established at the same time, and the data is input into the training through the input interface;

[0020] S33: Perform forward inference through tensorflow and backpropagate correction training, read image data in batches, and import deep convolutional model for iterative training.

[0021] Preferably, step S5 further includes the following steps:

[0022] S51: Call the tensorflow model to generate random points evenly distributed on the image;

[0023] S52: Check whether each random point is within the pixel range, and count the number of random points within the pixel range;

[0024] S53: Calculate the ratio of irregular image points to all random points in the entire image, and multiply it by the area of ​​the entire image to obtain the area of ​​the segmented image.

[0025] Preferably, in step S52, the pixel values ​​RGB of the area on the picture are 0, 0, 255, and the background is 0, 0, 0.

[0026] The present invention has the following advantages: the present invention annotates and trains ultrasound data and transmits it to an Android system for display, obtains image segmentation data through segmentation reasoning, calculates the cavity area in real time, displays the cavity volume, outputs a high volume when the cavity maximum value is obtained, and outputs a low volume when the cavity contracts to the minimum value, thereby realizing cardiac audio output, thereby avoiding doctors from observing monitoring equipment for a long time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A structural diagram of the method flow for real-time monitoring of cardiac status;

[0028] Figure 2 A structural diagram of the deep convolutional model inference training method process. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0030] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0032] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0033] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use, or the positions or positional relationships commonly understood by those skilled in the art, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0034] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0035] In this embodiment, if Figure 1 As shown, a method for real-time monitoring of cardiac status based on ultrasound imaging comprises the following steps:

[0036] S1: Build and annotate clinical ultrasound datasets;

[0037] S2: According to the annotated original data, the annotated data is changed into images, and the data is integrated into the train, test, and mask folders to make the training images correspond to the mask files one by one;

[0038] S3: Import deep convolutional model inference training;

[0039] S4: The model parameter file is obtained through training, converted into a unified model format of the middleware using tensorflow, and recognized and run on the Android system; specifically, the model format is converted from pb to tflite format to facilitate recognition and operation on the Android system, and the Android interface and the calling model are interconnected so that the Android front end can display the function of two image windows to display the original ultrasound image and the effect after segmentation in real time.

[0040] S5: Perform model segmentation reasoning through the Android system to obtain image segmentation data, and perform a Monte Carlo algorithm to calculate the area of ​​each image segmentation data to achieve a zoom display of a heart chamber;

[0041] S6: The heart condition is obtained through the variance method, beat threshold and beat frequency assessment. When the frequency threshold or beat threshold is exceeded, the system starts an audio alarm. After ultrasound data annotation training, it is transmitted to the Android system for display, and image segmentation data is obtained through segmentation reasoning. The cavity area is calculated in real time, and the cavity volume is displayed. When the maximum value of the cavity is obtained, a high volume is output, and when the minimum value of the cavity contraction is output, a low volume is output, so as to realize the cardiac audio output, thereby avoiding the doctor from observing the monitoring equipment for a long time.

[0042] Furthermore, step S1 further includes the following steps:

[0043] S11: Collect cardiac short-axis section data video;

[0044] S12: Use the ffmpeg open source package to split the data video into pictures with a frequency of tens of Hz;

[0045] S13: label the short axis area on the image using the third-party segmentation and labeling program labelme;

[0046] S14: The ultrasonic short-axis section on the image is segmented through the image convolution segmentation model. Specifically, the collected cardiac short-axis section data video removes the patient's personal information, and the video is segmented into images with a frequency of tens of Hz, such as 10 Hz, and saved in png format. The image size is uniformly adjusted to 800*600 pixels. When annotating, the python environment is used to run and deploy, and the short-axis area on the image is annotated through labelme to make it a closed circle. The annotation information is uniformly saved. In step S14, the model adopts up and down sampling and direct connection methods of convolution stacking. The convolution stacking method can extract detailed information of different images in each layer, and the direct connection method can effectively reduce the gradient explosion of the model.

[0047] Furthermore, step S3 further includes the following steps:

[0048] S31: data processing, format normalization, obtaining image data and label algorithm display definition;

[0049] S32: By defining and calling the deep convolution model, the input interface and output interface are established at the same time, and the data is input into the training through the input interface;

[0050] S33: Perform forward inference through tensorflow and backpropagate correction training, read image data in batches, and import deep convolutional model for iterative training.

[0051] In this embodiment, step S5 further includes the following steps:

[0052] S51: Call the tensorflow model to generate random points evenly distributed on the image;

[0053] S52: Check whether each random point is within the pixel range, and count the number of random points within the pixel range; preferably, in step S52, the pixel value RGB of the area on the picture is 0, 0, 255, and the background is 0, 0, 0.

[0054] S53: Calculate the proportion of irregular image points to all random points in the entire image, and multiply by the area of ​​the entire image to obtain the area of ​​the segmented image. Specifically, the calculation principle is to obtain the area size of the segmented image after 2 to 3 seconds of reasoning, and include multiple unconnected areas of the segmentation in the calculation. By calculating the number of pixels at the unconnected point positions, several largest segmented images are clustered in a distance clustering manner, and the discontinuous surfaces within a certain range are aggregated into statistical values ​​to calculate the area. The largest area is used as the maximum area of ​​the heart chamber to release, and the smallest area is used as the minimum area of ​​the heart contraction. Real-time output, high volume is output when the maximum value of the chamber is reached, and low volume is output when the minimum value of the chamber contraction is reached. A short audio is output at a frequency of 10 to 25HZ. The ultrasonic sampling frequency here is tens of ms, such as 10ms, to achieve the output of the beat cycle audio. When the frequency threshold or the beat threshold is exceeded, the system starts the audio high-frequency alarm, and the real-time cardiac audio output and alarm of the heart's atrioventricular contraction are achieved, thereby avoiding the doctor from observing the monitoring equipment for a long time.

[0055] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for real-time monitoring of cardiac status based on ultrasound imaging, characterized in that: The following steps are involved: S1: Build and annotate clinical ultrasound datasets; S2: According to the annotated original data, the annotated data is changed into images, and the data is integrated into the train, test, and mask folders to make the training images correspond to the mask files one by one; S3: Import deep convolutional model inference training; S4: Get the model parameter file through training, use tensorflow to convert it into the unified model format of the middleware, and recognize and run it on the Android system; S5: Perform model segmentation reasoning through the Android system to obtain image segmentation data, and perform a Monte Carlo algorithm to calculate the area of ​​each image segmentation data to achieve a zoom display of a heart chamber; S6: The heart condition is obtained by variance method, beat threshold and beat frequency evaluation. When the frequency threshold or beat threshold is exceeded, the system starts audio alarm; The step S1 further includes the following steps: S11: Collect cardiac short-axis section data video; S12: Use the ffmpeg open source package to split the data video into pictures with a frequency of 10 Hz; S13: label the short axis area on the image using the third-party segmentation and labeling program labelme; S14: The ultrasound short-axis section on the image is segmented through the image convolution segmentation model. The model adopts up-down sampling and direct connection in a convolutional stacking manner; The step S3 further includes the following steps: S31: data processing, format normalization, obtaining image data and label algorithm display definition; S32: By defining and calling the deep convolution model, the input interface and output interface are established at the same time, and the data is input into the training through the input interface; S33: Perform forward reasoning through tensorflow, and perform backpropagation correction training, read image data in batches, and import deep convolutional model for iterative training; The step S5 further includes the following steps: S51: Call the tensorflow model to generate random points evenly distributed on the image; S52: Check whether each random point is within the pixel range, and count the number of random points within the pixel range; S53: Calculate the ratio of irregular image points to all random points in the whole image, and multiply it by the area of ​​the whole image to obtain the area of ​​the segmented image; The calculation principle is to obtain the area size of the segmented image after 2 to 3 seconds of inference, and include multiple unconnected areas of the segmentation into the calculation. By calculating the number of pixels at the unconnected point positions, the largest segmented image is clustered according to the distance, and the discontinuous surfaces within the set range are aggregated into the statistical value to calculate the area. The largest area is used as the maximum area of ​​the heart chamber when it is released, and the smallest area is used as the minimum area of ​​the heart contraction.

2. The method for real-time monitoring of cardiac status based on ultrasound imaging according to claim 1, characterized in that: In step S52, the pixel values ​​RGB of the area on the picture are 0, 0, 255, and the background is 0, 0, 0.