Method for identifying time phase of heart and method and device for measuring heart parameters
By identifying the opening and closing state of the valve in the cardiac ultrasound image and automatically identifying the time phase with neural network model, the problems of low efficiency and large error in manual recognition of the time phase in the prior art are solved, and efficient and accurate measurement of cardiac parameters is achieved.
Patent Information
- Application Number
- CN202411777771.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-03-14
AI Technical Summary
In the prior art, the time phase efficiency of relying on manual recognition of cardiac ultrasound images is low and prone to time phase error recognition, resulting in an increase in the measurement error of cardiac parameters.
By acquiring the ultrasound image sequence of the heart, identifying the opening and closing state of the valve in each frame of ultrasound images, using neural network models to automatically identify the time phase images of the end-diastolic and end-systolic stages, and then measuring cardiac parameters.
The time phase of automatic recognition of cardiac ultrasound images is realized, which improves recognition efficiency, reduces the error rate of manual recognition, and simplifies the measurement process of cardiac parameters.
Smart Images
Figure CN119625422A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of medical image processing, and in particular to a cardiac time phase recognition method, a cardiac parameter measurement method and a device. Background Art
[0002] In a typical cardiac ultrasound examination, the doctor needs to adjust the probe position to multiple standard cardiac sections, stabilize the probe to save the ultrasound video and perform some specified measurements at specific time phases. Figure 1 As shown in the figure, taking the apical four-chamber view as an example, after obtaining a two-second ultrasound video, the doctor needs to manually drag the video progress bar to select the specified phase, namely, end-diastolic (ED) and end-systolic (ES) according to clinical experience. At the end of diastolic phase, the doctor needs to judge the position and range of each chamber by himself, and manually pull a straight line to measure the transverse diameter, middle transverse diameter and long diameter of the right ventricle base; at the end of systolic phase, measure the transverse diameter and long diameter of the left atrium, and the transverse diameter and long diameter of the right atrium; at the end of systolic phase and end of diastolic phase, the right ventricular contour needs to be depicted to calculate the right ventricular area change rate. In the measurement process of the apical four-chamber view, the doctor needs to perform at least two phase identifications, 7 linear measurements and two contour measurements. Under other standard sections, the required operation steps are roughly the same.
[0003] It can be seen that this method, which completely relies on the doctor's identification of phases and manual measurement, requires the doctor to repeatedly drag the progress bar to confirm the phase and draw straight lines to measure the inner diameter and draw contours to measure the area. In actual practice, this will lead to problems such as long cardiac ultrasound examinations and low efficiency. At the same time, the measurement results are easily affected by problems such as high noise in ultrasound images and unclear key structures. When the scanning doctor lacks clinical experience and anatomical expertise, it is easy to misidentify the phase and misestimate the cavity contour and boundary, resulting in increased measurement errors. Summary of the invention
[0004] The technical problem to be solved by the present disclosure is to overcome the defects of the prior art that reliance on manual phase identification is not only inefficient but also prone to phase misidentification, and to provide a cardiac time phase identification method, cardiac parameter measurement method and device.
[0005] The present invention solves the above technical problems through the following technical solutions:
[0006] In a first aspect, a method for identifying a cardiac time phase is provided, comprising:
[0007] Acquire a sequence of ultrasonic images of the heart; the sequence of ultrasonic images comprises multiple frames of ultrasonic images;
[0008] Identify the open and closed states of the valves contained in the heart in each frame of ultrasound image;
[0009] According to the opening and closing state of the valve in each frame of ultrasound image, a phase image is identified from the multiple frames of ultrasound image; the phase image includes an end-diastolic image and / or an end-systolic image.
[0010] Optionally, identifying the open and closed states of valves included in the heart in each frame of ultrasound image includes:
[0011] The ultrasound image is input into a state detection model, and the opening and closing state of the valve in each frame of the ultrasound image is determined according to the state detection model; the state detection model is obtained by training a first neural network based on a first ultrasound image sample, and the first ultrasound image sample is marked with the opening and closing state of the valve.
[0012] Optionally, identifying the open and closed states of valves included in the heart in each frame of ultrasound image includes:
[0013] Inputting the ultrasound image into a key point detection model, and determining the key points of the valve in the ultrasound image according to the key point detection model; the key point detection model is obtained by training a second neural network according to a second ultrasound image sample, and the second ultrasound image sample is annotated with the key points of the valve;
[0014] The area where the valve is located in the ultrasound image is determined according to the key points, and the area where the valve is located is input into a binary classification model, and the opening and closing state of the valve is determined according to the binary classification model; the binary classification model is obtained by training a third neural network based on a third ultrasound image sample, and the third ultrasound image sample is marked with the opening and closing state of the valve.
[0015] Optionally, the key point detection model includes a first encoding layer, a heat map decoding layer and a key point output layer;
[0016] Inputting the ultrasound image into a key point detection model, and determining key points of the valve in the ultrasound image according to the key point detection model, comprising:
[0017] The ultrasound image is input into the first coding layer so that the first coding layer extracts the first coding features related to the key points of the valve from the ultrasound image, the heat map decoding layer generates a key point heat map according to the first coding features, and the key point output layer outputs the key points of the valve according to the key point heat map.
[0018] Optionally, an ultrasound image whose recognition result of the open / closed state is a closed state and whose recognition result of the next frame is an open state is determined as the end-systolic image;
[0019] The ultrasonic image whose recognition result of the opening and closing state is the closed state and whose recognition result of the previous frame is the open state is determined as the end-diastolic image.
[0020] In a second aspect, a method for measuring a cardiac parameter is provided, comprising:
[0021] Identify a time phase image in a sequence of ultrasound images of the heart according to the time phase identification method described in any one of the first aspects;
[0022] Identifying cavity contours and parameter measurement key points in the phase image;
[0023] The cardiac parameters are calculated based on the chamber contour and the parameter measurement key points.
[0024] Optionally, identifying the cavity contour and parameter measurement key points in the phase image includes:
[0025] The phase image is input into a measurement model, so that a second coding layer included in the measurement model extracts a second coding feature related to the cavity contour and the parameter measurement key point from the phase image, a segmentation decoding layer included in the measurement model determines the cavity contour according to the second coding feature, and a key point detection decoding layer included in the measurement model determines the parameter measurement key point according to the second coding feature;
[0026] Optionally, the cardiac parameter includes a ventricular area change rate; the ventricular area change rate is determined according to a ventricular contour identified by the end-diastolic image and a ventricular contour identified by the end-systolic image;
[0027] And / or, the cardiac parameter includes the long diameter of the ventricle; the parameter measurement key points include the first midpoint of the tricuspid valve of the heart and the apex key point of the ventricle in the end-diastolic image; the long diameter of the ventricle is determined based on the first midpoint and the apex key point; the first midpoint is determined based on the left and right root key points of the tricuspid valve in the end-diastolic image;
[0028] And / or, the cardiac parameters include the transverse diameter of the right ventricle base; the transverse diameter of the right ventricle base is determined according to a line segment in the right ventricle contour base area that is parallel to a line connecting left and right root key points of the tricuspid valve and passes through the midpoint of the ventricular long diameter;
[0029] And / or, the cardiac parameters include the mid-ventricular transverse diameter; the mid-ventricular transverse diameter is determined based on a parallel line connecting the left and right root key points of the tricuspid valve in the mid-region of the ventricular contour;
[0030] And / or, the cardiac parameter includes the long diameter of the atrium; the key points for parameter measurement include the second midpoint of the tricuspid valve and the atrial apex in the end-systolic image; the long diameter of the atrium is determined based on the second midpoint and the atrial apex; the second midpoint is determined based on the left and right root key points of the tricuspid valve in the end-systolic image;
[0031] And / or, the cardiac parameters include atrial transverse diameter; the atrial transverse diameter is determined based on a line segment in the atrial contour that passes through the atrial centroid and is perpendicular to the atrial long diameter.
[0032] In a third aspect, a cardiac time phase identification device is provided, comprising:
[0033] An acquisition module, used for acquiring a sequence of ultrasonic images of the heart; the sequence of ultrasonic images comprises multiple frames of ultrasonic images;
[0034] A state recognition module, used to recognize the open and closed states of the valves contained in the heart in each frame of ultrasound image;
[0035] The first phase recognition module is used to recognize phase images from the multiple frames of ultrasound images according to the opening and closing status of the valve in each frame of ultrasound image; the phase images include end-diastolic images and / or end-systolic images.
[0036] In a fourth aspect, a method for measuring cardiac parameters is provided, comprising:
[0037] A second time phase identification module, configured to identify a time phase image in the ultrasound image sequence of the heart according to the time phase identification method described in any one of the first aspects;
[0038] A key point recognition module, used to recognize the cavity contour and parameter measurement key points in the phase image;
[0039] A measurement module is used to calculate the cardiac parameters according to the chamber contour and the parameter measurement key points.
[0040] In a fifth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, wherein the processor implements any one of the methods in the first aspect when executing the computer program.
[0041] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.
[0042] The positive progress of the present disclosure is that the present disclosure automatically identifies the end-diastolic image and the end-systolic image in the ultrasound image sequence by using the opening and closing state of the valve, without manual identification, with high phase identification efficiency and the ability to reduce the error rate of manual identification. In addition, the phase identification of this embodiment does not need to rely on the electrocardiogram or the heart thickness change curve, occupies fewer resources and is more efficient. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 A flowchart of a cardiac time phase identification method provided by an exemplary embodiment of the present disclosure;
[0044] Figure 2 A schematic diagram of an application scenario of a key point detection model used in a cardiac time phase recognition method provided by an exemplary embodiment of the present disclosure;
[0045] Figure 3 A schematic diagram of an application scenario of a binary classification model adopted in a cardiac time phase recognition method provided by an exemplary embodiment of the present disclosure;
[0046] Figure 4 A flowchart of a method for measuring cardiac parameters provided by an exemplary embodiment of the present disclosure;
[0047] Figure 5 A flowchart of another cardiac parameter measurement method provided by an exemplary embodiment of the present disclosure;
[0048] Figure 6 A schematic diagram of an application scenario of a measurement model used in a method for measuring cardiac parameters provided by an exemplary embodiment of the present disclosure;
[0049] Figure 7 The present invention provides a schematic structural diagram of an electronic device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION
[0050] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.
[0051] Prefixes such as "first" and "second" are used in the embodiments of the present disclosure only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present disclosure does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitation should be constituted due to the use of such prefixes. In addition, in the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.
[0052] In the embodiments of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information (such as ultrasound image sequences) involved are in compliance with the relevant laws and regulations and do not violate public order and good morals.
[0053] Figure 1 A flowchart of a cardiac time phase identification method provided by an exemplary embodiment of the present disclosure, the time phase identification method comprising the following steps:
[0054] Step 101: Acquire a sequence of ultrasound images of the heart.
[0055] The ultrasound image sequence includes multiple frames of ultrasound images, and the length of the ultrasound image sequence, that is, the number of ultrasound images included in the ultrasound image sequence, can be set according to actual conditions. It can be understood that in order to effectively identify the end-diastolic image and the end-systolic image, the ultrasound image sequence includes at least one ultrasound image acquired during a cardiac cycle.
[0056] The ultrasound image may be any section image that can identify the opening and closing state of the valve, for example, the ultrasound image is an apical four-chamber heart section, an apical two-chamber heart section, a parasternal left ventricular long-axis section, etc.
[0057] Step 102: Identify the open and closed states of the valves contained in the heart in each frame of the ultrasound image.
[0058] The valves include: a mitral valve located between the left atrium and the left ventricle of the heart, and / or a tricuspid valve located between the right atrium and the right ventricle.
[0059] In this embodiment, a binary classification state of an open state or a closed state is used to represent the valve state in the ultrasound image.
[0060] Step 103 : identifying a phase image from multiple frames of ultrasound images according to the opening and closing state of the valve in each frame of ultrasound image; the phase image includes an end-diastolic image and / or an end-systolic image.
[0061] Generally, during a cardiac cycle, the valve will continue to open (diastole) and then continue to close (systole), so the end-diastolic image and end-systolic image can be determined based on the valve status.
[0062] In one embodiment, an ultrasound image whose recognition result of the open / closed state is the closed state and whose recognition result of the next frame is the open state is determined as an end-systolic image. An ultrasound image whose recognition result of the open / closed state is the closed state and whose recognition result of the previous frame is the open state is determined as an end-diastolic image. That is, the frame in which the valve at the end of the diastolic period changes from opening to closing is determined as the end-diastolic image (ED frame image), and the frame in which the valve at the end of the systolic period changes from closing to opening is determined as the end-systolic image (ES frame image).
[0063] For example, an ultrasound image sequence includes 10 frames of ultrasound images, "1" indicates that the valve state in the ultrasound image is closed, and "0" indicates that the valve state in the ultrasound image is open. The valve states of the 10 frames of ultrasound images are 0001111000 respectively. The 4th frame of ultrasound image is the end-diastolic image, and the 7th frame of ultrasound image is the end-systolic image.
[0064] Since the recognition of the valve status does not rely on other frames of ultrasound images, each frame of ultrasound images received can be recognized in real time, and the open and closed status of the valve in each frame of ultrasound images can be determined and cached. When it is necessary to find the ED and ES frames in the current ultrasound image sequence, the recognition results of the valve status of each cached ultrasound image frame are analyzed, and the ED and ES frame numbers can be immediately given, which is accurate and efficient.
[0065] In this embodiment, the end-diastolic image and the end-systolic image in the ultrasound image sequence are automatically identified by using the opening and closing state of the valve, without manual identification, and the phase identification efficiency is high, and the error rate of manual identification can be reduced. In addition, the phase identification of this embodiment does not need to rely on the electrocardiogram or the heart thickness change curve, and occupies fewer resources and is more efficient.
[0066] In one embodiment, the open and closed state of the valve is identified by a deep learning intelligent algorithm. Step 102 includes: inputting the ultrasound image into a state detection model, and determining the open and closed state of the valve in each frame of the ultrasound image according to the state detection model.
[0067] The state detection model is obtained by training the first neural network according to the first ultrasound image sample, and the first ultrasound image sample is marked with the open and closed state of the valve. The output result of the state detection model is a binary classification result. The network architecture of the first neural network can be selected according to actual conditions, for example, a convolutional neural network is used.
[0068] The ultrasound image input to the state detection model may be an original image (complete ultrasound image) acquired by a scanning device, or an image after preprocessing the original image. The preprocessing may include, but is not limited to, identifying and cropping the valve region in the original image, and performing image enhancement processing on the original image.
[0069] In this embodiment, the opening and closing state of the valve is identified by using a pre-trained state detection model, which can improve the efficiency and accuracy of valve state identification.
[0070] In one embodiment, the open and closed state of the valve is identified based on a key point detection model and a binary classification model. Step 102 includes: inputting the ultrasound image into the key point detection model, and determining the key points of the valve in the ultrasound image according to the key point detection model. Determining the area where the valve is located in the ultrasound image according to the key points, and inputting the area where the valve is located into the binary classification model, and determining the open and closed state of the valve according to the binary classification model. The key points of the valve may include, but are not limited to, the root position points at both ends (left and right) of the valve and the middle position points.
[0071] The key point detection model is obtained by training the second neural network according to the second ultrasound image sample, the second ultrasound image sample is annotated with the key points of the valve, and the second ultrasound image sample is the acquired complete ultrasound image.
[0072] The binary classification model is obtained by training the third neural network according to the third ultrasound image sample, and the third ultrasound image sample is marked with the opening and closing state of the valve. The third ultrasound image sample can be a valve region image obtained by cutting the second ultrasound image sample according to the valve position.
[0073] In this embodiment, the area where the valve is located in the ultrasound image is first identified through a key point detection model, and then the area where the valve is located is input into a binary classification model to identify the valve state, thereby eliminating the interference of the non-valvular area on the area where the valve is located, thereby improving the accuracy of valve state identification.
[0074] In one embodiment, the key point detection model includes a first encoding layer, a heat map decoding layer and a key point output layer. The first encoding layer is used to extract a first encoding feature related to the key points of the valve from the ultrasound image. The heat map decoding layer is used to generate a key point heat map based on the first encoding feature. The key point output layer is used to output the key points of the valve according to the key point heat map.
[0075] See also Figure 2 , inputting the ultrasound image into the key point detection model, and determining the key points of the valve in the ultrasound image according to the key point detection model specifically includes: inputting the ultrasound image into the first coding layer, so that the first coding layer extracts the first coding features related to the key points of the valve from the ultrasound image, the heat map decoding layer generates a key point heat map according to the first coding features, and the key point output layer outputs the key points of the valve according to the key point heat map. Figure 2 The left root key point A1, the right root key point A3, and the midpoint A2 of the mitral valve are taken as examples for explanation. In actual use, the positions and the number of the key points can be set according to the actual situation.
[0076] In this embodiment, the key point detection model combines the encoding layer and the heat map decoding layer, which can more accurately locate the key points, improve the accuracy of key point detection, and have better robustness.
[0077] In one embodiment, the network architecture of the binary classification model adopts ConvNeXt-Tiny, that is, the third neural network is ConvNeXt-Tiny. The loss function of the binary classification model adopts cross entropy loss, and the AdamW optimizer is used in the model training process.
[0078] See also Figure 3 For each frame of ultrasound image in the ultrasound image sequence, the area where the valve is located in the ultrasound image determined according to the key points is cropped and input into the binary classification model, so that each frame of ultrasound image can be divided into the valve open state stage and the valve closed state stage in turn. According to the position of the intersection of each stage, the ED and ES frame positions in the ultrasound image sequence can be determined.
[0079] The present disclosure also provides a method for measuring cardiac parameters. Figure 4 and Figure 5 , the measuring method comprises the following steps:
[0080] Step 401: Identify a phase image in a sequence of cardiac ultrasound images.
[0081] In step 401, a time phase image in a cardiac ultrasound image sequence is identified according to the time phase identification method provided by any of the above embodiments. The time phase image includes an end-diastolic image and / or an end-systolic image.
[0082] Step 402: Identify the cavity contour and parameter measurement key points in the phase image.
[0083] The chamber contour is the contour of each chamber of the heart, including the left atrium contour, the right atrium contour, the left ventricle contour and the right ventricle contour. The parameter measurement key points are determined according to the heart parameters to be measured, such as the key points of the left and right roots of the mitral valve, the key points of the left and right roots of the tricuspid valve and the key points of the right ventricle apex.
[0084] Step 403: Calculate cardiac parameters based on the chamber contour and parameter measurement key points.
[0085] The cardiac parameters may include, but are not limited to, at least one of the following: right ventricular base transverse diameter, right ventricular mid-transverse diameter, right ventricular long diameter, left atrial transverse diameter and long diameter, right atrial transverse diameter and long diameter, right ventricular area change rate, etc.
[0086] In this embodiment, fully automatic measurement of cardiac parameters is achieved. There is no need for the doctor to manually drag the progress bar to select ED and ES frames and manually pull line segments and contours to measure cardiac parameters. The heart's phase image (key frame) recognition and cardiac parameter measurement can be automatically completed, which greatly improves the efficiency of ultrasound examination and the objectivity of the measurement results.
[0087] In one embodiment, cardiac parameters are annotated in the phase image and the measurement results are displayed for easy viewing by doctors / medical staff.
[0088] In one embodiment, the cardiac parameter includes a ventricular area change rate; the ventricular area change rate is determined based on a ventricular contour identified from an end-diastolic image and a ventricular contour identified from an end-systolic image.
[0089] Taking the right ventricular area change rate as an example, the calculation formula is as follows:
[0090] ;
[0091] in, is the area of the right ventricle in the end-diastolic image, is the area of the right ventricle in the end-systolic image.
[0092] The calculation method of the left ventricular area change rate is similar to that of the right ventricular area change rate, which will not be described again here.
[0093] In one embodiment, the cardiac parameters include the long diameter of the ventricle; the parameter measurement key points include the first midpoint of the tricuspid valve and the apex key point of the ventricle in the end-diastolic image; the long diameter of the ventricle is determined based on the first midpoint and the apex key point. The first midpoint of the tricuspid valve is determined based on the left and right root key points of the tricuspid valve in the ED frame, and the first midpoint is also the midpoint of the line connecting the left and right root key points of the tricuspid valve in the ED frame.
[0094] Taking the calculation of the long diameter of the right ventricle as an example, in the ED frame, a line is drawn between the first midpoint of the tricuspid valve and the key point of the right ventricular apex to obtain the long diameter of the right ventricle. The calculation method of the long diameter of the left ventricle is similar to that of the long diameter of the right ventricle, and will not be repeated here.
[0095] In one embodiment, the cardiac parameters include the right ventricular base transverse diameter; the right ventricular base transverse diameter is determined according to the longest parallel line connecting the left and right root key points of the tricuspid valve in the right ventricular contour base area.
[0096] Specifically, a parallel line connecting two key points of the tricuspid valve (left and right root key points) is drawn in the right ventricular contour base area of the ED frame, and the longest parallel line is taken to determine the transverse diameter of the right ventricular base.
[0097] In one embodiment, the cardiac parameters include the mid-ventricular transverse diameter; the mid-ventricular transverse diameter is determined based on a line segment in the mid-region of the ventricular contour that is parallel to a line connecting left and right tricuspid valve root key points and passes through the midpoint of the ventricular long diameter.
[0098] Taking the right ventricular mid-diameter as an example, a line segment passing through the midpoint of the ventricular long diameter and parallel to the line connecting the left and right root key points of the tricuspid valve is drawn in the middle area of the right ventricular contour of the ED frame. The right ventricular mid-diameter can be determined based on this line segment. The calculation method of the left ventricular mid-diameter is similar to that of the right ventricular mid-diameter, which will not be repeated here.
[0099] In one embodiment, the cardiac parameters include the atrial long diameter; the parameter measurement key points include the second midpoint of the tricuspid valve and the atrial apex in the end-systolic image; the atrial long diameter is determined based on the second midpoint and the atrial apex; wherein the second midpoint of the tricuspid valve is determined based on the left and right root key points of the tricuspid valve in the ES frame, and the second midpoint is also the midpoint of the line connecting the left and right root key points of the tricuspid valve in the ES frame.
[0100] Taking the long diameter of the right atrium as an example, in the ES frame, the long diameter of the right atrium can be obtained by connecting the second midpoint with the top of the right atrium. The calculation method of the long diameter of the left atrium is similar to that of the long diameter of the right atrium, which will not be repeated here.
[0101] In one embodiment, the cardiac parameter includes a transverse diameter of the atrium; the transverse diameter of the atrium is determined according to a line segment in the contour of the atrium that passes through the centroid of the atrium and is perpendicular to the long diameter of the atrium.
[0102] Taking the right atrium transverse diameter as an example, a long perpendicular line is drawn through the atrial centroid in the middle area of the right atrium in the ES frame, and the right atrium transverse diameter is determined based on the line segment intersecting with the right atrium contour. The calculation method of the left atrium transverse diameter is similar to that of the right atrium transverse diameter, which will not be repeated here.
[0103] In one embodiment, in step 402, the measurement model identifies the cavity contour and parameter measurement key points in the phase image. The measurement model includes a second coding layer, a segmentation decoding layer, and a key point detection decoding layer. The second coding layer is used to extract a second coding feature related to the cavity contour and the parameter measurement key points. The segmentation decoding layer is used to determine the cavity contour according to the second coding feature. The key point detection decoding layer is used to determine the parameter measurement key points according to the second coding feature.
[0104] In this embodiment, the measurement results of cardiac parameters on ED and ES frames can be obtained fully automatically. The doctor only needs to confirm the results without any other unnecessary operations, which greatly simplifies the cardiac scanning process and improves the doctor's scanning efficiency and the objectivity of the measurement results.
[0105] See also Figure 6Step 402 specifically includes: inputting the phase image into the measurement model, extracting the second coding features related to the cavity contour and the parameter measurement key points from the phase image by the second coding layer included in the measurement model, determining the cavity contour according to the second coding features by the segmentation decoding layer included in the measurement model, and determining the parameter measurement key points according to the second coding features by the key point detection decoding layer included in the measurement model. By post-processing the cavity contour output by the segmentation decoding layer and the parameter measurement key points output by the key point detection decoding layer, the cardiac parameters can be obtained.
[0106] In this embodiment, a deep learning algorithm is used to segment and extract contours of the heart chambers and detect key points, which can improve the efficiency and accuracy of chamber contours and parameter measurement key points.
[0107] Corresponding to the aforementioned embodiments of the method for identifying cardiac time phase and the method for measuring cardiac parameters, the present disclosure also provides embodiments of a device for identifying cardiac time phase and a device for measuring cardiac parameters.
[0108] The present disclosure provides a cardiac time phase identification device, comprising:
[0109] An acquisition module, used for acquiring a sequence of ultrasonic images of the heart; the sequence of ultrasonic images comprises multiple frames of ultrasonic images;
[0110] A state recognition module, used to recognize the open and closed states of the valves contained in the heart in each frame of ultrasound image;
[0111] The first phase recognition module is used to recognize phase images from the multiple frames of ultrasound images according to the opening and closing status of the valve in each frame of ultrasound image; the phase images include end-diastolic images and / or end-systolic images.
[0112] Optionally, the first time phase identification module is specifically used for:
[0113] The ultrasound image is input into a state detection model, and the opening and closing state of the valve in each frame of the ultrasound image is determined according to the state detection model; the state detection model is obtained by training a first neural network based on a first ultrasound image sample, and the first ultrasound image sample is marked with the opening and closing state of the valve.
[0114] Optionally, the first time phase identification module is specifically used for:
[0115] Inputting the ultrasound image into a key point detection model, and determining the key points of the valve in the ultrasound image according to the key point detection model; the key point detection model is obtained by training a second neural network according to a second ultrasound image sample, and the second ultrasound image sample is annotated with the key points of the valve;
[0116] The area where the valve is located in the ultrasound image is determined according to the key points, and the area where the valve is located is input into a binary classification model, and the opening and closing state of the valve is determined according to the binary classification model; the binary classification model is obtained by training a third neural network based on a third ultrasound image sample, and the third ultrasound image sample is marked with the opening and closing state of the valve.
[0117] Optionally, the key point detection model includes a first encoding layer, a heat map decoding layer and a key point output layer;
[0118] When the ultrasound image is input into a key point detection model and key points of the valve in the ultrasound image are determined according to the key point detection model, the first time phase recognition module is specifically used to:
[0119] The ultrasound image is input into the first coding layer so that the first coding layer extracts the first coding features related to the key points of the valve from the ultrasound image, the heat map decoding layer generates a key point heat map according to the first coding features, and the key point output layer outputs the key points of the valve according to the key point heat map.
[0120] Optionally, an ultrasound image whose recognition result of the open / closed state is a closed state and whose recognition result of the next frame is an open state is determined as the end-systolic image;
[0121] The ultrasonic image whose recognition result of the opening and closing state is the closed state and whose recognition result of the previous frame is the open state is determined as the end-diastolic image.
[0122] The present disclosure also provides a device for measuring cardiac parameters, including:
[0123] A second time phase identification module, configured to identify a time phase image in the ultrasound image sequence of the heart according to the time phase identification method provided by any of the above embodiments;
[0124] A key point recognition module, used to recognize the cavity contour and parameter measurement key points in the phase image;
[0125] A measurement module is used to calculate the cardiac parameters according to the chamber contour and the parameter measurement key points.
[0126] Optionally, the key point identification module is specifically used for:
[0127] The phase image is input into a measurement model, so that a second coding layer included in the measurement model extracts a second coding feature related to the cavity contour and the parameter measurement key point from the phase image, a segmentation decoding layer included in the measurement model determines the cavity contour according to the second coding feature, and a key point detection decoding layer included in the measurement model determines the parameter measurement key point according to the second coding feature;
[0128] Optionally, the cardiac parameter includes a ventricular area change rate; the ventricular area change rate is determined according to a ventricular contour identified by the end-diastolic image and a ventricular contour identified by the end-systolic image;
[0129] And / or, the cardiac parameters include the long diameter of the ventricle; the parameter measurement key points include the first midpoint of the tricuspid valve of the heart and the apex key point of the ventricle in the end-diastolic image; the long diameter of the ventricle is determined based on the first midpoint and the apex key point; the first midpoint is determined based on the left and right root key points of the tricuspid valve in the end-diastolic image, and the first midpoint is also the midpoint of the consecutive left and right root key points of the tricuspid valve in the end-diastolic image;
[0130] And / or, the cardiac parameters include the transverse diameter of the right ventricle base; the transverse diameter of the right ventricle base is determined according to a line segment in the right ventricle contour base area that is parallel to a line connecting left and right root key points of the tricuspid valve and passes through the midpoint of the ventricular long diameter;
[0131] And / or, the cardiac parameters include the mid-ventricular transverse diameter; the mid-ventricular transverse diameter is determined based on a parallel line connecting the left and right root key points of the tricuspid valve in the mid-region of the ventricular contour;
[0132] And / or, the cardiac parameter includes the long diameter of the atrium; the key points for parameter measurement include the second midpoint of the tricuspid valve and the atrial apex in the end-systolic image; the long diameter of the atrium is determined based on the second midpoint and the atrial apex; the second midpoint is determined based on the left and right root key points of the tricuspid valve in the end-systolic image;
[0133] And / or, the cardiac parameters include atrial transverse diameter; the atrial transverse diameter is determined based on a line segment in the atrial contour that passes through the atrial centroid and is perpendicular to the atrial long diameter.
[0134] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only illustrative, wherein the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution.
[0135] Figure 7 This is a schematic diagram of the structure of an electronic device showing an example embodiment of the present disclosure, the electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor, and the processor implements the method described in any of the above embodiments when executing the computer program. Figure 7 The electronic device 70 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0136] like Figure 7 As shown, the electronic device 70 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 70 may include, but are not limited to: at least one processor 71, at least one memory 72, and a bus 73 connecting different system components (including the memory 72 and the processor 71).
[0137] The bus 73 includes a data bus, an address bus, and a control bus.
[0138] The memory 72 may include a volatile memory, such as a random access memory (RAM) 721 and / or a cache memory 722 , and may further include a read only memory (ROM) 723 .
[0139] The memory 72 may also include a program tool 725 (or utility) having a set (at least one) of program modules 724, such program modules 724 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.
[0140] The processor 71 executes various functional applications and data processing by running the computer program stored in the memory 72, such as the method provided in any of the above embodiments.
[0141] The electronic device 70 can also communicate with one or more external devices 74 (e.g., keyboards, pointing devices, etc.). This communication can be carried out through an input / output (I / O) interface 75. In addition, the electronic device 70 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 76. As shown in the figure, the network adapter 76 communicates with other modules of the electronic device 70 through a bus 73. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 70, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.
[0142] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.
[0143] The embodiments of the present disclosure further provide a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method provided in any of the above embodiments is implemented.
[0144] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.
[0145] The present disclosure also provides a computer program product, including a computer program, which implements any of the above methods when executed by a processor.
[0146] Among them, the program code for executing the computer program product of the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.
[0147] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.
Claims
1. A method for identifying a cardiac time phase, characterized in that: include: Acquire a sequence of ultrasonic images of the heart; the sequence of ultrasonic images comprises multiple frames of ultrasonic images; Identify the open and closed states of the valves contained in the heart in each frame of ultrasound image; According to the opening and closing state of the valve in each frame of ultrasound image, a phase image is identified from the multiple frames of ultrasound image; the phase image includes an end-diastolic image and / or an end-systolic image.
2. The cardiac time phase identification method according to claim 1, characterized in that: Identify the open and closed states of the heart valves in each frame of ultrasound image, including: The ultrasound image is input into a state detection model, and the opening and closing state of the valve in each frame of the ultrasound image is determined according to the state detection model; the state detection model is obtained by training a first neural network based on a first ultrasound image sample, and the first ultrasound image sample is marked with the opening and closing state of the valve.
3. The cardiac time phase identification method according to claim 1, characterized in that: Identify the open and closed states of the heart valves in each frame of ultrasound image, including: Inputting the ultrasound image into a key point detection model, and determining the key points of the valve in the ultrasound image according to the key point detection model; the key point detection model is obtained by training a second neural network according to a second ultrasound image sample, and the second ultrasound image sample is annotated with the key points of the valve; The area where the valve is located in the ultrasound image is determined according to the key points, and the area where the valve is located is input into a binary classification model, and the opening and closing state of the valve is determined according to the binary classification model; the binary classification model is obtained by training a third neural network based on a third ultrasound image sample, and the third ultrasound image sample is marked with the opening and closing state of the valve.
4. The cardiac time phase identification method according to claim 3, characterized in that: The key point detection model includes a first encoding layer, a heat map decoding layer and a key point output layer; Inputting the ultrasound image into a key point detection model, and determining key points of the valve in the ultrasound image according to the key point detection model, comprising: The ultrasound image is input into the first coding layer so that the first coding layer extracts the first coding features related to the key points of the valve from the ultrasound image, the heat map decoding layer generates a key point heat map according to the first coding features, and the key point output layer outputs the key points of the valve according to the key point heat map.
5. The cardiac time phase identification method according to any one of claims 1 to 4, characterized in that: Determine the ultrasound image whose recognition result of the opening and closing state is the closed state and whose recognition result of the next frame is the open state as the end-systolic image; The ultrasonic image whose recognition result of the opening and closing state is the closed state and whose recognition result of the previous frame is the open state is determined as the end-diastolic image.
6. A method for measuring cardiac parameters, characterized in that: include: Identify a time phase image in a sequence of ultrasound images of the heart according to the time phase identification method according to any one of claims 1 to 5; identifying cavity contours and parameter measurement key points in the phase image; The cardiac parameters are calculated based on the chamber contour and the parameter measurement key points.
7. The method for measuring cardiac parameters according to claim 6, characterized in that: Identifying the cavity contour and parameter measurement key points in the phase image, including: The phase image is input into the measurement model, and the second coding layer included in the measurement model extracts second coding features related to the cavity contour and the parameter measurement key points from the phase image, the segmentation decoding layer included in the measurement model determines the cavity contour according to the second coding features, and the key point detection decoding layer included in the measurement model determines the parameter measurement key points according to the second coding features.
8. The method for measuring cardiac parameters according to claim 6, characterized in that: The cardiac parameter includes a ventricular area change rate; the ventricular area change rate is determined according to the ventricular contour identified by the end-diastolic image and the ventricular contour identified by the end-systolic image; And / or, the cardiac parameter includes the long diameter of the ventricle; the parameter measurement key points include the first midpoint of the tricuspid valve of the heart and the apex key point of the ventricle in the end-diastolic image; the long diameter of the ventricle is determined based on the first midpoint and the apex key point; the first midpoint is determined based on the left and right root key points of the tricuspid valve in the end-diastolic image; And / or, the cardiac parameters include the transverse diameter of the right ventricle base; the transverse diameter of the right ventricle base is determined according to the longest parallel line connecting the left and right root key points of the tricuspid valve in the right ventricle contour base area; And / or, the cardiac parameters include the mid-ventricular transverse diameter; the mid-ventricular transverse diameter is determined according to a line segment in the mid-region of the ventricular contour that is parallel to a line connecting the left and right root key points of the tricuspid valve and passes through the midpoint of the ventricular long diameter; And / or, the cardiac parameter includes the long diameter of the atrium; the key points for parameter measurement include the second midpoint of the tricuspid valve and the atrial apex in the end-systolic image; the long diameter of the atrium is determined based on the second midpoint and the atrial apex; the second midpoint is determined based on the left and right root key points of the tricuspid valve in the end-systolic image; And / or, the cardiac parameters include atrial transverse diameter; the atrial transverse diameter is determined based on a line segment in the atrial contour that passes through the atrial centroid and is perpendicular to the atrial long diameter.
9. A cardiac time phase recognition device, characterized in that: include: An acquisition module, used for acquiring a sequence of ultrasonic images of the heart; the sequence of ultrasonic images comprises multiple frames of ultrasonic images; A state recognition module, used to recognize the open and closed states of the valves contained in the heart in each frame of ultrasound image; The first phase recognition module is used to recognize phase images from the multiple frames of ultrasound images according to the opening and closing status of the valve in each frame of ultrasound image; the phase images include end-diastolic images and / or end-systolic images.
10. A device for measuring cardiac parameters, characterized in that: include: A second time phase identification module, configured to identify a time phase image in the ultrasound image sequence of the heart according to the time phase identification method according to any one of claims 1 to 5; A key point recognition module, used to recognize the cavity contour and parameter measurement key points in the phase image; A measurement module is used to calculate the cardiac parameters according to the chamber contour and the parameter measurement key points.