Heart structure parameter acquisition method, heart video processing method, device and equipment

Abstract

The invention relates to the technical field of heart structure parameter measurement, and provides a heart structure parameter acquisition method, and a heart video processing method, device and equipment, which can measure relatively accurate heart structure parameters. The method comprises the following steps: obtaining a left ventricle surrounding key point set, a left ventricle cardiac cavity structure boundary and a left ventricle surrounding tissue structure boundary according to a cardiac image map of a left ventricle long-axis section; according to the left chamber long axis direction determined by the key point set around the left chamber, an inner diameter measuring line of the left chamber is obtained; and determining the inner diameter of the left ventricle and the thickness of the tissue around the left ventricle according to the intersection information of the inner diameter measuring line and the heart cavity structure boundary and the tissue structure boundary.

Description

Technical Field

[0001] The present application relates to the technical field of cardiac structural parameter measurement, and in particular to a method and device for obtaining cardiac structural parameters, a method and device for processing cardiac videos, computer equipment, storage media, and computer program products. Background Art

[0002] The parasternal left ventricular long axis section (also called PLAX section, the full name of PLAX in English is Parasternal Long Axis) is a more important cardiac observation section, which provides a comprehensive view of the heart structure. The cardiac images of this section may include the left ventricle (also called left ventricle), ventricular septum, left ventricular posterior wall, left atrium, mitral valve, aorta and aortic root, etc. Many more important cardiac structure parameters can be measured by the cardiac images of this section, such as left ventricular internal diameter, ventricular septum thickness, and left ventricular posterior wall thickness.

[0003] Current solutions generally only perform key point detection, which has great limitations in actual application and makes it difficult to measure more accurate cardiac structural parameters. Summary of the invention

[0004] Based on this, it is necessary to provide a method and device for obtaining cardiac structure parameters, a method and device for processing cardiac videos, computer equipment, storage medium and computer program product to address the above technical problems.

[0005] The present application provides a method for acquiring cardiac structural parameters, the method comprising:

[0006] According to the cardiac image of the long axis section of the left ventricle, the key point set around the left ventricle, the cardiac cavity structure boundary of the left ventricle and the tissue structure boundary of the tissue around the left ventricle are obtained;

[0007] Obtaining a measurement line of the inner diameter of the left ventricle according to the long axis direction of the left ventricle determined by the key point set around the left ventricle;

[0008] The inner diameter of the left ventricle and the thickness of the tissue surrounding the left ventricle are determined according to the intersection information of the inner diameter measurement line with the cardiac cavity structure boundary and the tissue structure boundary.

[0009] In one embodiment, determining the long axis direction of the left ventricle according to the left ventricle peripheral key point set includes:

[0010] Acquire the upper left key point of the left ventricle, the upper right key point of the left ventricle, the lower left key point of the left ventricle, and the lower right key point of the left ventricle;

[0011] The long axis direction of the left ventricle is determined according to the upper left key point, the upper right key point, the lower left key point and the lower right key point.

[0012] In one embodiment, the upper left key point, the upper right key point, the lower left key point and the lower right key point determine the long axis direction of the left ventricle, including:

[0013] Determine a first direction according to the slope of a line segment between the upper left key point and the upper right key point, and the slope of a line segment between the lower left key point and the lower right key point;

[0014] Determine a second direction according to the midpoint of the line segment between the upper left key point and the lower left key point, and the midpoint of the line segment between the upper right key point and the lower right key point;

[0015] determining a target direction according to at least one of the first direction and the second direction;

[0016] The long axis direction of the left ventricle is determined according to the target direction.

[0017] In one embodiment, determining the long axis direction of the left ventricle according to the target direction includes:

[0018] According to the tissue structure boundary, a center line of the tissue region toward the apex of the heart is obtained;

[0019] The long axis direction of the left ventricle is determined according to the target direction and the center line of the tissue region.

[0020] In one embodiment, obtaining the inner diameter measurement line of the left ventricle according to the long axis direction of the left ventricle determined by the key point set around the left ventricle includes:

[0021] Determine the inner diameter measurement positioning point;

[0022] The inner diameter measurement line of the left ventricle is obtained according to the perpendicular direction between the inner diameter measurement positioning point and the long axis direction of the left ventricle.

[0023] In one embodiment, determining the inner diameter measurement positioning point includes:

[0024] According to the attachment points of the left ventricular posterior wall and the posterior leaflet of the mitral valve where the key points around the left ventricle are concentrated, the positioning points for measuring the inner diameter are determined;

[0025] Alternatively, when the tissue surrounding the left ventricle includes the interventricular septum, the inner diameter measurement positioning point is determined according to the center line of the interventricular septum region obtained from the tissue structure boundary of the interventricular septum and the upper right key point of the left ventricle;

[0026] Alternatively, the inner diameter measurement positioning point is determined according to the motion area of ​​the mitral valve anterior leaflet key points concentrated around the left ventricle.

[0027] In one embodiment, determining the thickness of the tissue surrounding the left ventricle according to the intersection information of the inner diameter measurement line and the tissue structure boundary includes:

[0028] According to the tissue structure boundary, a center line of the tissue region toward the apex of the heart is obtained;

[0029] Obtaining an intersection line according to the intersection point of the inner diameter measurement line and the tissue structure boundary;

[0030] Obtaining a thickness measurement positioning point according to the intersection point of the tissue area center line and the intersection line;

[0031] The thickness of the left ventricular periventricular tissue is determined according to the thickness measurement positioning point and the tissue structure boundary.

[0032] In one embodiment, determining the thickness of the tissue surrounding the left ventricle according to the thickness measurement positioning point and the tissue structure boundary includes:

[0033] Determine a thickness measurement line among several lines passing through the thickness measurement positioning point; the thickness measurement line is perpendicular to a target tangent line, and the target tangent line is a line passing through the thickness measurement positioning point and tangent to the center line of the tissue region;

[0034] The thickness of the tissue surrounding the left ventricle is determined according to the intersection information between the thickness measurement line and the tissue structure boundary.

[0035] In one embodiment, the method further includes: obtaining a curve of changes in the left ventricular internal diameter based on the inner diameter of the left ventricle in each frame of the cardiac image in the cardiac video of the long axis section of the left ventricle; obtaining a curve of changes in the thickness of the tissue surrounding the left ventricle based on the thickness of the tissue surrounding the left ventricle in each frame of the cardiac image in the cardiac video; and displaying the curve of changes in the left ventricular internal diameter and the curve of changes in the thickness of the tissue surrounding the left ventricle.

[0036] The present application provides a device for acquiring cardiac structural parameters, the device comprising:

[0037] An image processing module, for obtaining a set of key points around the left ventricle, a cardiac cavity structure boundary of the left ventricle, and a tissue structure boundary of tissues around the left ventricle according to a cardiac image of a long axis section of the left ventricle;

[0038] An inner diameter measurement line determination module, used to obtain the inner diameter measurement line of the left ventricle according to the left ventricle long axis direction determined by the left ventricle peripheral key point set;

[0039] The structural parameter acquisition module is used to determine the inner diameter of the left ventricle and the thickness of the tissue surrounding the left ventricle according to the intersection information of the inner diameter measurement line with the cardiac cavity structure boundary and the tissue structure boundary.

[0040] The present application provides a method for processing a cardiac video, the method comprising:

[0041] Obtain a cardiac video of the long-axis view of the left ventricle;

[0042] According to the above cardiac structure parameter acquisition method, a plurality of cardiac image frames of the cardiac video are processed to obtain the inner diameter sequence of the left ventricle;

[0043] Determine at least one of an initial frame of the end-systole and an initial frame of the end-diastole of the heart according to the inner diameter sequence of the left ventricle;

[0044] According to the plurality of cardiac image frames of the cardiac video, at least one of a first distance sequence and a second distance sequence is obtained; the first distance is the distance between a key point of the anterior leaflet of the mitral valve and a key point of the posterior leaflet of the mitral valve; the second distance is the distance between a right upper key point of the left ventricle and a key point of the anterior leaflet of the mitral valve;

[0045] At least one of an end-systolic target frame and an end-diastolic target frame is obtained according to at least one of the first distance sequence and the second distance sequence and at least one of an end-systolic initial frame and an end-diastolic initial frame.

[0046] In one embodiment, the method further comprises:

[0047] Obtaining a target frame measurement result according to at least one cardiac structural parameter of the end-systolic target frame and the end-diastolic target frame and displaying the target frame measurement result;

[0048] and / or,

[0049] The marking information for the end-systolic target frame and the end-diastolic target frame is displayed on the curve of the change of the left ventricular internal diameter and the curve of the change of the left ventricular peripheral tissue thickness.

[0050] In one embodiment, obtaining at least one of the first distance sequence and the second distance sequence according to a plurality of cardiac image frames of the cardiac video includes:

[0051] If the end-systolic initial frame is determined according to the left ventricular inner diameter sequence, key point extraction is performed on multiple cardiac image frames located before and after the end-systolic initial frame to obtain at least one of a first distance sequence and a second distance sequence;

[0052] If the end-diastolic initial frame is determined according to the left ventricular inner diameter sequence, key point extraction processing is performed on multiple frames of cardiac images located before and after the end-diastolic initial frame to obtain at least one of a first distance sequence and a second distance sequence.

[0053] The present application provides a device for processing a cardiac video, the device comprising:

[0054] A video acquisition module, used for acquiring a cardiac video of a left ventricular long axis section;

[0055] A left ventricular internal diameter sequence acquisition module is used to process a plurality of cardiac image frames of the cardiac video according to the cardiac structure parameter acquisition method to obtain the left ventricular internal diameter sequence;

[0056] An initial value determination module, used to determine at least one of an end-systolic initial frame and an end-diastolic initial frame of the heart according to the inner diameter sequence of the left ventricle;

[0057] A distance sequence acquisition module is used to obtain at least one of a first distance sequence and a second distance sequence according to a plurality of cardiac image frames of the cardiac video; the first distance is the distance between a key point of the anterior leaflet of the mitral valve and a key point of the posterior leaflet of the mitral valve; the second distance is the distance between a right upper key point of the left ventricle and a key point of the anterior leaflet of the mitral valve;

[0058] The target frame determination module is used to obtain at least one of the end-systolic target frame and the end-diastolic target frame according to at least one of the first distance sequence and the second distance sequence, and at least one of the end-systolic initial frame and the end-diastolic initial frame.

[0059] The present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the above method.

[0060] The present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to execute the above method.

[0061] The present application provides a computer program product, on which a computer program is stored, and the computer program is executed by a processor to perform the above method.

[0062] The present application can obtain the cardiac cavity structure boundary of the left ventricle and the tissue structure boundary of the left ventricular surrounding tissue (such as the ventricular septum or the left ventricular posterior wall) by segmenting the cardiac image image of the left ventricular long axis section, and can obtain the left ventricular surrounding key point set by performing key point detection on the cardiac image image of the left ventricular long axis section; then, according to the left ventricular long axis direction determined by the left ventricular surrounding key point set, the left ventricular inner diameter measurement line is obtained, and combined with the intersection information of the left ventricular inner diameter measurement line with the cardiac cavity structure boundary and the tissue structure boundary, more accurate left ventricular inner diameter and thickness of the left ventricular surrounding tissue (the thickness of the left ventricular surrounding tissue such as the ventricular septum thickness or the left ventricular posterior wall thickness) can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 It is a flowchart of a method for acquiring cardiac structural parameters in one embodiment;

[0065] Figure 2 is a schematic diagram of the structure of a heart under a PLAX section in one embodiment;

[0066] Figure 3 A cardiac image under a PLAX section obtained by ultrasound technology in one embodiment;

[0067] Figure 4 is a schematic diagram of the long axis direction of the left ventricle in one embodiment;

[0068] Figure 5 A schematic diagram of an orientation coordinate system of a cardiac image and key points around the left ventricle in one embodiment;

[0069] Figure 6 is a schematic diagram of the center line of the left ventricular posterior wall area and the target direction in one embodiment;

[0070] Figure 7 is a schematic diagram of the center line of the ventricular septum region and the target direction in one embodiment;

[0071] FIG8 (a) is a characteristic diagram for determining a left ventricular posterior wall thickness measurement line in one embodiment;

[0072] FIG8( b ) is a characteristic diagram for determining the interventricular septum thickness measurement line in one embodiment;

[0073] Fig. 9 is a schematic diagram of change curves of various cardiac structural parameters in one embodiment;

[0074] Fig.10 is a flow chart of a method for processing a cardiac video in one embodiment;

[0075] Fig.11 is a schematic diagram of change curves of various cardiac structural parameters in another embodiment;

[0076] Fig.12 It is a flowchart of a method for acquiring cardiac structural parameters in another embodiment;

[0077] Fig.13A schematic diagram of key points marked in an embodiment;

[0078] Fig.14 A schematic diagram of a measurement effect obtained by applying the method provided by the present application in one embodiment;

[0079] Fig.15 It is a structural block diagram of a device for acquiring cardiac structural parameters in one embodiment;

[0080] Fig.16 is a structural block diagram of a device for processing cardiac video in one embodiment;

[0081] Fig.17 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0083] The cardiac structural parameter acquisition method provided in the present application can measure multiple structural parameters on a PLAX section. The method can be executed by a computer device, including: Figure 1 Steps shown.

[0084] Step S101, obtaining a set of left ventricular periphery key points, a left ventricular cardiac cavity structure boundary, and a tissue structure boundary of left ventricular periphery tissue according to a cardiac image of a left ventricular long axis section;

[0085] The cardiac structure in the long-axis section of the left ventricle beside the sternum is shown below. Figure 2 As shown, it specifically includes the right ventricle, left ventricle, left atrium, left ventricular posterior wall, ventricular septum, ascending aorta, right coronary sinus, non-coronary sinus, anterior leaflet of mitral valve, posterior leaflet of mitral valve and descending aorta.

[0086] Ultrasound technology can be used to obtain cardiac images of the long axis section of the left ventricle, such as Figure 3 shown.

[0087] By segmenting the cardiac image, the cardiac cavity structure boundaries of the left ventricle, the cardiac cavity structure boundaries of the left atrium, the cardiac cavity structure boundaries of the right ventricle, and the structural boundaries of the relevant tissues under the long axis section of the left ventricle can be obtained. The structural boundaries of the relevant tissues under the long axis section of the left ventricle include the structural boundaries of the ventricular septum, the structural boundaries of the left ventricular posterior wall, the structural boundaries of the ascending aorta, the structural boundaries of the right coronary sinus, the structural boundaries of the non-coronary sinus, the structural boundaries of the anterior leaflet of the mitral valve, the structural boundaries of the posterior leaflet of the mitral valve, and the structural boundaries of the descending aorta.

[0088] Among them, the left ventricle belongs to a cavity, and the cavity is formed by a boundary, which can be called the cardiac cavity structural boundary of the left ventricle. The tissue around the cardiac cavity structural boundary of the left ventricle can be called the left ventricular periventricular tissue. The tissue structural boundary of the ventricular septum and the tissue structural boundary of the left ventricular posterior wall belong to the tissue structural boundary of the left ventricular periventricular tissue.

[0089] In some embodiments, based on the key structural information of the left ventricular long axis section, points around the left ventricular cavity can be pre-set as key points and marked on the cardiac imaging training sample of the left ventricular long axis section; in addition, the structural boundaries involved can also be marked on the cardiac imaging training sample. Model training is performed based on the marked cardiac imaging training samples to obtain a hybrid multi-output model, which includes a segmentation branch and a key point detection branch.

[0090] In the inference stage, after the cardiac image is input into the hybrid multi-output model, the cardiac segmentation map output by the segmentation branch can be obtained. The cardiac cavity structure boundary of the left ventricle and the tissue structure boundary of the tissue around the left ventricle can be determined based on the cardiac segmentation map. The key point detection result output by the key point detection branch can be obtained, and the left ventricular periventricular key point set can be obtained based on the key point detection result.

[0091] Step S102, obtaining the left ventricular inner diameter measurement line according to the left ventricular long axis direction determined by the left ventricular peripheral key point set.

[0092] The long axis of the left ventricle is the line connecting the midpoint of the mitral valve and the apex. The direction of the long axis of the left ventricle on the cardiac image can be called the long axis direction of the left ventricle. The midpoint of the mitral valve is the midpoint of the junction of the anterior leaflet of the mitral valve and the posterior leaflet of the mitral valve, that is, the center point where the anterior leaflet of the mitral valve and the posterior leaflet of the mitral valve meet when closed.

[0093] Due to the limited observation range, some cardiac images will not include the apex structure. Therefore, in order to determine the long axis direction of the left ventricle, this application uses the left ventricular periphery key point set. The left ventricular periphery key point set includes key points distributed around the left ventricular cavity. With the help of the left ventricular periphery key point set, the long axis direction of the left ventricle can be determined. In some scenes, the determined long axis direction of the left ventricle is as follows: Figure 4 shown.

[0094] The left ventricular inner diameter measurement line is generally perpendicular to the left ventricular long axis direction. Therefore, the slope of the left ventricular inner diameter measurement line can be determined based on the left ventricular long axis direction. The left ventricular inner diameter measurement line can be determined based on the slope and the left ventricular inner diameter measurement positioning point.

[0095] Step S103, determining the inner diameter of the left ventricle and the thickness of the tissue surrounding the left ventricle according to the intersection information of the inner diameter measurement line with the cardiac cavity structure boundary and the tissue structure boundary.

[0096] After determining the inner diameter measurement line of the left ventricle on the cardiac image, the two intersection points of the inner diameter measurement line and the cardiac cavity structure boundary of the left ventricle on the cardiac image can be determined, and the inner diameter of the left ventricle is obtained based on the distance between the two intersection points, which corresponds to the aforementioned cardiac image.

[0097] If the tissue surrounding the left ventricle includes the left ventricular posterior wall, accordingly, the tissue structure boundary includes the left ventricular posterior wall structure boundary, and the two intersection points of the aforementioned inner diameter measurement line and the left ventricular posterior wall structure boundary can be determined. According to the distance between these two intersection points, the thickness of the left ventricular posterior wall corresponding to the aforementioned cardiac image can be obtained, and the thickness corresponds to the aforementioned cardiac image.

[0098] If the tissue surrounding the left ventricle includes the ventricular septum, correspondingly, the tissue structure boundary includes the ventricular septum structure boundary, and the two intersection points of the aforementioned inner diameter measurement line and the ventricular septum structure boundary can be determined. According to the distance between these two intersection points, the thickness of the ventricular septum corresponding to the aforementioned cardiac image can be obtained, and the thickness corresponds to the aforementioned cardiac image.

[0099] The obtained left ventricular internal diameter, left ventricular posterior wall thickness and interventricular septum thickness are cardiac structural parameters.

[0100] In the above-mentioned method for acquiring cardiac structure parameters, the cardiac cavity structure boundary of the left ventricle and the tissue structure boundary of the tissue surrounding the left ventricle (such as the ventricular septum or the left ventricular posterior wall) can be obtained by segmenting the cardiac image of the long axis section of the left ventricle, and the key point set around the left ventricle can be obtained by performing key point detection on the cardiac image of the long axis section of the left ventricle; then, the left ventricular inner diameter measurement line is obtained according to the left ventricular long axis direction determined by the left ventricular surrounding key point set, and combined with the intersection information of the left ventricular inner diameter measurement line with the cardiac cavity structure boundary and the tissue structure boundary, a more accurate left ventricular inner diameter and the thickness of the tissue surrounding the left ventricle (the thickness of the tissue surrounding the left ventricle such as the ventricular septum thickness or the left ventricular posterior wall thickness) can be obtained.

[0101] In one embodiment, determining the long axis direction of the left ventricle based on a set of key points around the left ventricle includes: acquiring the upper left key point of the left ventricle, the upper right key point of the left ventricle, the lower left key point of the left ventricle, and the lower right key point of the left ventricle; determining the long axis direction of the left ventricle based on the upper left key point, the upper right key point, the lower left key point, and the lower right key point.

[0102] In the scenario where the aforementioned cardiac image is obtained using ultrasound technology, the azimuth coordinate system of the cardiac image can be set according to the propagation direction of the central ultrasound wave of the ultrasound beam. The azimuth coordinate system describes the four directions of the cardiac image: up, down, left, and right. Figure 5 shown.

[0103] The left ventricular peripheral key point set includes the upper left key point of the left ventricle, the upper right key point of the left ventricle, the lower left key point of the left ventricle and the lower right key point of the left ventricle. The orientation concepts involved in these key points, such as "up, down, left, right", are based on the orientation coordinate system of the set cardiac image.

[0104] In some embodiments, reference Figure 5 , the upper left key point of the left ventricle is A, the upper right key point of the left ventricle is B, the lower left key point of the left ventricle is C, and the lower right key point of the left ventricle is D. The upper left key point A of the left ventricle can be the leftmost point inside the ventricular septum, the upper right key point B of the left ventricle is located at the junction of the ventricular septum and the aorta, the lower left key point C of the left ventricle can be the endpoint of the left side of the pericardium, and the lower right key point D can be the endpoint of the right side of the pericardium.

[0105] Since the left upper key point of the left ventricle, the right upper key point of the left ventricle, the left lower key point of the left ventricle and the right lower key point of the left ventricle are distributed around the left ventricular cavity, the long axis direction of the left ventricle can be determined based on these key points.

[0106] In one embodiment, the long axis direction of the left ventricle is determined based on the upper left key point, the upper right key point, the lower left key point and the lower right key point, including: determining the first direction based on the slope of the line segment between the upper left key point and the upper right key point, and the slope of the line segment between the lower left key point and the lower right key point; determining the second direction based on the midpoint of the line segment between the upper left key point and the lower left key point, and the midpoint of the line segment between the upper right key point and the lower right key point; determining the target direction based on at least one of the first direction and the second direction; determining the long axis direction of the left ventricle based on the target direction.

[0107] by Figure 5 The key points are shown as an example:

[0108] Method 1: Based on the line segment between the upper left key point A of the left ventricle and the upper right key point B of the left ventricle, the slope of the line segment in the azimuth coordinate system of the cardiac image can be determined, and this slope is called the first slope; based on the lower left key point C of the left ventricle and the lower right key point D of the left ventricle, the slope of the line segment in the azimuth coordinate system of the cardiac image can be determined, and this slope can be called the second slope; the first slope and the second slope can be averaged or weighted summed, and the obtained slope can be called the first target slope, and the direction with the slope of the first target slope can be used as the first direction.

[0109] After the first direction is determined, the first direction can be directly used as the target direction to obtain the long axis direction of the left ventricle.

[0110] Method 2: Based on the line segment between the upper left key point A of the left ventricle and the lower left key point C of the left ventricle, the midpoint of the line segment can be determined, and the midpoint becomes the first midpoint; based on the line segment between the upper right key point B of the left ventricle and the lower right key point D of the left ventricle, the midpoint of the line segment can be determined, and the midpoint becomes the second midpoint; based on the line segment between the first midpoint and the second midpoint, the slope of the line segment in the azimuth coordinate system of the cardiac image can be determined, and the slope is called the second target slope, and the direction with the slope of the second target slope can be used as the second direction.

[0111] After the second direction is determined, the second direction can be directly used as the target direction to obtain the long axis direction of the left ventricle.

[0112] Method three: After the first direction and the second direction are determined, the first direction and the second direction may be averaged or weighted summed, and the result obtained is used as the target direction, thereby obtaining the left ventricular long axis direction.

[0113] In one embodiment, determining the long axis direction of the left ventricle according to the target direction includes: obtaining a center line of a tissue region toward the apex according to a tissue structure boundary; and determining the long axis direction of the left ventricle according to the target direction and the center line of the tissue region.

[0114] Method 1: If the left ventricular peripheral tissue includes the left ventricular posterior wall, then the tissue structure boundary includes the left ventricular posterior wall structure boundary. After obtaining the left ventricular posterior wall structure boundary, the center line of the left ventricular posterior wall structure boundary can be determined. This center line can be called the left ventricular posterior wall area center line. The determined left ventricular posterior wall area center line is oriented toward the apex, such as Figure 6 shown.

[0115] The center line of the left ventricular posterior wall region and the target direction may be averaged or weighted summed, and the result obtained is used as the left ventricular long axis direction.

[0116] Method 2: If the left ventricular periventricular tissue includes the ventricular septum, the tissue structure boundary includes the ventricular septum structure boundary. After obtaining the ventricular septum structure boundary, the center line of the ventricular septum structure boundary can be determined. The center line can be called the ventricular septum region center line. The determined ventricular septum region center line is oriented toward the apex, such as Figure 7 shown.

[0117] The center line of the ventricular septum region and the target direction may be averaged or weighted summed, and the result obtained is used as the long axis direction of the left ventricle.

[0118] Method three: If the tissue surrounding the left ventricle includes the left ventricular posterior wall and the interventricular septum, the center line of the left ventricular posterior wall area and the center line of the interventricular septum area can be determined. Then, the target directions of the center line of the left ventricular posterior wall area and the center line of the interventricular septum area are averaged or weighted summed, and the result is used as the long axis direction of the left ventricle.

[0119] In one embodiment, the inner diameter measurement line of the left ventricle is obtained according to the left ventricular long axis direction determined by the left ventricular peripheral key point set, including: determining the inner diameter measurement positioning point; and obtaining the inner diameter measurement line of the left ventricle according to the perpendicular direction between the inner diameter measurement positioning point and the left ventricular long axis direction.

[0120] A point can be determined on the aforementioned cardiac image and used as the inner diameter measurement positioning point. A line passing through the inner diameter measurement positioning point and perpendicular to the long axis of the left ventricle can be used as the inner diameter measurement line of the left ventricle.

[0121] After the inner diameter measurement line of the left ventricle is obtained, two intersection points between the inner diameter measurement line and the cardiac cavity structure boundary of the left ventricle can be determined, and the inner diameter of the left ventricle can be obtained according to the distance between the two intersection points.

[0122] In one embodiment, determining the inner diameter measurement positioning point includes: determining the inner diameter measurement positioning point based on the attachment point of the left ventricular posterior wall and the posterior leaflet of the mitral valve in the left ventricular peripheral key point concentration; or, when the left ventricular peripheral tissue includes the ventricular septum, determining the inner diameter measurement positioning point based on the center line of the ventricular septum area obtained from the tissue structure boundary of the ventricular septum and the upper right key point of the left ventricle; or, determining the inner diameter measurement positioning point based on the movement area of ​​the key point of the anterior leaflet of the mitral valve in the left ventricular peripheral key point concentration.

[0123] The inner diameter measurement positioning point can be determined by any of the following methods:

[0124] Method 1: The attachment point of the left ventricular posterior wall and the posterior leaflet of the mitral valve can be determined from the key points around the left ventricle, and the point where the attachment point is offset by a certain distance along the contour of the left ventricular posterior wall toward the apex is determined, and the point is used as the inner diameter measurement positioning point, and the line passing through the inner diameter measurement positioning point and perpendicular to the long axis of the left ventricle is used as the left ventricular inner diameter measurement line. The offset distance can be an empirical value.

[0125] Method 2: When the left ventricular periventricular tissue includes the ventricular septum, the tissue structure boundary includes the ventricular septum structure boundary, and the center line of the ventricular septum structure boundary can be determined. This center line is called the ventricular septum area center line and is oriented toward the apex, such as Figure 7As shown; extract a point close to the upper right key point of the left ventricle on the center line of the ventricular septum region, use the point as the inner diameter measurement positioning point, and use a line passing through the inner diameter measurement positioning point and perpendicular to the long axis of the left ventricle as the inner diameter measurement line of the left ventricle. Extract a point close to the upper right key point of the left ventricle on the center line of the ventricular septum region, use the point as the inner diameter measurement positioning point, which may specifically include the following steps: determine a point where the upper right key point of the left ventricle is offset by a certain distance along the center line of the ventricular septum region toward the apex, and use the point as the inner diameter measurement positioning point; wherein the offset distance may be an empirical value.

[0126] Method three: The cardiac image images adjacent to the aforementioned cardiac image images are used as candidate cardiac image images. For example, the cardiac image images of the first few frames and the cardiac image images of the last few frames of the aforementioned cardiac image images can be used as candidate cardiac image images. The aforementioned cardiac image images can be subjected to key point detection with the help of the key point detection branch included in the pre-constructed hybrid multi-output model to extract the key points of the mitral valve anterior leaflet of the aforementioned cardiac image images; the candidate cardiac image images can be subjected to key point detection with the help of the key point detection branch included in the pre-constructed hybrid multi-output model to extract the key points of the mitral valve anterior leaflet of the candidate cardiac image images. The key points of the mitral valve anterior leaflet of the aforementioned cardiac image images and the key points of the mitral valve anterior leaflet of the candidate cardiac image images are used as the mitral valve anterior leaflet key point set. According to the position of each point in the mitral valve anterior leaflet key point set, the motion area of ​​the mitral valve anterior leaflet key point can be obtained. The mitral valve anterior leaflet key point belongs to a point in the left ventricular periphery key point set. After determining the motion area of ​​the key point of the anterior mitral leaflet, the long axis of the motion area can be determined; in the set of key points of the anterior mitral leaflet, the key point of the anterior mitral leaflet that is far from the long axis and close to the apex is determined, and the key point of the anterior mitral leaflet is used as the inner diameter measurement positioning point. The line passing through the inner diameter measurement positioning point and perpendicular to the long axis direction of the left ventricle is used as the inner diameter measurement line of the left ventricle.

[0127] In one embodiment, the thickness of the tissue surrounding the left ventricle is determined based on the intersection information of the inner diameter measurement line and the tissue structure boundary, including: obtaining the center line of the tissue area toward the apex according to the tissue structure boundary; obtaining the intersection line according to the intersection point of the inner diameter measurement line and the tissue structure boundary; obtaining the thickness measurement positioning point according to the intersection point of the tissue area center line and the intersection line; and determining the thickness of the tissue surrounding the left ventricle based on the thickness measurement positioning point and the tissue structure boundary.

[0128] If the left ventricular surrounding tissue includes the left ventricular posterior wall, correspondingly, the tissue structure boundary includes the left ventricular posterior wall structure boundary, and the center line of the left ventricular posterior wall structure boundary can be determined. The center line can be called the left ventricular posterior wall area center line. The determined left ventricular posterior wall area center line is toward the apex, such as Figure 6 shown.

[0129] After determining the inner diameter measurement line of the left ventricle, two intersection points of the inner diameter measurement line of the left ventricle and the boundary of the left ventricular posterior wall structure can be obtained. These two intersection points can form a line, which is called the intersection line. Determine the intersection point of the intersection line with the center line of the left ventricular posterior wall area, such as point P shown in Figure 8 (a), and use point P as the left ventricular posterior wall thickness measurement positioning point; determine the two intersection points of the thickness measurement positioning point and the boundary of the left ventricular posterior wall structure, and obtain the thickness of the left ventricular posterior wall according to the distance between the two intersection points.

[0130] If the tissue surrounding the left ventricle includes the ventricular septum, the tissue structure boundary includes the ventricular septum structure boundary. After obtaining the ventricular septum structure boundary, the center line of the ventricular septum structure boundary can be determined. The center line can be called the ventricular septum region center line. The determined ventricular septum region center line is oriented toward the apex, such as Figure 7 shown.

[0131] After determining the inner diameter measurement line of the left ventricle, two intersection points of the inner diameter measurement line of the left ventricle and the boundary of the interventricular septum structure can be obtained. These two intersection points can form a line, which is called the intersection line. Determine the intersection point of the intersection line with the center line of the interventricular septum area, such as point O shown in Figure 8 (b), and use point O as the thickness measurement positioning point of the interventricular septum; determine the two intersection points of the thickness measurement positioning point and the boundary of the interventricular septum structure, and obtain the thickness of the interventricular septum according to the distance between the two intersection points.

[0132] In one embodiment, the thickness of the tissue around the left ventricle is determined based on the thickness measurement positioning point and the tissue structure boundary, including: determining a thickness measurement line among several lines passing through the thickness measurement positioning point; the thickness measurement line is perpendicular to a target tangent line, the target tangent line is a line passing through the thickness measurement positioning point and tangent to the center line of the tissue area; and determining the thickness of the tissue around the left ventricle based on the intersection information of the thickness measurement line and the tissue structure boundary.

[0133] If the left ventricular peripheral tissue includes the left ventricular posterior wall, then the center line of the tissue region includes the center line of the left ventricular posterior wall region; using the above example, after point P shown in FIG8 (a) is used as the left ventricular posterior wall thickness measurement positioning point, the line passing through point P and tangent to the center line of the left ventricular posterior wall region is used as the target tangent. Among the lines passing through point P, there is a line perpendicular to the target tangent, and the line perpendicular to the target tangent is used as the left ventricular posterior wall thickness measurement line.

[0134] The thickness of the left ventricular posterior wall can be obtained according to the two intersection points of the left ventricular posterior wall thickness measurement line and the left ventricular posterior wall structure boundary.

[0135] If the tissue around the left ventricle includes the ventricular septum, the center line of the tissue region includes the center line of the ventricular septum region; using the above example, point O shown in FIG8 (b) is used as the location point for measuring the thickness of the ventricular septum, and the line passing through point O and tangent to the center line of the ventricular septum region is used as the target tangent. Among the lines passing through point O, there is one line that is perpendicular to the target tangent, and the line perpendicular to the target tangent is used as the thickness measurement line of the ventricular septum.

[0136] The thickness of the ventricular septum can be obtained according to the two intersection points of the ventricular septum thickness measurement line and the ventricular septum structure boundary.

[0137] In one embodiment, the method for acquiring cardiac structure parameters provided by the present application also includes: obtaining a curve of changes in the left ventricular inner diameter based on the inner diameter of the left ventricle in each frame of the cardiac image in the cardiac video of the left ventricular long axis section; obtaining a curve of changes in the thickness of the tissue surrounding the left ventricle based on the thickness of the tissue surrounding the left ventricle in each frame of the cardiac image in the cardiac video; and displaying the curve of changes in the left ventricular inner diameter and the curve of changes in the thickness of the tissue surrounding the left ventricle.

[0138] In the case where the tissue surrounding the left ventricle includes the ventricular septum and the left ventricular posterior wall, the inner diameter of the left ventricle in each frame of the cardiac image in the cardiac video of the left ventricular long axis section can be obtained according to the above embodiment, thereby obtaining a curve of changes in the inner diameter of the left ventricle; the thickness of the ventricular septum in each frame of the cardiac image in the cardiac video can also be obtained according to the above embodiment, thereby obtaining a curve of changes in the thickness of the ventricular septum tissue; the thickness of the left ventricular posterior wall in each frame of the cardiac image in the cardiac video can also be obtained according to the above embodiment, thereby obtaining a curve of changes in the thickness of the left ventricular posterior wall. The curve of changes in the inner diameter of the left ventricle, the curve of changes in the thickness of the ventricular septum tissue and the curve of changes in the thickness of the left ventricular posterior wall can be displayed, such as Fig. 9 As shown, the horizontal axis represents the video frame and the vertical axis represents the distance.

[0139] The present application also provides a method for processing a cardiac video, which can be executed by a computer device, including: Fig.10 The steps shown are:

[0140] Step S1001, obtaining a cardiac video of the long axis section of the left ventricle;

[0141] Step S1002, according to the above cardiac structure parameter acquisition method, a plurality of cardiac image frames of the cardiac video are processed to obtain a sequence of inner diameters of the left ventricle;

[0142] After obtaining the cardiac video of the long axis section of the left ventricle, key point detection and segmentation can be performed on each frame of the cardiac image in the cardiac video according to the embodiment of the above-mentioned cardiac structure parameter acquisition method to obtain the inner diameter of the left ventricle in each frame of the cardiac image and form a left ventricular inner diameter sequence.

[0143] Step S1003, determining at least one of the initial frame of the end-systole and the initial frame of the end-diastole of the heart according to the inner diameter sequence of the left ventricle.

[0144] The inner diameter of the left ventricle is the smallest at the end of systole and the largest at the end of diastole. Based on this, the minimum inner diameter in the left ventricular inner diameter sequence can be determined, and the frame corresponding to the minimum inner diameter is used as the initial frame at the end of systole. Alternatively, the maximum inner diameter in the left ventricular inner diameter sequence can be determined, and the frame corresponding to the maximum inner diameter is used as the initial frame at the end of diastole.

[0145] Step S1004, obtaining at least one of a first distance sequence and a second distance sequence according to a plurality of cardiac image frames of the cardiac video; the first distance is the distance between a key point of the anterior leaflet of the mitral valve and a key point of the posterior leaflet of the mitral valve; the second distance is the distance between a key point of the upper right of the left ventricle and a key point of the anterior leaflet of the mitral valve;

[0146] After obtaining the cardiac video of the long axis section of the left ventricle, key point detection can be performed on each frame of the cardiac image in the cardiac video to detect the key points of the anterior leaflet of the mitral valve and the key points of the posterior leaflet of the mitral valve in each frame of the cardiac image, and determine the distance between the key points of the anterior leaflet of the mitral valve and the key points of the posterior leaflet of the mitral valve. This distance is called the first distance, so as to obtain the first distance corresponding to each frame of the cardiac image and form a first distance sequence.

[0147] Alternatively, after obtaining a cardiac video of the long axis section of the left ventricle, key point detection can be performed on each frame of the cardiac image in the cardiac video to detect the key point of the anterior leaflet of the mitral valve and the upper right key point of the left ventricle in each frame of the cardiac image, and determine the distance between the key point of the anterior leaflet of the mitral valve and the upper right key point of the left ventricle. This distance is called the second distance, so as to obtain the second distance corresponding to each frame of the cardiac image and form a second distance sequence.

[0148] Step S1005, obtaining at least one of an end-systolic target frame and an end-diastolic target frame according to at least one of the first distance sequence and the second distance sequence, and at least one of an end-systolic initial frame and an end-diastolic initial frame.

[0149] During the diastole of the heart, before the mitral valve closes, the distance between the key point of the anterior leaflet of the mitral valve and the key point of the posterior leaflet of the mitral valve decreases and the decrease is large. In contrast, the distance between the key point of the anterior leaflet of the mitral valve and the upper right key point of the left ventricle will gradually increase. From the systole of the heart to before the opening of the mitral valve, the change in the distance between the key point of the anterior leaflet of the mitral valve and the upper right key point of the left ventricle is small. After the mitral valve opens, the change in the distance between the key point of the anterior leaflet of the mitral valve and the upper right key point of the left ventricle will increase rapidly, and the distance between the key point of the anterior leaflet of the mitral valve and the key point of the posterior leaflet of the mitral valve will gradually decrease.

[0150] If the first distance sequence is obtained, the processing process is as follows:

[0151] After determining the end-systolic initial frame based on the minimum inner diameter in the left ventricular inner diameter sequence, the first distance corresponding to the end-systolic initial frame and the first distances corresponding to multiple nearby frames of the end-systolic initial frame can be obtained based on the first distance sequence. Among these first distances, the minimum first distance is determined, and the frame corresponding to the minimum first distance is used as the end-systolic target frame.

[0152] Alternatively, after determining the end-diastolic initial frame based on the minimum inner diameter in the left ventricular inner diameter sequence, the first distance corresponding to the end-diastolic initial frame and the first distances corresponding to multiple nearby frames of the end-diastolic initial frame can be obtained based on the first distance sequence. Among these first distances, the largest first distance is determined, and the frame corresponding to the largest first distance is used as the end-diastolic target frame.

[0153] If the second distance sequence is obtained, the processing process is as follows:

[0154] After determining the end-systolic initial frame based on the minimum inner diameter in the left ventricular inner diameter sequence, the second distance corresponding to the end-systolic initial frame and the second distances corresponding to multiple nearby frames of the end-systolic initial frame can be obtained based on the second distance sequence. Among these second distances, the largest first distance is determined, and the frame corresponding to the largest second distance is used as the end-systolic target frame.

[0155] Alternatively, after determining the end-diastolic initial frame based on the minimum inner diameter in the left ventricular inner diameter sequence, the second distance corresponding to the end-diastolic initial frame and the second distances corresponding to multiple nearby frames of the end-diastolic initial frame can be obtained based on the second distance sequence. Among these second distances, the minimum second distance is determined, and the frame corresponding to the minimum second distance is used as the end-diastolic target frame.

[0156] In this embodiment, at least one of the end-systolic initial frame and the end-diastolic initial frame is first determined with the help of the inner diameter sequence of the left ventricle, and then at least one of the first distance (i.e., the distance between the key point of the anterior leaflet of the mitral valve and the key point of the posterior leaflet of the mitral valve) and the second distance (i.e., the distance between the key point of the anterior leaflet of the mitral valve and the upper right key point of the left ventricle) is introduced for adjustment to obtain more accurate end-systolic frames and end-diastolic frames.

[0157] In one embodiment, at least one of a first distance sequence and a second distance sequence is obtained based on several frames of cardiac image images of a cardiac video, including: if the initial frame of end-systole is determined based on the inner diameter sequence of the left ventricle, key point extraction processing is performed on multiple frames of cardiac image images located before and after the initial frame of end-systole to obtain at least one of the first distance sequence and the second distance sequence; if the initial frame of end-diastole is determined based on the inner diameter sequence of the left ventricle, key point extraction processing is performed on multiple frames of cardiac image images located before and after the initial frame of end-diastole to obtain at least one of the first distance sequence and the second distance sequence.

[0158] If the initial frame of end-systole is determined based on the inner diameter sequence of the left ventricle, multiple frames of cardiac images located before and after the initial frame of end-systole and adjacent to the initial frame of end-systole can be determined in the cardiac video, and key point extraction processing is performed on these cardiac images and the initial frame of end-systole, and the key points of the anterior leaflet of the mitral valve and the posterior leaflet of the mitral valve in each frame are detected to obtain the first distance corresponding to each frame to form a first distance sequence, or the key points of the anterior leaflet of the mitral valve and the upper right key points of the left ventricle in each frame are detected to obtain the second distance corresponding to each frame to form a second distance sequence.

[0159] If the end-diastolic initial frame is determined based on the inner diameter sequence of the left ventricle, multiple frames of cardiac images located before and after the end-diastolic initial frame and adjacent to the end-diastolic initial frame can be determined in the cardiac video, and key point extraction processing is performed on these cardiac images and the end-diastolic initial frame, and the key points of the anterior leaflet of the mitral valve and the posterior leaflet of the mitral valve in each frame are detected to obtain the first distance corresponding to each frame to form a first distance sequence, or the key points of the anterior leaflet of the mitral valve and the upper right key points of the left ventricle in each frame are detected to obtain the second distance corresponding to each frame to form a second distance sequence.

[0160] In one embodiment, the method for processing cardiac videos provided in the present application also includes: obtaining and displaying target frame measurement results based on at least one of the cardiac structural parameters of the end-systolic target frame and the end-diastolic target frame; and / or, displaying marking information for the end-systolic target frame and the end-diastolic target frame on the change curve of the left ventricular internal diameter and the change curve of the thickness of the tissue surrounding the left ventricle.

[0161] After obtaining the cardiac video of the long axis section of the left ventricle, each frame of the cardiac image of the cardiac video can be processed according to the above-mentioned cardiac structure parameter acquisition method to obtain the inner diameter of the left ventricle, the first distance (i.e., the distance between the key point of the anterior leaflet of the mitral valve and the key point of the posterior leaflet of the mitral valve), and the second distance (i.e., the distance between the upper right key point of the left ventricle and the key point of the anterior leaflet of the mitral valve) of each frame of the cardiac image.

[0162] Exemplarily, based on the inner diameter of the left ventricle in each frame of cardiac image, a curve of change of the inner diameter of the left ventricle can be obtained; based on the first distance of each frame of cardiac image, a curve of change of the first distance can be obtained; based on the second distance of each frame of cardiac image, a curve of change of the second distance can be obtained.

[0163] Reference Fig.11 , Fig.11 The curves of the change of the left ventricular internal diameter, the change of the first distance and the change of the second distance are shown, wherein the horizontal axis represents the video frame and the vertical axis represents the distance. Fig.11 The three change curves shown show the marking information for the end-systolic target frame and the end-diastolic target frame, such as Fig.11 There are 8 vertical lines shown, which are used to mark which frames are the end-systolic target frames and the end-diastolic target frames. Fig.11 The eight vertical lines shown correspond to: ED target frame, ES target frame, ED target frame, ES target frame, ED target frame, ES target frame, ED target frame, ES target frame. ED is the abbreviation of end diastole, and the full name of ED is End Diastolic. ES is the abbreviation of end systole, and the full name of ES is End Systolic.

[0164] Take the cardiac structural parameters including left ventricular internal diameter as an example:

[0165] After determining several end-systolic target frames in the cardiac video, the left ventricular internal diameter of each end-systolic target frame can be obtained, and the left ventricular internal diameter of these end-systolic target frames can be statistically analyzed by methods such as averaging, maximum value, minimum value, etc., so as to obtain the target frame measurement result corresponding to the left ventricular internal diameter, and display the target frame measurement result.

[0166] Alternatively, after determining several end-diastolic target frames in the cardiac video, the left ventricular internal diameter of each end-diastolic target frame can be obtained, and the left ventricular internal diameter of these end-diastolic target frames can be statistically analyzed using statistical methods such as averaging, maximum value, minimum value, etc., so as to obtain the target frame measurement result corresponding to the left ventricular internal diameter, and display the target frame measurement result.

[0167] Since the target frame measurement result corresponds to the left ventricular internal diameter, the target frame measurement result can be displayed on the left ventricular internal diameter change curve. Of course, in other embodiments, the target frame measurement result may not be displayed on the left ventricular internal diameter change curve.

[0168] In order to better understand the above method, an application example of the method provided by the present application is described in detail below.

[0169] According to the real-time ultrasound scanning data, the section classification model constructed by the deep learning algorithm can automatically determine whether the current observation section is a PLAX section. If so, the heart video of a certain length under the PLAX section can be automatically obtained. With the help of the pre-built hybrid multi-output model, each frame of the heart image under the PLAX section is processed. The processing flow of each frame of the heart image includes: Fig.12 Steps shown.

[0170] A hybrid multi-output model can be obtained based on the deep learning method. The hybrid multi-output model can analyze the heart video frame by frame with the help of TensorRT and CUDA code optimization. The annotation scheme of the training samples used to build the hybrid multi-output model includes:

[0171] (1) Annotate the structural boundaries of the ventricular septum, the left ventricular cavity, and the left ventricular posterior wall on the cardiac image training samples;

[0172] (2) Mark five key points on each cardiac image training sample for positioning and auxiliary measurement of the long axis of the left ventricle. The five key points include: left upper key point A of the left ventricle (this key point is located inside the ventricular septum), right upper key point B of the left ventricle (this key point is located at the junction of the ventricular septum and the aorta), left lower key point C of the left ventricle (this key point is located in the pericardium), right lower key point D of the left ventricle (this key point is located in the pericardium), attachment point E of the left ventricular posterior wall and the posterior leaflet of the mitral valve, key point F of the anterior leaflet of the mitral valve, and key point G of the posterior leaflet of the mitral valve, as shown in Figure 1. Fig.13 As shown; wherein, the cusp point of the anterior leaflet of the mitral valve can be used as the key point F of the anterior leaflet of the mitral valve, and the cusp point of the posterior leaflet of the mitral valve can be used as the key point G of the anterior leaflet of the mitral valve.

[0173] The following, combined Fig.12 This section describes the process of processing a single-frame cardiac image using a pre-built hybrid multi-output model:

[0174] The cardiac image map can be input into the hybrid multi-output model to obtain a cardiac segmentation map output by a segmentation branch of the hybrid multi-output model and a key point detection result output by a key point detection branch of the hybrid multi-output model.

[0175] According to the heart segmentation map, the cardiac cavity structural boundary of the left ventricle in the heart image map, the structural boundary of the ventricular septum in the heart image map, and the structural boundary of the left ventricular posterior wall in the heart image map can be extracted.

[0176] According to the key point detection results, A, B, C and D can be obtained, and these key points belong to the left ventricular periphery key point set. According to A, B, C and D, the left ventricular long axis direction can be determined. The specific processing process of this step can refer to the introduction of the above embodiment.

[0177] Determine a line that passes through the inner diameter measurement positioning point and is perpendicular to the long axis of the left ventricle, and use this line as the inner diameter measurement line of the left ventricle.

[0178] The inner diameter of the left ventricle is obtained based on the two intersection points of the inner diameter measurement line of the left ventricle and the boundary of the left ventricular cardiac cavity structure; the thickness of the ventricular septum is obtained based on the two intersection points of the inner diameter measurement line of the left ventricle and the boundary of the interventricular septum structure; the thickness of the left ventricular posterior wall is obtained based on the two intersection points of the inner diameter measurement line of the left ventricle and the boundary of the left ventricular posterior wall structure.

[0179] With the help of Fig.12 The processing flow shown in the figure determines the left ventricular internal diameter, interventricular septum thickness and left ventricular posterior wall thickness of each frame of cardiac image. The measurement effect is as follows: Fig.14 As shown; in addition, the distance between FG points (ie, the first distance) and the distance between BF points (ie, the second distance) of each frame of the cardiac image can also be obtained.

[0180] According to the left ventricular internal diameter of each frame of the cardiac image, the left ventricular internal diameter sequence is obtained. Since the left ventricular internal diameter is the smallest at the end of the heart's systole and the largest at the end of the heart's diastole, the minimum internal diameter in the left ventricular internal diameter sequence can be determined, and the frame corresponding to the minimum internal diameter is used as the initial frame of the end of the systole. The maximum internal diameter in the left ventricular internal diameter sequence can be determined, and the frame corresponding to the maximum internal diameter is used as the initial frame of the end of the diastole.

[0181] In the cardiac video, multiple frames of cardiac images located before and after the initial frame of the end-systole and adjacent to the initial frame of the end-systole are determined. According to the B, F and G points of these cardiac images, the distance between the FG points corresponding to each frame is obtained to form a first distance sequence, and the distance between the BF points corresponding to each frame is obtained to form a second distance sequence.

[0182] In the cardiac video, multiple frames of cardiac images located before and after the initial frame of end-diastole and adjacent to the initial frame of end-diastole are determined. According to the B, F and G points of these cardiac images, the distance between the FG points corresponding to each frame is obtained to form a first distance sequence, and the distance between the BF points corresponding to each frame is obtained to form a second distance sequence.

[0183] According to the first distance sequence, the first distance corresponding to the initial frame of the end-systole and the first distances corresponding to multiple nearby frames of the initial frame of the end-systole are obtained, among these first distances, the smallest first distance is determined, and the frame corresponding to the smallest first distance is used as the target frame of the end-systole; or, according to the second distance sequence, the second distance corresponding to the initial frame of the end-systole and the second distances corresponding to multiple nearby frames of the initial frame of the end-systole are obtained, among these second distances, the largest first distance is determined, and the frame corresponding to the largest second distance is used as the target frame of the end-systole.

[0184] According to the first distance sequence, the first distance corresponding to the initial frame of end diastole and the first distances corresponding to multiple nearby frames of the initial frame of end diastole are obtained, among these first distances, the largest first distance is determined, and the frame corresponding to the largest first distance is used as the end-diastole target frame; or, according to the second distance sequence, the second distance corresponding to the initial frame of end diastole and the second distances corresponding to multiple nearby frames of the initial frame of end diastole are obtained, among these second distances, the smallest second distance is determined, and the frame corresponding to the smallest second distance is used as the end-diastole target frame.

[0185] With the above method, more accurate end-systolic frames and end-diastolic frames can be obtained, and each movement cycle of the heart can be divided. In each movement cycle of the heart, this application example can give the ventricular septum thickness, left ventricular internal diameter and left ventricular posterior wall thickness in the movement cycle, and combined with the measurement results of cardiac structural parameters of other sections, other cardiac indicators such as EF value, left ventricular mass, etc. can be calculated.

[0186] This embodiment can realize automatic extraction of multiple cardiac structural parameters under the PLAX section by analyzing the cardiac video under the section, and can also realize measurement of multiple cardiac structural parameters on the end-systolic target frame and the end-diastolic target frame. The multiple cardiac structural parameters include but are not limited to the ventricular septum thickness, the left ventricular internal diameter and the left ventricular posterior wall thickness; the change curves of various cardiac structural parameters in a single motion cycle can be drawn in the form of charts and graphs, and the reference Fig. 9 and Fig.11 , the changing curves of various cardiac structural parameters are displayed in conjunction with cardiac imaging images, helping doctors to evaluate the heart's state and structural information from a wider time dimension. Under the condition of extracting richer measurement data, doctors can obtain more and more meaningful information, which will be more representative than limited measurement data.

[0187] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0188] Based on the same inventive concept, the embodiment of the present application also provides a cardiac structure parameter acquisition device for implementing the cardiac structure parameter acquisition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more cardiac structure parameter acquisition device embodiments provided below can refer to the limitations of the cardiac structure parameter acquisition method above, and will not be repeated here.

[0189] In one embodiment, Fig.15 As shown, a device for acquiring cardiac structural parameters is provided, comprising:

[0190] The image processing module 1501 is used to obtain a set of key points around the left ventricle, a cardiac cavity structure boundary of the left ventricle, and a tissue structure boundary of tissues around the left ventricle according to the cardiac image of the long axis section of the left ventricle;

[0191] An inner diameter measurement line determination module 1502 is used to obtain the inner diameter measurement line of the left ventricle according to the left ventricle long axis direction determined by the left ventricle peripheral key point set;

[0192] The structural parameter acquisition module 1503 is used to determine the inner diameter of the left ventricle and the thickness of the tissue surrounding the left ventricle according to the intersection information of the inner diameter measurement line with the cardiac cavity structure boundary and the tissue structure boundary.

[0193] In one embodiment, the device also includes a left ventricular long axis direction determination module, which is used to: obtain the upper left key point of the left ventricle, the upper right key point of the left ventricle, the lower left key point of the left ventricle and the lower right key point of the left ventricle; determine the long axis direction of the left ventricle based on the upper left key point, the upper right key point, the lower left key point and the lower right key point.

[0194] In one embodiment, the left ventricular long axis direction determination module is also used to: determine the first direction according to the slope of the line segment between the upper left key point and the upper right key point, and the slope of the line segment between the lower left key point and the lower right key point; determine the second direction according to the midpoint of the line segment between the upper left key point and the lower left key point, and the midpoint of the line segment between the upper right key point and the lower right key point; determine the target direction according to at least one of the first direction and the second direction; determine the left ventricular long axis direction according to the target direction.

[0195] In one embodiment, the left ventricular long axis direction determination module is further used to: obtain a tissue region centerline toward the apex according to the tissue structure boundary; and determine the left ventricular long axis direction according to the target direction and the tissue region centerline.

[0196] In one embodiment, the inner diameter measurement line determination module 1502 is further used to: determine an inner diameter measurement positioning point; and obtain the inner diameter measurement line of the left ventricle according to a perpendicular direction between the inner diameter measurement positioning point and the long axis direction of the left ventricle.

[0197] In one embodiment, the inner diameter measurement line determination module 1502 is also used to: determine the inner diameter measurement positioning point based on the attachment point of the left ventricular posterior wall and the posterior leaflet of the mitral valve in the left ventricular peripheral key point concentration; or, when the left ventricular peripheral tissue includes the interventricular septum, determine the inner diameter measurement positioning point based on the center line of the interventricular septum area obtained from the boundary of the interventricular septum structure and the upper right key point of the left ventricle; or, determine the inner diameter measurement positioning point based on the movement area of ​​the key point of the anterior leaflet of the mitral valve in the left ventricular peripheral key point concentration.

[0198] In one embodiment, the structural parameter acquisition module 1503 is also used to: obtain the center line of the tissue area toward the apex according to the tissue structure boundary; obtain the intersection line according to the intersection point of the inner diameter measurement line and the tissue structure boundary; obtain the thickness measurement positioning point according to the intersection point of the tissue area center line and the intersection line; determine the thickness of the tissue around the left ventricle according to the thickness measurement positioning point and the tissue structure boundary.

[0199] In one embodiment, the structural parameter acquisition module 1503 is also used to: determine a thickness measurement line among several lines passing through the thickness measurement positioning point; the thickness measurement line is perpendicular to a target tangent line, and the target tangent line is a line that is tangent to the thickness measurement positioning point and is tangent to the center line of the tissue area; and determine the thickness of the tissue surrounding the left ventricle based on the intersection information between the thickness measurement line and the tissue structure boundary.

[0200] In one embodiment, the device also includes a display processing module, which is used to: obtain a change curve of the left ventricular internal diameter based on the inner diameter of the left ventricle in each frame of the cardiac image in the cardiac video of the left ventricular long axis section; obtain a change curve of the thickness of the tissue surrounding the left ventricle based on the thickness of the tissue surrounding the left ventricle in each frame of the cardiac image in the cardiac video; and display the change curve of the left ventricular internal diameter and the change curve of the thickness of the tissue surrounding the left ventricle.

[0201] Each module in the above cardiac structure parameter acquisition device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0202] Based on the same inventive concept, the embodiment of the present application also provides a device for processing cardiac video for implementing the method for processing cardiac video involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more devices for processing cardiac video provided below can refer to the limitations of the method for processing cardiac video above, and will not be repeated here.

[0203] In one embodiment, Fig.16 As shown, a device for processing a cardiac video is provided, comprising:

[0204] The video acquisition module 1601 is used to acquire a cardiac video of the long axis section of the left ventricle;

[0205] A left ventricular internal diameter sequence acquisition module 1602 is used to process a plurality of cardiac image frames of the cardiac video according to the cardiac structure parameter acquisition method to obtain the left ventricular internal diameter sequence;

[0206] An initial value determination module 1603, configured to determine at least one of an initial frame of the end-systole and an initial frame of the end-diastole of the heart according to the inner diameter sequence of the left ventricle;

[0207] The distance sequence acquisition module 1604 is used to obtain at least one of a first distance sequence and a second distance sequence according to a plurality of cardiac image frames of the cardiac video; the first distance is the distance between a key point of the anterior leaflet of the mitral valve and a key point of the posterior leaflet of the mitral valve; the second distance is the distance between a right upper key point of the left ventricle and a key point of the anterior leaflet of the mitral valve;

[0208] The target frame determination module 1605 is used to obtain at least one of the end-systolic target frame and the end-diastolic target frame according to at least one of the first distance sequence and the second distance sequence, and at least one of the end-systolic initial frame and the end-diastolic initial frame.

[0209] In one embodiment, the device also includes an information display module, which is used to: obtain a target frame measurement result and display the target frame measurement result based on at least one of the cardiac structural parameters of the end-systolic target frame and the end-diastolic target frame; and / or, display marking information for the end-systolic target frame and the end-diastolic target frame on the change curve of the left ventricular internal diameter and the change curve of the thickness of the tissue around the left ventricle.

[0210] In one embodiment, the distance sequence acquisition module 1604 is further used for: if the end-systolic initial frame is determined according to the inner diameter sequence of the left ventricle, then key point extraction processing is performed on the multiple frames of cardiac images located before and after the end-systolic initial frame to obtain at least one of the first distance sequence and the second distance sequence; if the end-diastolic initial frame is determined according to the inner diameter sequence of the left ventricle, then key point extraction processing is performed on the multiple frames of cardiac images located before and after the end-diastolic initial frame to obtain at least one of the first distance sequence and the second distance sequence.

[0211] Each module in the above-mentioned cardiac video processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0212] In an exemplary embodiment, a computer device is provided, the internal structure of which can be shown as follows: Fig.17 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the above method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for obtaining cardiac structure parameters or a method for processing cardiac video is implemented.

[0213] Those skilled in the art will understand that Fig.17 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0214] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned various method embodiments when executing the computer program.

[0215] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0216] In one embodiment, a computer program product is provided, on which a computer program is stored, and the computer program is used by a processor to execute the steps in the above-mentioned various method embodiments.

[0217] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0218] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0219] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0220] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for obtaining cardiac structural parameters, characterized in that: The method comprises: According to the cardiac image of the long axis section of the left ventricle, the key point set around the left ventricle, the cardiac cavity structure boundary of the left ventricle and the tissue structure boundary of the tissue around the left ventricle are obtained; Obtaining a measurement line of the inner diameter of the left ventricle according to the long axis direction of the left ventricle determined by the key point set around the left ventricle; The inner diameter of the left ventricle and the thickness of the tissue surrounding the left ventricle are determined according to the intersection information of the inner diameter measurement line with the cardiac cavity structure boundary and the tissue structure boundary.

2. The method according to claim 1, characterized in that Determining the long axis direction of the left ventricle according to the left ventricle peripheral key point set includes: Acquire the upper left key point of the left ventricle, the upper right key point of the left ventricle, the lower left key point of the left ventricle, and the lower right key point of the left ventricle; The long axis direction of the left ventricle is determined according to the upper left key point, the upper right key point, the lower left key point and the lower right key point.

3. The method according to claim 2, characterized in that Determining the long axis direction of the left ventricle according to the upper left key point, the upper right key point, the lower left key point and the lower right key point includes: Determine a first direction according to the slope of a line segment between the upper left key point and the upper right key point, and the slope of a line segment between the lower left key point and the lower right key point; Determine a second direction according to the midpoint of the line segment between the upper left key point and the lower left key point, and the midpoint of the line segment between the upper right key point and the lower right key point; determining a target direction according to at least one of the first direction and the second direction; The long axis direction of the left ventricle is determined according to the target direction.

4. The method according to claim 3, characterized in that Determining the long axis direction of the left ventricle according to the target direction includes: According to the tissue structure boundary, a center line of the tissue region toward the apex of the heart is obtained; The long axis direction of the left ventricle is determined according to the target direction and the center line of the tissue region.

5. The method according to claim 1, characterized in that Obtaining the inner diameter measurement line of the left ventricle according to the left ventricle long axis direction determined by the left ventricle peripheral key point set, including: Determine the inner diameter measurement positioning point; The inner diameter measurement line of the left ventricle is obtained according to the perpendicular direction between the inner diameter measurement positioning point and the long axis direction of the left ventricle.

6. The method according to claim 5, characterized in that Determine the inner diameter measurement positioning point, including: According to the attachment points of the left ventricular posterior wall and the posterior leaflet of the mitral valve where the key points around the left ventricle are concentrated, the positioning points for measuring the inner diameter are determined; Alternatively, when the tissue surrounding the left ventricle includes the interventricular septum, the inner diameter measurement positioning point is determined according to the center line of the interventricular septum region obtained from the structural boundary of the interventricular septum and the upper right key point of the left ventricle; Alternatively, the inner diameter measurement positioning point is determined according to the motion area of ​​the mitral valve anterior leaflet key points concentrated around the left ventricle.

7. The method according to claim 1, characterized in that Determining the thickness of the tissue surrounding the left ventricle according to the intersection information of the inner diameter measurement line and the tissue structure boundary includes: According to the tissue structure boundary, a center line of the tissue region toward the apex of the heart is obtained; Obtaining an intersection line according to the intersection point of the inner diameter measurement line and the tissue structure boundary; Obtaining a thickness measurement positioning point according to the intersection point of the tissue area center line and the intersection line; The thickness of the left ventricular periventricular tissue is determined according to the thickness measurement positioning point and the tissue structure boundary.

8. The method according to claim 7, characterized in that Determining the thickness of the left ventricular peripheral tissue according to the thickness measurement positioning point and the tissue structure boundary includes: Determine a thickness measurement line among several lines passing through the thickness measurement positioning point; the thickness measurement line is perpendicular to a target tangent line, and the target tangent line is a line passing through the thickness measurement positioning point and tangent to the center line of the tissue region; The thickness of the tissue surrounding the left ventricle is determined according to the intersection information between the thickness measurement line and the tissue structure boundary.

9. The method according to claim 1, characterized in that: The method further comprises: According to the inner diameter of the left ventricle in each frame of the cardiac image in the cardiac video of the long axis section of the left ventricle, a curve of the change of the inner diameter of the left ventricle is obtained; Obtaining a curve of thickness of left ventricular periventricular tissue according to the thickness of left ventricular periventricular tissue in each frame of cardiac image in the cardiac video; The changing curve of the left ventricular internal diameter and the changing curve of the left ventricular surrounding tissue thickness are displayed.

10. A method for processing a cardiac video, characterized in that: The method comprises: Obtain a cardiac video of the long-axis view of the left ventricle; According to the method according to any one of claims 1 to 9, a plurality of frames of cardiac image images of the cardiac video are processed to obtain the inner diameter sequence of the left ventricle; Determine at least one of an initial frame of the end-systole and an initial frame of the end-diastole of the heart according to the inner diameter sequence of the left ventricle; According to the plurality of cardiac image frames of the cardiac video, at least one of a first distance sequence and a second distance sequence is obtained; the first distance is the distance between a key point of the anterior leaflet of the mitral valve and a key point of the posterior leaflet of the mitral valve; the second distance is the distance between a right upper key point of the left ventricle and a key point of the anterior leaflet of the mitral valve; At least one of an end-systolic target frame and an end-diastolic target frame is obtained according to at least one of the first distance sequence and the second distance sequence and at least one of an end-systolic initial frame and an end-diastolic initial frame.

11. The method according to claim 10, characterized in that The method further comprises: Obtaining a target frame measurement result according to at least one cardiac structural parameter of the end-systolic target frame and the end-diastolic target frame and displaying the target frame measurement result; and / or, The marking information for the end-systolic target frame and the end-diastolic target frame is displayed on the curve of the change of the left ventricular internal diameter and the curve of the change of the left ventricular peripheral tissue thickness.

12. The method according to claim 10, characterized in that Obtaining at least one of a first distance sequence and a second distance sequence according to a plurality of cardiac image frames of the cardiac video, including: If the end-systolic initial frame is determined according to the left ventricular inner diameter sequence, key point extraction is performed on multiple cardiac image frames located before and after the end-systolic initial frame to obtain at least one of a first distance sequence and a second distance sequence; If the end-diastolic initial frame is determined according to the left ventricular inner diameter sequence, key point extraction processing is performed on multiple frames of cardiac images located before and after the end-diastolic initial frame to obtain at least one of a first distance sequence and a second distance sequence.

13. A device for acquiring cardiac structural parameters, characterized in that: The device comprises: An image processing module, for obtaining a set of key points around the left ventricle, a cardiac cavity structure boundary of the left ventricle, and a tissue structure boundary of tissues around the left ventricle according to a cardiac image of a long axis section of the left ventricle; An inner diameter measurement line determination module, used to obtain the inner diameter measurement line of the left ventricle according to the left ventricle long axis direction determined by the left ventricle peripheral key point set; The structural parameter acquisition module is used to determine the inner diameter of the left ventricle and the thickness of the tissue surrounding the left ventricle according to the intersection information of the inner diameter measurement line with the cardiac cavity structure boundary and the tissue structure boundary.

14. A device for processing cardiac video, characterized in that: The device comprises: A video acquisition module, used for acquiring a cardiac video of a left ventricular long axis section; A left ventricular internal diameter sequence acquisition module, used for processing a plurality of cardiac image frames of the cardiac video according to the method described in any one of claims 1 to 9 to obtain the left ventricular internal diameter sequence; An initial value determination module, used to determine at least one of an end-systolic initial frame and an end-diastolic initial frame of the heart according to the inner diameter sequence of the left ventricle; A distance sequence acquisition module is used to obtain at least one of a first distance sequence and a second distance sequence according to a plurality of cardiac image frames of the cardiac video; the first distance is the distance between a key point of the anterior leaflet of the mitral valve and a key point of the posterior leaflet of the mitral valve; the second distance is the distance between a right upper key point of the left ventricle and a key point of the anterior leaflet of the mitral valve; The target frame determination module is used to obtain at least one of the end-systolic target frame and the end-diastolic target frame according to at least one of the first distance sequence and the second distance sequence, and at least one of the end-systolic initial frame and the end-diastolic initial frame.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 12 is implemented.

Images