A method and apparatus for processing feature dimensions based on cardiac ultrasound video measurements.

Abstract

This invention relates to a method and apparatus for processing characteristic dimensions based on cardiac ultrasound video measurements. The method includes: receiving a first video; extracting video frame images; detecting key points in each frame image based on a key point detection model; filtering diastolic and systolic frame images; filtering mid-systolic ventricular frame images; annotating key points and characteristic dimensions of the right ventricular diameter, interventricular septum thickness, left ventricular diameter, and left ventricular posterior wall thickness on the diastolic frame images; annotating key points and characteristic dimensions of the left ventricular diameter on the systolic frame images; annotating key points and characteristic dimensions of the left atrial anteroposterior diameter, aortic valve annulus diameter, aortic sinus diameter, and ascending aortic diameter on the mid-systolic ventricular frame images; and outputting the annotated diastolic, systolic, and mid-systolic ventricular frame images as the processing result. This invention can solve the problem of manual intervention in conventional methods and improve measurement efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a processing method and apparatus based on the characteristic dimensions of cardiac ultrasound video measurements. Background Technology

[0002] Echocardiography is a non-invasive technique that uses ultrasound waves to examine the anatomical structure and activity of the heart and great vessels. Two-dimensional echocardiography, specifically, involves an ultrasound beam emitted from a probe entering the chest wall and scanning in a fan shape. Depending on the probe's position and angle, different types of cross-sectional images (parasternal long-axis view, parasternal short-axis view, apical view, subxiphoid view, etc.) are obtained. On the parasternal long-axis view obtained by two-dimensional echocardiography, the cross-sectional structures of the left ventricle, right ventricle, interventricular septum, left ventricular posterior wall, left atrium, aortic valve annulus, aortic sinus, and ascending aorta can be observed. The observation primarily involves measuring the characteristic dimensions of these structures (left ventricular diameter, right ventricular diameter, interventricular septum thickness, left ventricular posterior wall thickness, left atrial anteroposterior diameter, aortic valve annulus diameter, aortic sinus diameter, and ascending aorta diameter).

[0003] Currently, most procedures for measuring the characteristic dimensions of the parasternal long-axis section are as follows: The operator places the ultrasound probe of the echocardiography equipment on the patient's chest at the corresponding examination site in the parasternal long-axis section and performs a two-dimensional echocardiogram, outputting a video data segment, i.e., the echocardiography video. The duration of this echocardiography video is approximately one cardiac cycle of the patient. Then, using image framing technology, this echocardiography video is divided into multiple single-frame images, each of which is an independent parasternal long-axis section. Finally, it is manually screened... The selection method involves choosing three typical time points (diastole, systole, and mid-systole) from multiple single-frame images. Then, through manual observation, characteristic dimensions (left ventricular diameter, right ventricular diameter, interventricular septum thickness, left ventricular posterior wall thickness, left atrial anteroposterior diameter, aortic annulus diameter, aortic sinus diameter, and ascending aorta diameter) of different cross-sectional structures (left ventricle, right ventricle, interventricular septum, left ventricular posterior wall, left atrium, aortic valve annulus, aortic sinus diameter, and ascending aorta diameter) are measured on these three typical time point images. Clearly, this conventional method of measuring characteristic dimensions in the parasternal long-axis section relies heavily on manual labor. Due to human factors, this conventional method cannot achieve high measurement efficiency or guarantee stable measurement quality. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a processing method, apparatus, electronic device, and computer-readable storage medium based on the measurement of characteristic dimensions in cardiac ultrasound video. First, frame images are extracted from the cardiac ultrasound video. Then, based on a keypoint detection model, keypoint detection is performed on each frame image to obtain sixteen key points and eight characteristic dimensions corresponding to eight cross-sectional structures (left ventricle, right ventricle, interventricular septum, left ventricular posterior wall, left atrium, aortic valve annulus, aortic sinus, and ascending aorta). Finally, based on the left ventricular characteristic dimension (i.e., left ventricular diameter), diastolic and systolic frames are selected from multiple frames, using the characteristic dimensions of the aortic valve annulus and aortic sinus. Based on the dimensions, namely the diameter of the aortic valve annulus and the diameter of the aortic sinus, mid-systolic frames are selected from multiple images. Key points and characteristic dimensions are marked on the diastolic frames for the right ventricular ventricular diameter, interventricular septal thickness, left ventricular diameter, and left ventricular posterior wall thickness. Key points and characteristic dimensions are marked on the systolic frames for the left ventricular diameter. Key points and characteristic dimensions are marked on the mid-systolic frames for the left atrial anteroposterior diameter, aortic valve annulus diameter, aortic sinus diameter, and ascending aortic diameter. Finally, the diastolic, systolic, and mid-systolic frames with marked key points and characteristic dimensions are output as the processing result. This invention solves the problem of requiring manual intervention in conventional processing methods, improves measurement efficiency, and ensures the stability of measurement quality.

[0005] To achieve the above objectives, a first aspect of the present invention provides a processing method based on the characteristic dimensions of cardiac ultrasound video measurements, the method comprising:

[0006] Receive cardiac ultrasound video as the corresponding first video;

[0007] The first video is processed to extract video frame images to generate a corresponding first frame image sequence; the first frame image sequence consists of multiple first frame images P i The images are arranged chronologically, with each first frame image corresponding to a first frame timestamp T. i Frame image index i ≥ 1;

[0008] Based on a preset key point detection model, each of the first frame images P is analyzed. i Keypoint detection processing yields sixteen keypoints and eight feature dimensions; the eight feature dimensions include the first left ventricular diameter d. vs,i Diameter d of the first aortic valve annulus avr,i and the diameter d of the first aortic sinus as,i ;

[0009] Based on all the first left ventricular diameters d vs,iThe first frame image sequence is processed by filtering diastolic and systolic frame images to obtain the corresponding first diastolic frame image and first systolic frame image; and based on the diameter d of all the first aortic valve annulus... avr,i and the diameter d of the first aortic sinus as,i The first frame image sequence is subjected to mid-ventricular systolic frame image filtering processing to obtain the corresponding first mid-ventricular systolic frame image;

[0010] Key points and feature dimensions were marked on the right ventricular diameter, interventricular septum thickness, left ventricular diameter, and left ventricular posterior wall thickness in the first diastolic frame image; key points and feature dimensions were marked on the left ventricular diameter in the first systolic frame image; and key points and feature dimensions were marked on the left atrial anterior-posterior diameter, aortic valve annulus diameter, aortic sinus diameter, and ascending aortic diameter in the first mid-systolic frame image.

[0011] The first diastolic frame image, the first systolic frame image, and the first mid-systolic frame image, after key point marking and feature size annotation processing, are output as the processing results of the feature size of this cardiac ultrasound video measurement.

[0012] Preferably, the keypoint detection model is implemented based on the Keypoint-RCNN model; the keypoint detection model includes a first feature extraction network, a first region proposal network, a first region of interest alignment network, a first fully connected network, a second fully connected network, a third fully connected network, a first keypoint head network, and a first fusion module; the first feature extraction network is connected to the first region proposal network; the first region proposal network is connected to the first region of interest alignment network; the first region of interest alignment network is connected to both the first fully connected network and the first keypoint head network; the first fully connected network is connected to both the second fully connected network and the third fully connected network; the first fusion module is connected to the first keypoint head network, the second fully connected network, and the third fully connected network; the first feature extraction network is a ResNet50 network; the first keypoint head network includes multiple convolutional neural networks;

[0013] The first feature extraction network is used to perform backbone feature extraction processing on the input image of the model to generate the corresponding first feature map;

[0014] The first region proposal network is used to perform target region pre-identification processing on the first feature map to generate multiple first preselected boxes; the first preselected box includes a first preselected box offset position and a first preselected box size; the first preselected box offset position is the offset position of the first preselected box in the first feature map; the target region includes sixteen types of target regions, namely two key regions of the left ventricle, two key regions of the aortic valve annulus, two key regions of the aortic sinus, two key regions of the right ventricle, two key regions of the interventricular septum, two key regions of the posterior wall of the left ventricle, two key regions of the left atrium, and two key regions of the ascending aorta; the number of first preselected boxes is greater than 16;

[0015] The first region of interest alignment network is used to merge all the first preselected boxes to generate sixteen first final selection boxes, and extract the sub-feature maps of each first final selection box on the first feature map as the corresponding first sub-feature map; the first final selection box includes the first final selection box offset position and the first final selection box size; the first final selection box offset position is the offset position of the first final selection box in the first feature map; the sixteen first sub-feature maps correspond to the regional features of sixteen key points, specifically: the regional feature map of the first key point of the left ventricle, the regional feature map of the second key point of the left ventricle, and the aortic valve. Feature maps of the first key point region of the aortic valve annulus, the second key point region of the aortic valve annulus, the first key point region of the aortic sinus, the second key point region of the aortic sinus, the first key point region of the right ventricle, the second key point region of the right ventricle, the first key point region of the interventricular septum, the second key point region of the interventricular septum, the first key point region of the posterior wall of the left ventricle, the second key point region of the posterior wall of the left ventricle, the first key point region of the left atrium, the second key point region of the left atrium, the first key point region of the ascending aorta, and the second key point region of the ascending aorta.

[0016] The first fully connected network is used to perform global vector transformation on each of the first sub-feature maps to generate a corresponding first global vector; and to perform fully connected computation on each of the first global vectors to generate a corresponding first fully connected vector; the second fully connected network is used to perform regression prediction on the offset position of the current first sub-feature map in the model input image based on the first fully connected vector corresponding to each of the first sub-feature maps to generate a corresponding first sub-feature map offset position; the third fully connected network is used to perform keypoint classification prediction on the current first sub-feature map based on the first fully connected vector corresponding to each of the first sub-feature maps to generate sixteen first type labels; the value of the first sub-feature map type label includes 1 or 0; The six first-type labels correspond to sixteen key point types, specifically: left ventricle first key point type, left ventricle second key point type, aortic valve annulus first key point type, aortic valve annulus second key point type, aortic sinus first key point type, aortic sinus second key point type, right ventricle first key point type, right ventricle second key point type, interventricular septum first key point type, interventricular septum second key point type, left ventricular posterior wall first key point type, left ventricular posterior wall second key point type, left atrium first key point type, left atrium second key point type, ascending aorta first key point type, and ascending aorta second key point type; among the sixteen first-type labels, only one label has a value of 1, and the rest are 0;

[0017] The first key point network is used to identify the key point positions of the sixteen first sub-feature maps to obtain sixteen key points of the first sub-feature maps; each key point of the first sub-feature map corresponds to a first key point offset position; the sixteen key points of the first sub-feature maps include the first left ventricle key point, the second left ventricle key point, the first aortic valve annulus key point, the second aortic valve annulus key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricle key point, the second right ventricle key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricular posterior wall key point, the second left ventricular posterior wall key point, the first left atrium key point, the second left atrium key point, the first ascending aorta key point, and the second ascending aorta key point;

[0018] The first fusion module is used to generate corresponding first predicted keypoint image coordinates by predicting the absolute position of keypoints based on the offset position of the first final selection box and the offset position of the first keypoints corresponding to each first sub-feature map; and to select the keypoint type corresponding to the first type label with a value of 1 from the sixteen first type labels corresponding to each first sub-feature map as the corresponding first predicted keypoint type; and to form the corresponding output feature vector by the first predicted keypoint image coordinates and the first predicted keypoint type corresponding to each first sub-feature map; and to form the corresponding model output tensor by the sixteen output feature vectors and output it.

[0019] Preferably, the key point detection model based on the preset key point detection model is used for each of the first frame images P i Keypoint detection processing yields sixteen keypoints and eight feature dimensions, specifically including:

[0020] The first frame image P i The keypoint detection model is input and processed to generate a corresponding first model output tensor; the first model output tensor includes sixteen first output feature vectors; the first output feature vectors include the first predicted keypoint image coordinates and the first predicted keypoint type;

[0021] Based on the image coordinates of the first predicted key points of the first and second left ventricular key points, calculate the length of the straight line segment connecting the first and second left ventricular key points, and denote it as the corresponding first left ventricular intraventricular diameter d. vs,i Based on the image coordinates of the first predicted key points of the first and second aortic valve annulus key points, the length of the straight line segment connecting the first and second aortic valve annulus key points is calculated and denoted as the corresponding diameter d of the first aortic valve annulus. avr,i Based on the image coordinates of the first predicted key points of the first and second aortic sinus key points, the length of the straight line segment connecting the first and second aortic sinus key points is calculated and denoted as the corresponding diameter d of the first aortic sinus. as,i Based on the image coordinates of the first predicted key points of the first and second right ventricular key points, the length of the straight line segment connecting the first and second right ventricular key points is calculated and denoted as the corresponding first right ventricular intraventricular diameter d. vd,i Based on the image coordinates of the first predicted key points of the first and second interventricular septum key points, the length of the straight line segment connecting the first and second interventricular septum key points is calculated and denoted as the corresponding first interventricular septum thickness d. IS,i Based on the image coordinates of the first predicted key points of the first and second left ventricular posterior wall key points, the length of the straight line segment connecting the first and second left ventricular posterior wall key points is calculated and denoted as the corresponding first left ventricular posterior wall thickness d.lvpw,i Based on the image coordinates of the first predicted key points of the first and second left atrium, the length of the straight line segment connecting the first and second left atrium key points is calculated and denoted as the corresponding anteroposterior diameter d of the first left atrium. la,i Based on the image coordinates of the first predicted key points of the first and second ascending aortas, the length of the straight line segment connecting the first and second ascending aortas is calculated and denoted as the corresponding inner diameter d of the first ascending aorta. aa,i ;

[0022] The sixteen key points are composed of sixteen key points corresponding to the sixteen first output feature vectors; the sixteen key points include the first left ventricle key point, the second left ventricle key point, the first aortic valve annulus key point, the second aortic valve annulus key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricle key point, the second right ventricle key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricular posterior wall key point, the second left ventricular posterior wall key point, the first left atrium key point, the second left atrium key point, the first ascending aorta key point, and the second ascending aorta key point.

[0023] The eight characteristic dimensions are formed by connecting the lengths of the eight straight line segments obtained; the eight characteristic dimensions include the first left ventricular diameter d. vs,i The diameter d of the first aortic valve annulus avr,i The diameter d of the first aortic sinus as,i The first right ventricular diameter d vd,i The thickness d of the first interventricular septum IS,i The thickness d of the posterior wall of the first left ventricle lvpw,i The first left atrial anteroposterior diameter d la,i The inner diameter d of the first ascending aorta aa,i .

[0024] Preferably, the step is based on all the first left ventricular diameters d vs,i The first frame image sequence is subjected to diastolic and systolic frame image filtering to obtain the corresponding first diastolic frame image and first systolic frame image, specifically including:

[0025] A two-dimensional coordinate system is constructed with the left ventricular diameter as the vertical axis Y and time as the horizontal axis X, denoted as the corresponding left ventricular diameter-time coordinate system;

[0026] On the left ventricular diameter-time coordinate system, each of the first left ventricular diameters d vs,i The plotting point is the vertical axis coordinate, with the current first left ventricular diameter d as the reference. vs,iThe corresponding first frame timestamp T i The first plotting point is obtained by marking the plotting point with the x-axis coordinate.

[0027] The first curve is obtained by sequentially connecting the first to the last first measurement points; and the second curve is obtained by performing median filtering on the first curve.

[0028] The second curve is processed to filter the most significant peak points to obtain the corresponding first most significant peak point; the horizontal axis coordinate of the first most significant peak point is extracted as the corresponding first time point; and the first frame timestamp T with the smallest time interval to the first time point in the first frame image sequence is selected. i The corresponding first frame image P i As the corresponding first diastolic frame image;

[0029] The second curve is flipped vertically to obtain the corresponding third curve; the third curve is then processed to filter the most significant peak points to obtain the corresponding second most significant peak points; the horizontal axis coordinates of the second most significant peak points are extracted as the corresponding second time points; and the first frame timestamp T, which has the smallest time interval with the second time point in the first frame image sequence, is selected. i The corresponding first frame image P i This corresponds to the first contraction period frame image.

[0030] Furthermore, the most significant peak point filtering process specifically includes:

[0031] The second or third curve selected from the most significant peak points in this screening process is taken as the corresponding current curve; and each peak point of the current curve is marked as the corresponding first peak point.

[0032] The process iterates through each of the first peak points. During the iteration, the currently traversed first peak point is recorded as the corresponding current peak point. A straight line parallel to the horizontal axis X is drawn through the current peak point on the left ventricular diameter-time coordinate system and recorded as the corresponding first straight line. The intersection point of the first straight line with any descending edge of the current curve to the left of the current peak point is recorded as the corresponding first descending edge intersection point, and the intersection point of the first straight line with any ascending edge of the current curve to the right of the current peak point is recorded as the corresponding first ascending edge intersection point. The number of the first descending edge intersection points and the number of the first ascending edge intersection points are counted to generate the corresponding number of first descending edge intersection points and the number of first ascending edge intersection points. The process also checks whether the number of first descending edge intersection points is 0. If it is, the starting position of the current curve is taken as the corresponding first starting position; otherwise, the first first descending edge intersection point to the left of the current peak point is taken as the first starting position. The first starting position is defined as follows: The number of intersections of the first rising edges is identified as 0. If so, the end position of the current curve is taken as the first ending position; otherwise, the first intersection of the first rising edges to the right of the current peak point is taken as the first ending position. The minimum valley point on the current curve between the first ending position and the current peak point is taken as the first valley point, and the minimum valley point between the current peak point and the first ending position is taken as the second valley point. The larger of the first and second valley points is taken as the current valley point. The difference between the vertical coordinates of the current peak point and the current valley point is calculated to generate the corresponding first peak-valley difference. At the end of the traversal, the maximum value is selected from all the obtained first peak-valley differences as the corresponding maximum peak-valley difference, and the first peak point corresponding to the maximum peak-valley difference is taken as the corresponding current most significant peak point.

[0033] If the current curve is the second curve, then the current most significant peak point is output as the corresponding first most significant peak point; if the current curve is the third curve, then the current most significant peak point is output as the corresponding second most significant peak point.

[0034] Preferably, the step is based on the diameter d of all the first aortic valve annulus. avr,i and the diameter d of the first aortic sinus as,i The first frame image sequence is subjected to mid-systolic frame image filtering processing to obtain the corresponding first mid-systolic frame image, specifically including:

[0035] The diameter d of each of the first aortic valve annulus avr,i The image coordinates of the two first predicted key points corresponding to the first and second aortic valve annulus key points are denoted as the corresponding first and second coordinates s.1,i s 2,i ; and based on the first and second coordinates s 1,i s 2,i Generate a vector from the key point of the first aortic valve annulus to the key point of the second aortic valve annulus, denoted as the corresponding first vector s. 1->2,i ;s 1->2,i =s 2,i -s 1,i ;

[0036] The diameter d of each of the first aortic sinuses as,i The image coordinates of the two first predicted key points corresponding to the first and second aortic sinus key points are designated as the corresponding third and fourth coordinates s. 3,i s 4,i ; and based on the third and fourth coordinates s 3,i s 4,i Generate a vector from the first key point in the aortic sinus to the second key point in the aortic sinus, denoted as the corresponding second vector s. 3->4,i ;s 3->4,i =s 4,i -s 3,i ;

[0037] The first aortic valve annulus diameter d is the same as that of the frame image index i. avr,i The diameter d of the first aortic sinus as,i The first vector s 1->2,i and the second vector s 3->4,i Form the corresponding first data group;

[0038] A rationality analysis is performed on each of the first data groups to generate a corresponding first analysis result; and discrete data group identification is performed on all the first data groups whose first analysis results are rational to obtain multiple first discrete data groups; the first analysis results include rational and unreasonable.

[0039] In the first frame image sequence, take the first frame image P that corresponds to each of the first data groups whose first analysis result is unreasonable. i and the first frame image P corresponding to each of the first discrete data groups i Each of these is recorded as the corresponding first abnormal frame image; and abnormal frame image data correction processing is performed on each of the first abnormal frame images.

[0040] If the abnormal frame image data correction process is successful, then for each of the first frame images P i The corresponding first aortic valve annulus diameter d avr,i and the diameter d of the first aortic sinus as,iThe first sum data is obtained by adding the sums; the maximum value is selected from all the first sum data as the corresponding maximum sum data; and the first frame image P corresponding to the maximum sum data is set. i Output as the corresponding first ventricular mid-systolic frame image.

[0041] Furthermore, the step of performing a rationality analysis on each of the first data groups to generate corresponding first analysis results specifically includes:

[0042] The diameter d of the first aortic valve annulus in the first data set avr,i The diameter d of the first aortic sinus as,i The absolute value of the diameter difference is calculated to generate the corresponding first absolute diameter difference; and it is identified whether the first absolute diameter difference is lower than the preset absolute diameter difference threshold; if so, the corresponding first check bit is set to success, otherwise the corresponding first check bit is set to failure.

[0043] According to the first vector s of the first data group 1->2,i and the second vector s 3->4,i Perform vector angle estimation to generate the corresponding first angle. It also identifies whether the first included angle is less than a preset small angle threshold; if so, it sets the corresponding second check bit as successful, otherwise it sets the corresponding second check bit as failed.

[0044] The diameter d of the first aortic valve annulus in the first data set avr,i The corresponding straight line segment and the diameter d of the first aortic sinus as,i The system identifies whether the corresponding line segments intersect; if so, the corresponding third check bit is set to failure; otherwise, the corresponding third check bit is set to success.

[0045] The third check bit is identified; if the third check bit fails, the corresponding fourth check bit is set to fail; if the third check bit succeeds, the diameter d of the first aortic valve annulus in the first data group is set to fail. avr,i Does the corresponding straight line segment lie within the diameter d of the first aortic sinus? as,i The left side of the corresponding straight line segment is identified; if it is, the corresponding fourth check bit is set to success; otherwise, the corresponding fourth check bit is set to failure.

[0046] If the first, second, third, and fourth check bits are all successful, then the corresponding first analysis result is set as reasonable; if the first, second, third, or fourth check bits are unsuccessful, then the corresponding first analysis result is set as unreasonable.

[0047] Furthermore, the step of identifying multiple first discrete data groups from all first data groups whose first analysis results are reasonable specifically includes:

[0048] The first data group whose first analysis result is reasonable is recorded as the corresponding second data group;

[0049] The first aortic valve annulus diameter d of all the second data groups avr,i Form a corresponding first diameter set; and determine the diameter d of the first aortic valve annulus in the first diameter set. avr,i The first quantity N1 is generated by statistically analyzing the number of diameters; the first mean μ1 and the first standard deviation σ1 are calculated based on the mean and standard deviation of the first diameter set; a first limiting range is constructed based on the first mean μ1 and the first standard deviation σ1; and the diameter d of the first aortic valve annulus is set as follows. avr,i The second data group exceeding the first amplitude limit is denoted as the corresponding first discrete data group; wherein... The lower limit of the first limiting range is (μ1-α1*σ1), and the upper limit is (μ1+α1*σ1), where α1 is a preset coefficient;

[0050] The diameter d of the first aortic sinus of all the second data groups as,i Form a corresponding second diameter set; and determine the diameter d of the first aortic sinus in the second diameter set. as,i The number of diameters is statistically analyzed to generate a corresponding second number N2; the mean and standard deviation of the second diameter set are calculated to obtain the corresponding second mean μ2 and second standard deviation σ2; a corresponding second amplitude limiting range is constructed based on the second mean μ2 and the second standard deviation σ2; and the diameter d of the first aortic sinus is... as,i The second data group that exceeds the second amplitude limit is denoted as the corresponding first discrete data group; wherein... The lower limit of the second limiting range is (μ2-α2*σ2), and the upper limit is (μ2+α2*σ2), where α2 is a preset coefficient.

[0051] Furthermore, the abnormal frame image data correction processing for each of the first abnormal frame images specifically includes:

[0052] Take any one of the first abnormal frame images in the first frame image sequence as the corresponding current abnormal frame image; and take the previous and next non-abnormal frame images in the first frame image sequence as the corresponding previous normal frame image and next normal frame image.

[0053] The first left ventricular diameter d corresponding to the respective before and after normal frame imagesvs,i The diameter d of the first aortic valve annulus avr,i The diameter d of the first aortic sinus as,i The first right ventricular diameter d vd,i The thickness d of the first interventricular septum IS,i The thickness d of the posterior wall of the first left ventricle lvpw,i The first left atrial anteroposterior diameter d la,i and the inner diameter d of the first ascending aorta aa,i The first left ventricular diameter d corresponding to the current abnormal frame image vs,i The diameter d of the first aortic valve annulus avr,i The diameter d of the first aortic sinus as,i The first right ventricular diameter d vd,i The thickness d of the first interventricular septum IS,i The thickness d of the posterior wall of the first left ventricle lvpw,i The first left atrial anteroposterior diameter d la,i and the inner diameter d of the first ascending aorta aa,i Interpolation reset is performed using linear interpolation.

[0054] Using the image coordinates of the first predicted key points corresponding to the first left ventricular key points, second left ventricular key points, first aortic valve annulus key points, second aortic valve annulus key points, first aortic sinus key points, second aortic sinus key points, first right ventricular key points, second right ventricular key points, first interventricular septum key points, second interventricular septum key points, first left ventricular posterior wall key points, second left ventricular posterior wall key points, first left atrium key points, second left atrium key points, first ascending aorta key points, and second ascending aorta key points in the preceding and following normal frame images, for the current... The image coordinates of the first predicted key points corresponding to the first left ventricular key point, the second left ventricular key point, the first aortic valve annulus key point, the second aortic valve annulus key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricular key point, the second right ventricular key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricular posterior wall key point, the second left ventricular posterior wall key point, the first left atrium key point, the second left atrium key point, the first ascending aorta key point, and the second ascending aorta key point in the previous abnormal frame image are interpolated and reset using a linear interpolation method.

[0055] Preferably, the step of marking key points and annotating feature dimensions of the right ventricular diameter, interventricular septum thickness, left ventricular diameter, and left ventricular posterior wall thickness on the first diastolic frame image specifically includes:

[0056] On the first diastolic frame image, the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second right ventricular key points is set to a preset first color; and a connecting line segment is drawn between the two pixels corresponding to the first and second right ventricular key points based on the first color to obtain the corresponding first line segment; and the first right ventricular intraventricular diameter d corresponding to the first diastolic frame image is set to... vd,i The product of the first line segment and the preset unit scale is used as the corresponding second right ventricular diameter; and the display label information of the first line segment is set as the corresponding second right ventricular diameter.

[0057] On the first diastolic frame image, the color of two pixels corresponding to the image coordinates of the two first predicted key points of the first and second interventricular septum is set to a preset second color; and a connecting line segment is drawn between the two pixels corresponding to the first and second interventricular septum key points based on the second color to obtain the corresponding second line segment; and the first interventricular septum thickness d corresponding to the first diastolic frame image is set to... IS,i The product of the product with the unit scale is taken as the corresponding second interventricular septum thickness; and the display label information of the second line segment is set as the corresponding second interventricular septum thickness;

[0058] On the first diastolic frame image, the color of two pixels corresponding to the image coordinates of the two first predicted key points of the first and second left ventricular key points is set to a preset third color; and a connecting line segment is drawn between the two pixels corresponding to the first and second left ventricular key points based on the third color to obtain the corresponding third line segment; and the first left ventricular intraventricular diameter d corresponding to the first diastolic frame image is set to... vs,i The product of the product with the unit scale is taken as the corresponding second left ventricular diameter; and the display label information of the third line segment is set as the corresponding second left ventricular diameter;

[0059] On the first diastolic frame image, the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second left ventricular posterior wall key points is set to a preset fourth color; and a connecting line segment is drawn between the two pixels corresponding to the first and second left ventricular posterior wall key points based on the fourth color to obtain the corresponding fourth line segment; and the thickness d of the first left ventricular posterior wall corresponding to the first diastolic frame image is set to... lvpw,i The product of the product with the unit scale is taken as the corresponding second left ventricular posterior wall thickness; and the display label information of the fourth line segment is set as the corresponding second left ventricular posterior wall thickness.

[0060] Preferably, the step of marking key points and annotating feature dimensions of the left ventricular diameter on the first systolic frame image specifically includes:

[0061] On the first systolic frame image, the color of two pixels corresponding to the image coordinates of the first predicted key points of the first and second left ventricular key points is set to a preset fifth color; and a connecting line segment is drawn between the two pixels corresponding to the first and second left ventricular key points based on the fifth color to obtain the corresponding fifth line segment; and the first left ventricular intraventricular diameter d corresponding to the first systolic frame image is set to a preset fifth color. vs,i The product of the product with the preset unit scale is used as the corresponding third left ventricular diameter; and the display label information of the fifth line segment is set as the corresponding third left ventricular diameter.

[0062] Preferably, the step of marking key points and annotating features of the left atrial anteroposterior diameter, aortic valve annulus diameter, aortic sinus diameter, and ascending aortic diameter on the first mid-systolic frame image specifically includes:

[0063] In the first mid-systolic frame image of the ventricle, the color of the two pixels corresponding to the image coordinates of the first predicted key points of the first and second left atrium is set to a preset sixth color; and a connecting line segment is drawn between the two pixels corresponding to the first and second left atrium key points based on the sixth color to obtain the corresponding sixth line segment; and the anteroposterior diameter d of the first left atrium corresponding to the first mid-systolic frame image of the ventricle is set to the sixth color. la,i The product of the product with the preset unit scale is used as the corresponding second left atrial anteroposterior diameter; and the display label information of the sixth line segment is set as the corresponding second left atrial anteroposterior diameter;

[0064] In the first mid-systolic frame image, the color of two pixels corresponding to the image coordinates of the first and second aortic valve annulus key points is set to a preset seventh color; and a connecting line segment is drawn between the two pixels corresponding to the first and second aortic valve annulus key points based on the seventh color to obtain the corresponding seventh line segment; and the diameter d of the first aortic valve annulus corresponding to the first mid-systolic frame image is set to... avr,i The product of the product with the unit scale is taken as the corresponding second aortic valve annulus diameter; and the display annotation information of the seventh line segment is set as the corresponding second aortic valve annulus diameter;

[0065] In the first mid-systolic frame image, the color of two pixels corresponding to the image coordinates of the first and second aortic sinus key points is set to a preset eighth color; and based on the eighth color, a connecting line segment is drawn between the two pixels corresponding to the first and second aortic sinus key points to obtain the corresponding eighth line segment; and the diameter d of the first aortic sinus corresponding to the first mid-systolic frame image is set to...as,i The product of the product with the unit scale is used as the corresponding second aortic sinus diameter; and the display label information of the eighth line segment is set as the corresponding second aortic sinus diameter;

[0066] In the first mid-systolic frame image, the color of two pixels corresponding to the image coordinates of the first predicted key points of the first and second ascending aortas is set to a preset ninth color; and a connecting line segment is drawn between the two pixels corresponding to the first and second ascending aortas based on the ninth color to obtain the corresponding ninth line segment; and the inner diameter d of the first ascending aorta corresponding to the first mid-systolic frame image is set to... aa,i The product of the product with the unit scale is taken as the corresponding second ascending aortic inner diameter; and the display label information of the ninth line segment is set as the corresponding second ascending aortic inner diameter.

[0067] A second aspect of the present invention provides an apparatus for implementing the method described in the first aspect above, comprising: a receiving module, an image framing module, a key point detection module, a framed image filtering module, an image annotation module, and an output module;

[0068] The receiving module is used to receive cardiac ultrasound video as the corresponding first video.

[0069] The image framing module is used to extract video frame images from the first video to generate a corresponding first frame image sequence; the first frame image sequence consists of multiple first frame images P i The images are arranged chronologically, with each first frame image corresponding to a first frame timestamp T. i Frame image index i ≥ 1;

[0070] The key point detection module is used to perform key point detection on each of the first frame images P based on a preset key point detection model. i Keypoint detection processing yields sixteen keypoints and eight feature dimensions; the eight feature dimensions include the first left ventricular diameter d. vs,i Diameter d of the first aortic valve annulus avr,i and the diameter d of the first aortic sinus as,i ;

[0071] The framed image filtering module is used to filter images based on all the first left ventricular diameters d. vs,i The first frame image sequence is processed by filtering diastolic and systolic frame images to obtain the corresponding first diastolic frame image and first systolic frame image; and based on the diameter d of all the first aortic valve annulus... avr,i and the diameter d of the first aortic sinus as,iThe first frame image sequence is subjected to mid-ventricular systolic frame image filtering processing to obtain the corresponding first mid-ventricular systolic frame image;

[0072] The image annotation module is used to perform key point marking and feature size annotation on the right ventricular diameter, interventricular septum thickness, left ventricular diameter, and left ventricular posterior wall thickness on the first diastolic frame image; and to perform key point marking and feature size annotation on the left ventricular diameter on the first systolic frame image; and to perform key point marking and feature size annotation on the left atrial anteroposterior diameter, aortic valve annulus diameter, aortic sinus diameter, and ascending aortic diameter on the first mid-systolic frame image.

[0073] The output module is used to output the first diastolic frame image, the first systolic frame image, and the first mid-systolic frame image, which have undergone key point marking and feature size annotation processing, as the processing results of the feature size of this cardiac ultrasound video measurement.

[0074] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0075] The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method described in the first aspect above;

[0076] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0077] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect above.

[0078] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing characteristic dimensions based on cardiac ultrasound video. First, frame images are extracted from the cardiac ultrasound video. Then, based on a keypoint detection model, keypoint detection is performed on each frame image to obtain sixteen keypoints and eight characteristic dimensions corresponding to eight cross-sectional structures (left ventricle, right ventricle, interventricular septum, left ventricular posterior wall, left atrium, aortic valve annulus, aortic sinus, and ascending aorta). Based on the left ventricular characteristic dimension (left ventricular diameter), diastolic and systolic frames are selected from multiple frames. The characteristic dimensions of the aortic valve annulus and aortic sinus (aortic diameter) are then used to select diastolic and systolic frames. Based on the annular diameter and aortic sinus diameter, mid-systolic frames are selected from multiple images. Key points and characteristic dimensions are marked on the diastolic frames for the right ventricular diameter, interventricular septal thickness, left ventricular diameter, and left ventricular posterior wall thickness. Key points and characteristic dimensions are marked on the systolic frames for the left ventricular diameter. Key points and characteristic dimensions are marked on the mid-systolic frames for the left atrial anteroposterior diameter, aortic annular diameter, aortic sinus diameter, and ascending aortic diameter. Finally, the diastolic, systolic, and mid-systolic frames with marked key points and characteristic dimensions are output as the processing result. This invention solves the problem of requiring manual intervention in conventional processing methods, improves measurement efficiency, and ensures the stability of measurement quality. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of a processing method based on the feature size of cardiac ultrasound video measurement provided in Embodiment 1 of the present invention;

[0080] Figure 2 This is a module structure diagram of a processing device based on the feature size of cardiac ultrasound video measurement provided in Embodiment 2 of the present invention;

[0081] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0083] Embodiment 1 of the present invention provides a processing method based on the characteristic dimensions of cardiac ultrasound video measurements, such as... Figure 1The schematic diagram shows a processing method based on the feature size of cardiac ultrasound video measurement provided in Embodiment 1 of the present invention. This method mainly includes the following steps:

[0084] Step 1: Receive the cardiac ultrasound video as the corresponding first video.

[0085] Here, the cardiac ultrasound video received in Embodiment 1 of the present invention is a segment of cardiac ultrasound video obtained by the operator placing the ultrasound probe of the cardiac ultrasound examination device on the detection site corresponding to the long axis section of the chest and the sternum of the examinee and performing two-dimensional echocardiography. The duration of this cardiac ultrasound video is approximately one cardiac cycle of the examinee.

[0086] Step 2: Extract video frame images from the first video to generate the corresponding first frame image sequence;

[0087] The first frame image sequence consists of multiple first frame images P i The images are arranged chronologically, with each first frame corresponding to a first frame timestamp T. i , where the frame image index i ≥ 1.

[0088] Here, the first video is segmented into frames according to a preset framing frequency to obtain multiple first-frame images. Each first-frame image is actually a long-axis cross-section image of the sternum. Then, all the first-frame images are sorted in chronological order to obtain a sequence of first-frame images. Since each frame image has a specific time information, each first-frame image corresponds to a first-frame timestamp T. i .

[0089] In Embodiment 1 of the present invention, after obtaining the first frame image sequence, keypoint detection is performed on each frame image using a preset keypoint detection model in subsequent step 3. Before describing subsequent step 3, the keypoint detection model of Embodiment 1 of the present invention is described as follows:

[0090] The keypoint detection model in Embodiment 1 of this invention is implemented with reference to the model structure of the Keypoint-RCNN model. The Keypoint-RCNN model's structure is derived from the keypoint detection branch structure of Mask R-CNN, as detailed in the publicly available technical paper "Mask R-CNN," and will not be repeated here. The neural network of the keypoint detection model in Embodiment 1 of this invention includes a first feature extraction network, a first Region Proposal Networks (RPN) network, a first Region of Interest Align (ROI Align) network, a first Fully Connected (FC) network, a second Fully Connected network, a third Fully Connected network, a first Keypoint RCNNHead network, and a first fusion module. The first feature extraction network is a ResNet50 network; the first keypoint head network includes multiple convolutional neural networks.

[0091] The connection relationships of each network / module in the model are as follows: the first feature extraction network is connected to the first region proposal network; the first region proposal network is connected to the first region of interest alignment network; the first region of interest alignment network is connected to the first fully connected network and the first keypoint network respectively; the first fully connected network is connected to the second fully connected network and the third fully connected network respectively; the first fusion module is connected to the first keypoint network, the second fully connected network, and the third fully connected network respectively.

[0092] The first feature extraction network in Embodiment 1 of the present invention is used to perform backbone feature extraction processing on the input image of the model to generate the corresponding first feature map;

[0093] In Embodiment 1 of the present invention, the first region proposal network is used to perform target region pre-identification processing on the first feature map to generate multiple first preselected bounding boxes (bboxes);

[0094] The first preselected bounding box includes its offset position and size. The offset position is the position of the first preselected bounding box within the first feature map. The target regions of interest to the first region proposal network include sixteen categories, each involving one of the eight sectional structures of the heart (left ventricle, right ventricle, interventricular septum, left ventricular posterior wall, left atrium, aortic valve annulus, aortic sinus, and ascending aorta). Each sectional structure has two key point regions (also called key regions). Therefore, the sixteen target regions are: two key regions of the left ventricle, two key regions of the aortic valve annulus, two key regions of the aortic sinus, two key regions of the right ventricle, two key regions of the interventricular septum, two key regions of the left ventricular posterior wall, two key regions of the left atrium, and two key regions of the ascending aorta. Because the first region proposal network may generate multiple first preselected bounding boxes for a certain type of target region, the number of first preselected bounding boxes is greater than 16.

[0095] The first region of interest alignment network in Embodiment 1 of the present invention is used to merge all the first preselected boxes to generate sixteen first final selection boxes, and extract the sub-feature maps of each first final selection box on the first feature map as the corresponding first sub-feature map.

[0096] The first final bounding box includes its offset position and size. The offset position is its position within the first feature map. The first region of interest alignment network merges one or more first final bounding boxes corresponding to the same target region to generate a single first final bounding box, resulting in sixteen first final bounding boxes corresponding to sixteen target regions. The sub-feature maps covered by each first final bounding box on the first feature map are then extracted to obtain the corresponding first sub-feature maps, resulting in sixteen first sub-feature maps. These sixteen first sub-feature maps correspond to sixteen target regions, meaning they represent the region features of sixteen keypoints. Specifically, these include: feature maps of the first key point region of the left ventricle, the second key point region of the left ventricle, the first key point region of the aortic valve annulus, the second key point region of the aortic valve annulus, the first key point region of the aortic sinus, the second key point region of the aortic sinus, the first key point region of the right ventricle, the second key point region of the right ventricle, the first key point region of the interventricular septum, the second key point region of the interventricular septum, the first key point region of the posterior wall of the left ventricle, the second key point region of the posterior wall of the left ventricle, the first key point region of the left atrium, the second key point region of the left atrium, the first key point region of the ascending aorta, and the second key point region of the ascending aorta.

[0097] In Embodiment 1 of the present invention, the first fully connected network is used to perform global vector transformation on each first sub-feature map to generate a corresponding first global vector; and to perform fully connected computation on each first global vector to generate a corresponding first fully connected vector; the second fully connected network in Embodiment 1 of the present invention is used to perform regression prediction on the offset position of the current first sub-feature map in the model input image based on the first fully connected vector corresponding to each first sub-feature map to generate the corresponding first sub-feature map offset position; the third fully connected network in Embodiment 1 of the present invention is used to perform keypoint classification prediction on the current first sub-feature map based on the first fully connected vector corresponding to each first sub-feature map to generate sixteen first type labels;

[0098] The first sub-feature map type label has a value of 1 or 0; the sixteen first type labels correspond to sixteen key point types, specifically: left ventricle first key point type, left ventricle second key point type, aortic valve annulus first key point type, aortic valve annulus second key point type, aortic sinus first key point type, aortic sinus second key point type, right ventricle first key point type, right ventricle second key point type, interventricular septum first key point type, interventricular septum second key point type, left ventricular posterior wall first key point type, left ventricular posterior wall second key point type, left atrium first key point type, left atrium second key point type, ascending aorta first key point type, and ascending aorta second key point type; only one of the sixteen first type labels has a value of 1, and the rest are 0;

[0099] In Embodiment 1 of the present invention, the first key point network is used to identify the key point positions of sixteen first sub-feature maps to obtain sixteen key points of the first sub-feature maps;

[0100] Among them, the key point of the first sub-feature map corresponds to the offset position of a first key point; the sixteen key points of the first sub-feature map include the first left ventricle key point, the second left ventricle key point, the first aortic valve annulus key point, the second aortic valve annulus key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricle key point, the second right ventricle key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricular posterior wall key point, the second left ventricular posterior wall key point, the first left atrium key point, the second left atrium key point, the first ascending aorta key point, and the second ascending aorta key point;

[0101] In Embodiment 1 of the present invention, the first fusion module is used to predict the absolute position of key points based on the offset position of the first final selection box and the offset position of the corresponding first key point in each first sub-feature map, generating the corresponding first predicted key point image coordinates; and selects the key point type corresponding to the first type label with a value of 1 from the sixteen first type labels corresponding to each first sub-feature map as the corresponding first predicted key point type; and forms the corresponding output feature vector with the first predicted key point image coordinates and the first predicted key point type corresponding to each first sub-feature map; and forms the corresponding model output tensor with the obtained sixteen output feature vectors and outputs it.

[0102] In summary, the key point detection model in Embodiment 1 of the present invention is used to detect each first frame image P. i An artificial intelligence model for detecting sixteen key points (first left ventricular key point, second left ventricular key point, first aortic valve annulus key point, second aortic valve annulus key point, first aortic valve annulus key point, second aortic valve annulus key point, first aortic sinus key point, second aortic sinus key point, first right ventricular key point, second right ventricular key point, first interventricular septum key point, second interventricular septum key point, first left ventricular posterior wall key point, second left ventricular posterior wall key point, first left atrium key point, second left atrium key point, first ascending aorta key point, and second ascending aorta key point) of eight cross-sectional structures of the upper heart (left ventricle, right ventricle, interventricular septum, posterior wall of left ventricle, first left atrium key point, second left atrium key point, first ascending aorta key point, and second ascending aorta key point)

[0103] It should be noted that the first and second key points of the left ventricle are actually two characteristic points on the left ventricular section used to measure the internal diameter of the left ventricle; the first and second key points of the right ventricle are actually two characteristic points on the right ventricular section used to measure the internal diameter of the right ventricle; the first and second key points of the interventricular septum are actually two characteristic points on the interventricular septum section used to measure the thickness of the left and right interventricular septum (also known as the interventricular septum thickness); the first and second key points of the posterior wall of the left ventricle are actually two characteristic points on the left ventricular posterior wall section used to measure the thickness of the left ventricular posterior wall; the first and second key points of the left atrium are actually two characteristic points on the left atrium section used to measure the anteroposterior diameter of the left atrium; the first and second key points of the aortic valve annulus are actually two characteristic points on the aortic valve annulus section used to measure the diameter of the aortic valve annulus; the first and second key points of the aortic sinus are actually two characteristic points on the aortic sinus section used to measure the diameter of the aortic sinus; and the first and second key points of the ascending aorta are actually two characteristic points on the ascending aorta section used to measure the internal diameter of the ascending aorta. It should also be noted that before using the keypoint detection model of Embodiment 1 of the present invention, it needs to be trained. The model training method is supervised training, that is, training data with sixteen target region labels and sixteen keypoint labels is selected and fed into the keypoint detection model for training. During training, a multi-class cross-entropy loss function is used as the loss function for model training. The number of training iterations of the keypoint detection model of Embodiment 1 of the present invention is not fixed; a training frequency can be preset for periodic training. Continuous model training can continuously improve the image analysis quality of the model.

[0104] Step 3: Based on the preset key point detection model, process each first frame image P i Keypoint detection processing yields sixteen keypoints and eight feature dimensions.

[0105] Specifically, this includes: Step 31, transferring the first frame image P... i The keypoint detection model is input and processed to generate the corresponding first model output tensor.

[0106] The first model output tensor includes sixteen first output feature vectors; each first output feature vector includes the first predicted keypoint image coordinates and the first predicted keypoint type.

[0107] Here, as can be seen from the preceding text, the key point detection model of Embodiment 1 of the present invention performs the following steps on each first frame image P: i The first model outputs the tensor by detecting sixteen key points of eight cross-sectional structures of the upper heart (left ventricle, right ventricle, interventricular septum, posterior wall of left ventricle, left atrium, aortic valve annulus, aortic sinus, and ascending aorta).

[0108] Each first output feature vector in the first model output tensor corresponds to a key point; the first predicted key point type of the first output feature vector is the specific type of the corresponding key point, namely one of the sixteen key point types (first key point type of left ventricle, second key point type of left ventricle, first key point type of aortic valve annulus, second key point type of aortic valve annulus, first key point type of aortic sinus, second key point type of aortic sinus, first key point type of right ventricle, second key point type of right ventricle, first key point type of interventricular septum, second key point type of interventricular septum, first key point type of left ventricular posterior wall, second key point type of left ventricular posterior wall, first key point type of left atrium, second key point type of left atrium, first key point type of ascending aorta, second key point type of ascending aorta); the first predicted key point image coordinates of the first output feature vector are the corresponding key point in the current first frame image P. i Image coordinates on;

[0109] Step 32: Based on the image coordinates of the first predicted key points of the first and second left ventricles, calculate the length of the straight line segment connecting the key points of the first and second left ventricles, and denot it as the corresponding first left ventricle inner diameter d. vs,i Based on the image coordinates of the first predicted key points of the first and second aortic valve annulus key points, the length of the straight line segment connecting the key points of the first and second aortic valve annulus is calculated and denoted as the corresponding diameter d of the first aortic valve annulus. avr,i Based on the image coordinates of the first predicted key points of the first and second aortic sinus key points, the length of the straight line segment connecting the first and second aortic sinus key points is calculated and denoted as the corresponding diameter d of the first aortic sinus. as,i Based on the image coordinates of the first predicted key points of the first and second right ventricles, the length of the straight line segment connecting the key points of the first and second right ventricles is calculated and denoted as the corresponding first right ventricle inner diameter d. vd,i Based on the image coordinates of the first predicted key points of the first and second interventricular septum, the length of the straight line segment connecting the first and second interventricular septum key points is calculated and denoted as the corresponding first interventricular septum thickness d. IS,i Based on the image coordinates of the first predicted key points of the first and second left ventricular posterior walls, the length of the straight line segment connecting the key points of the first and second left ventricular posterior walls is calculated and denoted as the corresponding thickness d of the first left ventricular posterior wall. lvpw,i Based on the image coordinates of the first predicted key points of the first and second left atriums, the length of the straight line segment connecting the first and second left atriums is calculated and denoted as the corresponding anteroposterior diameter d of the first left atrium. la,i Based on the image coordinates of the first predicted key points of the first and second ascending aortas, the length of the straight line segment connecting the key points of the first and second ascending aortas is calculated and denoted as the corresponding inner diameter d of the first ascending aorta. aa,i ;

[0110] Here, with the image coordinates of sixteen key points for eight cross-sectional structures (left ventricle, right ventricle, interventricular septum, posterior wall of left ventricle, left atrium, aortic valve annulus, aortic sinus, and ascending aorta) known, the lengths of the eight straight-line segments connecting the two key points corresponding to each cross-sectional structure and calculating the length of the connecting line segment can be obtained: the first left ventricular diameter d vs,i Diameter d of the first aortic valve annulus avr,i Diameter d of the first aortic sinus as,i First right ventricular diameter d vd,i First interventricular septum thickness d IS,i Thickness d of the posterior wall of the first left ventricle lvpw,i 1. Anteroposterior diameter of the first left atrium d la,i The inner diameter of the first ascending aorta, d aa,i ;

[0111] Step 33: The sixteen first sub-feature map key points corresponding to the sixteen first output feature vectors are used to form the sixteen key points; the lengths of the eight line segments connected by the eight lines are used to form the eight feature dimensions.

[0112] The sixteen key points include the first left ventricle key point, the second left ventricle key point, the first aortic valve annulus key point, the second aortic valve annulus key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricle key point, the second right ventricle key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricle posterior wall key point, the second left ventricle posterior wall key point, the first left atrium key point, the second left atrium key point, the first ascending aorta key point, and the second ascending aorta key point.

[0113] The eight characteristic dimensions include the first left ventricular diameter d. vs,i Diameter d of the first aortic valve annulus avr,i Diameter d of the first aortic sinus as,i First right ventricular diameter d vd,i First interventricular septum thickness d IS,i Thickness d of the posterior wall of the first left ventricle lvpw,i 1. Anteroposterior diameter of the first left atrium d la,i The inner diameter of the first ascending aorta, d aa,i .

[0114] Step 4, based on all first left ventricular diameters d vs,i The first frame image sequence is processed by filtering diastolic and systolic frame images to obtain the corresponding first diastolic frame image and first systolic frame image; and based on the diameter d of all first aortic valve annulus... avr,i and the diameter d of the first aortic sinus as,iThe first frame image sequence is processed by filtering mid-systolic ventricular systolic frames to obtain the corresponding first mid-systolic ventricular systolic frame image;

[0115] Specifically, this includes: Step 41, based on all first left ventricular diameters d vs,i The first frame image sequence is processed by filtering diastolic and systolic frame images to obtain the corresponding first diastolic frame image and first systolic frame image;

[0116] Specifically, it includes: Step 411, constructing a two-dimensional coordinate system with the left ventricular diameter as the vertical axis Y and time as the horizontal axis X, denoted as the corresponding left ventricular diameter-time coordinate system;

[0117] Step 412, on the left ventricular diameter-time coordinate system, with each first left ventricular diameter d vs,i The plotting point is defined by the vertical axis coordinates, with the current first left ventricular diameter d as the reference. vs,i The corresponding first frame timestamp T i The first plotting point is obtained by marking the plotting point with the x-axis coordinate.

[0118] Step 413: Connect the first to the last first measurement points sequentially to obtain the corresponding first curve;

[0119] Here, the first curve obtained is actually a curve showing the change of the left ventricular diameter over time;

[0120] Step 414: Perform median filtering on the first curve to obtain the corresponding second curve;

[0121] Here, the first curve obtained under normal circumstances may have many spikes, so it needs to be filtered to make the curve smoother; Embodiment 1 of the present invention filters the first curve based on median filtering to obtain the corresponding second curve;

[0122] Step 415: Perform the most significant peak point filtering process on the second curve to obtain the corresponding first most significant peak point; extract the horizontal axis coordinate of the first most significant peak point as the corresponding first time point; and select the first frame timestamp T with the smallest time interval from the first time point in the first frame image sequence. i The corresponding first frame image P i As the corresponding first diastolic frame image;

[0123] Step 416: Flip the second curve vertically to obtain the corresponding third curve; perform the most significant peak point selection process on the third curve to obtain the corresponding second most significant peak point; extract the horizontal axis coordinate of the second most significant peak point as the corresponding second time point; and take the first frame timestamp T with the smallest time interval from the second time point in the first frame image sequence. iThe corresponding first frame image P i As the corresponding first contraction phase frame image;

[0124] Here, the most significant peak point screening process performed in step 415 and the current step 416 of Embodiment 1 of the present invention is the same process flow, and the specific steps of the most significant peak point screening process include:

[0125] Step A1: Take the second or third curve that has been processed for the most significant peak point screening as the corresponding current curve; and mark each peak point of the current curve as the corresponding first peak point;

[0126] Step A2: Traverse each first peak point. During traversal, record the currently traversed first peak point as the corresponding current peak point. Draw a straight line parallel to the horizontal axis X through the current peak point on the left ventricular diameter-time coordinate system, and record it as the corresponding first straight line. Record the intersection of the first straight line with any descending edge of the current curve to the left of the current peak point as the corresponding first descending edge intersection point, and the intersection of the first straight line with any ascending edge of the current curve to the right of the current peak point as the corresponding first ascending edge intersection point. Count the number of first descending edge intersection points and first ascending edge intersection points to generate the corresponding first descending edge intersection point count and first ascending edge intersection point count. Identify whether the number of first descending edge intersection points is 0. If it is, take the starting position of the current curve as the corresponding first starting position; otherwise, take the first descending edge intersection point to the left of the current peak point. The intersection point is taken as the corresponding first starting position; and it is identified whether the number of first rising edge intersection points is 0. If it is, the end position of the current curve is taken as the corresponding first ending position; otherwise, the first first rising edge intersection point to the right of the current peak point is taken as the corresponding first ending position; the minimum valley point between the first ending position and the current peak point on the current curve is taken as the corresponding first valley point, and the minimum valley point between the current peak point and the first ending position is taken as the corresponding second valley point. The larger value between the first and second valley points is taken as the corresponding current valley point; the difference between the vertical axis coordinates of the current peak point and the current valley point is calculated to generate the corresponding first peak-valley difference; at the end of the traversal, the maximum value is selected from all the obtained first peak-valley differences as the corresponding maximum peak-valley difference, and the first peak point corresponding to the maximum peak-valley difference is taken as the corresponding current most significant peak point;

[0127] Step A3: If the current curve is the second curve, then the current most significant peak point is output as the corresponding first most significant peak point; if the current curve is the third curve, then the current most significant peak point is output as the corresponding second most significant peak point.

[0128] Step 42, based on the diameter d of all first aortic valve annulus... avr,iand the diameter d of the first aortic sinus as,i The first frame image sequence is processed by filtering mid-systolic ventricular systolic frames to obtain the corresponding first mid-systolic ventricular systolic frame image;

[0129] Specifically, this includes: step 421, determining the diameter d of each first aortic valve annulus. avr,i The image coordinates of the two first predicted key points corresponding to the first and second aortic valve annulus key points are denoted as the corresponding first and second coordinates s. 1,i s 2,i ; and based on the first and second coordinates s 1,i s 2,i Generate a vector from the key point of the first aortic valve annulus to the key point of the second aortic valve annulus, denoted as the corresponding first vector s. 1->2,i ;s 1->2,i =s 2,i -s 1,i ;

[0130] Step 422, the diameter d of each first aortic sinus as,i The image coordinates of the two first predicted key points corresponding to the first and second aortic sinus key points are marked as the corresponding third and fourth coordinates s. 3,i s 4,i ; and based on the third and fourth coordinates s 3,i s 4,i Generate a vector from the key point of the first aortic sinus to the key point of the second aortic sinus, denoted as the corresponding second vector s. 3->4,i ;s 3->4,i =s 4,i -s 3,i ;

[0131] Step 423, the first aortic valve annulus diameter d is the same as that of the frame image index i. avr,i Diameter d of the first aortic sinus as,i The first vector s 1->2,i Second vector s 3->4,i Form the corresponding first data group;

[0132] Step 424: Perform a rationality analysis on each first data group to generate the corresponding first analysis results;

[0133] The first analysis results include both reasonable and unreasonable ones;

[0134] Specifically, this includes: step 4241, determining the diameter d of the first aortic valve annulus in the first data set. avr,i Diameter d of the first aortic sinus as,iThe absolute value of the diameter difference is calculated to generate the corresponding first absolute diameter difference; and it is identified whether the first absolute diameter difference is lower than the preset absolute diameter difference threshold; if so, the corresponding first check bit is set to success, otherwise the corresponding first check bit is set to failure.

[0135] In practical applications, the diameter difference between the aortic valve annulus diameter and the aortic sinus diameter should not be too large and should be within a reasonable range. Therefore, in Embodiment 1 of this invention, an absolute diameter difference threshold is preset, and this absolute diameter difference threshold is used as the first aortic valve annulus diameter d. avr,i Diameter d of the first aortic sinus as,i One of the constraints;

[0136] Step 4242, based on the first vector s of the first data group 1->2,i Second vector s 3->4,i Perform vector angle estimation to generate the corresponding first angle. It also identifies whether the first included angle is less than a preset small angle threshold; if so, it sets the corresponding second check bit as successful, otherwise it sets the corresponding second check bit as failed.

[0137] Here, in practical application scenarios, the diameter d of the first aortic valve annulus is... avr,i Diameter d of the first aortic sinus as,i The two corresponding line segments should be basically parallel. Therefore, in Embodiment 1 of the present invention, a small angle threshold close to 0 is preset, and this small angle threshold is used as the diameter d of the first aortic valve annulus. avr,i Diameter d of the first aortic sinus as,i Another constraint;

[0138] Step 4243, for the diameter d of the first aortic valve annulus in the first data set. avr,i The corresponding straight line segment and the diameter d of the first aortic sinus as,i The system identifies whether the corresponding line segments intersect; if so, it sets the corresponding third check bit as failure, otherwise it sets the corresponding third check bit as success.

[0139] Here, in practical application scenarios, the diameter d of the first aortic valve annulus is... avr,i Diameter d of the first aortic sinus as,i The two corresponding line segments should not intersect, that is, the two line segments should not have any intersection point. Therefore, in Embodiment 1 of the present invention, the diameter d of the first aortic valve annulus is taken as the condition that the two line segments cannot intersect. avr,i Diameter d of the first aortic sinus as,i Another constraint;

[0140] In the first embodiment of the present invention, the diameter d of the first aortic valve annulus in the first data group is...avr,i The corresponding straight line segment and the diameter d of the first aortic sinus as,i Multiple implementation methods are supported when identifying whether corresponding straight line segments intersect; one of these methods is to use the diameter d of the first aortic valve annulus. avr,i The two corresponding key points are denoted as points A and B. The diameter d of the first aortic sinus is... as,i The two corresponding key points are denoted as points C and D, and the vector from point A to point D is denoted as vector J. A->D Let the vector from point B to point D be denoted as vector J. B->D Let the vector from point C to point D be denoted as vector J. C->D and in (J A→D ×J C→D )·(J B→D ×J C→D When ) < 0, confirm the diameter d of the first aortic valve annulus. avr,i The corresponding straight line segment and the diameter d of the first aortic sinus as,i The corresponding straight line segments intersect at a point;

[0141] Step 4244: Identify the third check bit; if the third check bit fails, set the corresponding fourth check bit to fail; if the third check bit succeeds, then set the diameter d of the first aortic valve annulus in the first data group to succeed. avr,i Is the corresponding straight line segment within the diameter d of the first aortic sinus? as,i The left side of the corresponding straight line segment is identified; if it is, the corresponding fourth check bit is set to success; otherwise, the corresponding fourth check bit is set to failure.

[0142] Here, in practical application scenarios, the diameter d of the first aortic valve annulus is... avr,i The corresponding straight line segment should also be in the first aortic sinus region with a diameter d. as,i The corresponding straight line segment to the left, therefore, in Embodiment 1 of the present invention, under the premise of confirming that the two straight line segments do not intersect, further uses the diameter d of the first aortic valve annulus. avr,i The corresponding straight line segment must be within the diameter d of the first aortic sinus. as,i The left side of the corresponding straight line segment is taken as the diameter d of the first aortic valve annulus. avr,i Diameter d of the first aortic sinus as,i Another constraint;

[0143] In the first embodiment of the present invention, the diameter d of the first aortic valve annulus in the first data group is... avr,i Is the corresponding straight line segment within the diameter d of the first aortic sinus? as,i Multiple implementation methods are supported when identifying the left side of the corresponding straight line segment; one of these methods is: starting from the diameter d of the first aortic valve annulus. avr,iChoose one of the two corresponding key points as point E, and set the diameter d of the first aortic sinus as point E. as,i The two corresponding key points are denoted as points F and G, and the vector from point E to point G is denoted as vector J. E->G The vector from point F to point G is denoted as vector J. F->G and in (J F→G ×J E→G When ) < 0, confirm the diameter d of the first aortic valve annulus. avr,i The corresponding straight line segment has a diameter d at the first aortic sinus. as,i The left side of the corresponding straight line segment;

[0144] Step 4245: If the first, second, third, and fourth check bits are all successful, then the corresponding first analysis result is set as reasonable; if the first, second, third, or fourth check bits are unsuccessful, then the corresponding first analysis result is set as unreasonable.

[0145] Here, if the first, second, third, and fourth check bits are all successful, it indicates that the diameter d of the first aortic valve annulus in the current first data set is... avr,i Diameter d of the first aortic sinus as,i If all four constraints are met, then setting the corresponding first analysis result is reasonable; conversely, if any of the first, second, third, or fourth checkpoints fails, it indicates that the diameter d of the first aortic valve annulus in the current first data group is... avr,i Diameter d of the first aortic sinus as,i Since not all four constraints were met, the corresponding first analysis result was set to failure.

[0146] Step 425: For all first data groups whose first analysis results are reasonable, perform discrete data group identification to obtain multiple first discrete data groups;

[0147] Specifically, this includes: step 4251, recording the first data group whose first analysis result is reasonable as the corresponding second data group;

[0148] Step 4252, based on the first aortic valve annulus diameter d of all second data sets. avr,i Form a corresponding first diameter set; and determine the diameter d of the first aortic valve annulus in the first diameter set. avr,i The first quantity N1 is generated by statistically analyzing the number of diameters; the first mean μ1 and the first standard deviation σ1 are calculated from the mean and standard deviation of the first diameter set; the first limiting range is constructed based on the first mean μ1 and the first standard deviation σ1; and the diameter d of the first aortic valve annulus is set as follows. avr,i The second data group that exceeds the first limit range is denoted as the corresponding first discrete data group;

[0149] in, The lower limit of the first limiting range is (μ1-α1*σ1), and the upper limit is (μ1+α1*σ1), where α1 is a preset coefficient;

[0150] Here, in Embodiment 1 of the present invention, the discrete first aortic valve annulus diameter d is based on the overall standard deviation. avr,i The identification process is performed, and the second data set containing discrete aortic valve annulus diameters is recorded as the corresponding first discrete data set.

[0151] Step 4253, based on the diameter d of the first aortic sinus of all second data groups. as,i Form a corresponding second diameter set; and determine the diameter d of the first aortic sinus in the second diameter set. as,i The number of samples is statistically analyzed to generate a corresponding second quantity N2; the mean and standard deviation of the second diameter set are calculated to obtain the corresponding second mean μ2 and second standard deviation σ2; and a corresponding second amplitude limiting range is constructed based on the second mean μ2 and second standard deviation σ2; and the diameter d of the first aortic sinus is set... as,i The second data group that exceeds the second limit range is denoted as the corresponding first discrete data group;

[0152] in, The lower limit of the second limiting range is (μ2-α2*σ2), and the upper limit is (μ2+α2*σ2), where α2 is a preset coefficient;

[0153] Here, in Embodiment 1 of the present invention, the discrete first aortic sinus diameter d is based on the overall standard deviation. as,i The identification process is performed, and the second data set containing discrete aortic sinus diameters is recorded as the corresponding first discrete data set.

[0154] Step 426: Select the first frame image P from the first frame image sequence that corresponds to each first data group whose first analysis result is unreasonable. i and the first frame image P corresponding to each first discrete data group i All of these are recorded as the corresponding first abnormal frame image;

[0155] Step 427: Perform abnormal frame image data correction processing on each first abnormal frame image;

[0156] Specifically, it includes: step 4271, taking any first abnormal frame image in the first frame image sequence as the corresponding current abnormal frame image; and taking the previous and next non-abnormal frame images in the first frame image sequence as the corresponding previous normal frame image and next normal frame image.

[0157] Step 4272, use the first left ventricular diameter d corresponding to the previous and subsequent normal frame images. vs,i Diameter d of the first aortic valve annulus avr,iDiameter d of the first aortic sinus as,i First right ventricular diameter d vd,i First interventricular septum thickness d IS,i Thickness d of the posterior wall of the first left ventricle lvpw,i 1. Anteroposterior diameter of the first left atrium d la,i and the inner diameter of the first ascending aorta d aa,i The first left ventricular diameter d corresponding to the current abnormal frame image vs,i Diameter d of the first aortic valve annulus avr,i Diameter d of the first aortic sinus as,i First right ventricular diameter d vd,i First interventricular septum thickness d IS,i Thickness d of the posterior wall of the first left ventricle lvpw,i 1. Anteroposterior diameter of the first left atrium d la,i and the inner diameter of the first ascending aorta d aa,i Interpolation reset is performed using linear interpolation.

[0158] Specifically, the first size data corresponding to the preceding and following normal frame images are used as the corresponding preceding and following size data q. pre q aft And calculate the first frame timestamp T of the current abnormal frame image. i The timestamp T of the first frame of the preceding and following normal frame images i The time interval generates the corresponding time interval Δt before and after. pre , △t aft Based on the front and rear dimension data q pre q aft and the time interval Δt before and after pre , △t aft Linear interpolation is performed to obtain the corresponding first interpolation size data q. * ; and set the first size data corresponding to the current abnormal frame image as the corresponding first interpolation size data q. * ;

[0159] The first dimension data is the first left ventricular diameter d. vs,i Or the diameter d of the first aortic valve annulus avr,i Or the diameter d of the first aortic sinus as,i Or the first right ventricular diameter d vd,i Or the thickness of the first interventricular septum d IS,i Or the thickness d of the posterior wall of the first left ventricle lvpw,i Or the anteroposterior diameter of the first left atrium d la,i Or the inner diameter of the first ascending aorta d aa,i ;

[0160] Here, in Embodiment 1 of the present invention, the eight feature dimensions of the current first abnormal frame image are corrected and reset using the eight feature dimensions of the preceding and following normal frame images;

[0161] Step 4273: Using the first predicted keypoint image coordinates corresponding to the first left ventricle keypoint, second left ventricle keypoint, first aortic valve annulus keypoint, second aortic valve annulus keypoint, first aortic sinus keypoint, second aortic sinus keypoint, first right ventricle keypoint, second right ventricle keypoint, first interventricular septum keypoint, second interventricular septum keypoint, first left ventricular posterior wall keypoint, second left ventricular posterior wall keypoint, first left atrium keypoint, second left atrium keypoint, first ascending aorta keypoint, and second ascending aorta keypoint in the preceding and following normal frame images, the keypoints are used to... The image coordinates of the first predicted key points corresponding to the current abnormal frame image, namely the first left ventricle key point, the second left ventricle key point, the first aortic valve annulus key point, the second aortic valve annulus key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricle key point, the second right ventricle key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricle posterior wall key point, the second left ventricle posterior wall key point, the first left atrium key point, the second left atrium key point, the first ascending aorta key point, and the second ascending aorta key point, are interpolated and reset using linear interpolation.

[0162] Specifically, the coordinates of the first keypoint image corresponding to each of the preceding and following normal frame images are used as the corresponding preceding and following keypoint image coordinates k. pre k aft And calculate the first frame timestamp T of the current abnormal frame image. i The timestamp T of the first frame of the preceding and following normal frame images i The time interval generates the corresponding time interval Δt before and after. pre , △t aft Based on the image coordinates k of the front and rear key points. pre k aft and the time interval Δt before and after pre , △t aft The first interpolation coordinate k is obtained by performing linear interpolation calculation. * And set the first key point image coordinates corresponding to the current abnormal frame image as the corresponding first interpolation coordinate k. * ;

[0163] Wherein, the first key point image coordinates are the first predicted key point image coordinates of the first left ventricular key point, or the first predicted key point image coordinates of the second left ventricular key point, or the first predicted key point image coordinates of the first aortic valve annulus key point, or the first predicted key point image coordinates of the second aortic valve annulus key point, or the first predicted key point image coordinates of the first aortic sinus key point, or the first predicted key point image coordinates of the second aortic sinus key point, or the first predicted key point image coordinates of the first right ventricular key point, or the first predicted key point image coordinates of the second right ventricular key point, or the first predicted key point image coordinates of the first interventricular septum key point, or the first predicted key point image coordinates of the second interventricular septum key point, or the first predicted key point image coordinates of the first left ventricular posterior wall key point, or the first predicted key point image coordinates of the second left ventricular posterior wall key point, or the first predicted key point image coordinates of the first left atrium key point, or the first predicted key point image coordinates of the second left atrium key point, or the first predicted key point image coordinates of the first ascending aorta key point, or the first predicted key point image coordinates of the second ascending aorta key point.

[0164] Here, in Embodiment 1 of the present invention, the image coordinates of the sixteen key points of the preceding and following normal frame images are used to correct and reset the image coordinates of the sixteen key points of the current first abnormal frame image;

[0165] Step 428: If the abnormal frame image data correction process is successful, then process each first frame image P... i The corresponding first aortic valve annulus diameter d avr,i and the diameter d of the first aortic sinus as,i The first sum is obtained by adding the sums; the maximum value is selected from all the first sums as the maximum sum; and the first frame image P corresponding to the maximum sum is then processed. i Output as the corresponding mid-contraction frame image of the first ventricle.

[0166] Here, the present invention implements a method that uses the sum of the diameters of the aortic valve annulus and the aortic sinus in each frame image, i.e., the first sum data, as the screening criterion for mid-ventricular systolic frame images, and selects the first frame image P corresponding to the largest first sum data. i The screening result is the mid-systolic frame image of the first ventricle.

[0167] Step 5: On the first diastolic frame image, key points and feature dimensions are marked for the right ventricular diameter, interventricular septal thickness, left ventricular diameter, and left ventricular posterior wall thickness; on the first systolic frame image, key points and feature dimensions are marked for the left ventricular diameter; and on the first mid-systolic frame image, key points and feature dimensions are marked for the left atrial anteroposterior diameter, aortic valve annulus diameter, aortic sinus diameter, and ascending aortic diameter.

[0168] Specifically, this includes: Step 51, performing key point marking and feature size annotation on the right ventricular diameter, interventricular septal thickness, left ventricular diameter, and left ventricular posterior wall thickness on the first diastolic frame image;

[0169] Specifically, this includes: Step 511, on the first diastolic frame image, setting the color of two pixels corresponding to the image coordinates of the two first predicted key points of the first and second right ventricular key points to a preset first color; and drawing a connecting line segment between the two pixels corresponding to the first and second right ventricular key points based on the first color to obtain the corresponding first line segment; and setting the first right ventricular diameter d corresponding to the first diastolic frame image to... vd,i The product of the first line segment and the preset unit scale is used as the corresponding second right ventricular diameter; and the display label information of the first line segment is set as the corresponding second right ventricular diameter.

[0170] Here, the first right ventricular diameter d obtained in the aforementioned steps vd,i The unit is an image segment unit, not an actual length unit. The pre-set unit scale in Embodiment 1 of this invention is a conversion scale that converts image segment units to actual length units. The first right ventricular diameter d... vd,i Multiplying by the unit scale will give you the actual length of the right ventricular diameter;

[0171] Step 512: On the first diastolic frame image, set the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second interventricular septum to a preset second color; and draw the corresponding second line segment by connecting the two pixels corresponding to the first and second interventricular septum key points based on the second color; and set the first interventricular septum thickness d corresponding to the first diastolic frame image to a preset second color. IS,i The product of the product with the unit scale is used as the corresponding second interventricular septum thickness; and the display label information of the second line segment is set as the corresponding second interventricular septum thickness;

[0172] Here, the first interventricular septum thickness d obtained in the aforementioned steps IS,i The unit is an image segment unit, not an actual length unit. The thickness d of the first interventricular septum... IS,i Multiplying by the unit scale will give you the actual thickness of the interventricular septum.

[0173] Step 513: On the first diastolic frame image, set the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second left ventricular key points to a preset third color; and draw the corresponding third line segment by connecting the two pixels corresponding to the first and second left ventricular key points based on the third color; and set the first left ventricular intraventricular diameter d corresponding to the first diastolic frame image to a preset third color. vs,i The product of the product with the unit scale is used as the corresponding second left ventricular diameter; and the display label information of the third line segment is set as the corresponding second left ventricular diameter;

[0174] Here, the first left ventricular diameter d obtained in the aforementioned steps vs,i The unit is an image segment unit, not an actual length unit. The first left ventricular diameter d... vs,i Multiplying by the unit scale will give you the actual length of the left ventricular diameter;

[0175] Step 514: On the first diastolic frame image, set the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second left ventricular posterior wall key points to a preset fourth color; and draw the corresponding fourth line segment by connecting the two pixels corresponding to the first and second left ventricular posterior wall key points based on the fourth color; and set the thickness d of the first left ventricular posterior wall corresponding to the first diastolic frame image to... lvpw,i The product of the product with the unit scale is taken as the corresponding second left ventricular posterior wall thickness; and the display label information of the fourth line segment is set as the corresponding second left ventricular posterior wall thickness.

[0176] Here, the first left ventricular posterior wall thickness d obtained in the aforementioned steps lvpw,i The unit is an image segment unit, not an actual length unit. The thickness d of the posterior wall of the first left ventricle... lvpw,i Multiplying by the unit scale will give you the actual thickness of the left ventricular posterior wall.

[0177] Step 52: Mark key points and feature dimensions of the left ventricular diameter on the first systolic frame image;

[0178] Specifically, this includes: on the first systolic frame image, setting the color of two pixels corresponding to the image coordinates of two first predicted key points of the first and second left ventricles to a preset fifth color; drawing a corresponding fifth line segment by connecting the two pixels corresponding to the first and second left ventricles based on the fifth color; and setting the first left ventricular intraventricular diameter d corresponding to the first systolic frame image to a preset fifth color. vs,i The product of the product with the unit scale is taken as the corresponding third left ventricular diameter; and the display label information of the fifth line segment is set as the corresponding third left ventricular diameter;

[0179] Here, the first left ventricular diameter d obtained in the aforementioned steps vs,iThe unit is an image segment unit, not an actual length unit. The first left ventricular diameter d... vs,i Multiplying by the unit scale will give you the actual length of the left ventricular diameter;

[0180] Step 53: On the first ventricular mid-systolic frame image, key points and feature dimensions are marked and annotated for the left atrial anteroposterior diameter, aortic valve annulus diameter, aortic sinus diameter, and ascending aortic internal diameter.

[0181] Specifically, this includes: Step 531, on the mid-systolic frame image of the first ventricle, setting the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second left atrium key points to a preset sixth color; and drawing a connecting line segment between the two pixels corresponding to the first and second left atrium key points based on the sixth color to obtain the corresponding sixth line segment; and setting the anteroposterior diameter d of the first left atrium corresponding to the mid-systolic frame image of the first ventricle. la,i The product of the product with the unit scale is used as the corresponding second left atrial anteroposterior diameter; and the display label information of the sixth line segment is set as the corresponding second left atrial anteroposterior diameter;

[0182] Here, the first left atrial anteroposterior diameter d obtained in the aforementioned steps la,i The unit is an image segment unit, not an actual length unit. The anteroposterior diameter d of the first left atrium... la,i Multiplying by the unit scale will give you the actual length of the anteroposterior diameter of the left atrium;

[0183] Step 532: On the first ventricular mid-systolic frame image, set the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second aortic valve annulus key points to a preset seventh color; and draw the corresponding seventh line segment by connecting the two pixels corresponding to the first and second aortic valve annulus key points based on the seventh color; and set the diameter d of the first aortic valve annulus corresponding to the first ventricular mid-systolic frame image to a preset seventh color. avr,i The product of the product with the unit scale is used as the corresponding second aortic valve annulus diameter; and the display annotation information of the seventh line segment is set as the corresponding second aortic valve annulus diameter;

[0184] Here, the diameter d of the first aortic valve annulus obtained in the aforementioned steps avr,i The unit is an image segment unit, not an actual length unit. The diameter d of the first aortic valve annulus is... avr,i Multiplying by the unit scale gives the actual length of the aortic valve annulus diameter;

[0185] Step 533: On the mid-systolic frame image of the first ventricle, set the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second aortic sinus key points to a preset eighth color; and draw the corresponding eighth line segment by connecting the two pixels corresponding to the key points of the first and second aortic sinus based on the eighth color; and set the diameter d of the first aortic sinus corresponding to the mid-systolic frame image of the first ventricle to the preset eighth color. as,i The product of the product with the unit scale is used as the corresponding second aortic sinus diameter; and the display label information of the eighth line segment is set as the corresponding second aortic sinus diameter;

[0186] Here, the diameter d of the first aortic sinus obtained in the aforementioned steps as,i The unit is an image segment unit, not an actual length unit. The diameter d of the first aortic sinus is... as,i Multiplying by the unit scale gives the actual length of the aortic sinus diameter;

[0187] Step 534: On the mid-systolic frame image of the first ventricle, set the color of the two pixels corresponding to the image coordinates of the two first predicted key points of the first and second ascending aortas to a preset ninth color; and draw the corresponding ninth line segment by connecting the two pixels corresponding to the key points of the first and second ascending aortas based on the ninth color; and set the inner diameter d of the first ascending aorta corresponding to the mid-systolic frame image of the first ventricle to a preset ninth color. aa,i The product of the product with the unit scale is used as the corresponding second ascending aortic inner diameter; and the display label information of the ninth line segment is set as the corresponding second ascending aortic inner diameter.

[0188] Here, the first ascending aortic inner diameter d obtained in the aforementioned steps aa,i The unit is an image segment unit, not an actual length unit. The first ascending aortic inner diameter d... aa,i Multiplying by the unit scale gives the actual length of the ascending aorta's internal diameter.

[0189] Step 6: Output the first diastolic frame image, the first systolic frame image, and the first mid-systolic frame image of the heart as the processing results of the feature dimensions of this cardiac ultrasound video measurement after the key point marking and feature size annotation processing is completed.

[0190] Figure 2 This is a module structure diagram of a processing device based on the characteristic size of cardiac ultrasound video measurement provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the method of the embodiments of the present invention, or it can be a device connected to the aforementioned terminal device or server to implement the method of the embodiments of the present invention. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2As shown, the device includes: a receiving module 201, an image framing module 202, a key point detection module 203, a framed image filtering module 204, an image annotation module 205, and an output module 206.

[0191] The receiving module 201 is used to receive cardiac ultrasound video as the corresponding first video.

[0192] The image framing module 202 is used to extract video frame images from the first video to generate a corresponding first frame image sequence; the first frame image sequence consists of multiple first frame images P i The images are arranged chronologically, with each first frame corresponding to a first frame timestamp T. i , where the frame image index i ≥ 1.

[0193] The key point detection module 203 is used to perform key point detection on each first frame image P based on a preset key point detection model. i Keypoint detection processing yielded sixteen keypoints and eight feature dimensions; the eight feature dimensions include the first left ventricular diameter d. vs,i Diameter d of the first aortic valve annulus avr,i and the diameter d of the first aortic sinus as,i .

[0194] The framed image filtering module 204 is used to filter images based on all first left ventricular diameters d. vs,i The first frame image sequence is processed by filtering diastolic and systolic frame images to obtain the corresponding first diastolic frame image and first systolic frame image; and based on the diameter d of all first aortic valve annulus... avr,i and the diameter d of the first aortic sinus as,i The first frame image sequence is processed by filtering mid-systolic frames to obtain the corresponding first mid-systolic frame image.

[0195] The image annotation module 205 is used to perform key point marking and feature size annotation processing on the right ventricular diameter, interventricular septum thickness, left ventricular diameter and left ventricular posterior wall thickness on the first diastolic frame image; and to perform key point marking and feature size annotation processing on the left ventricular diameter on the first systolic frame image; and to perform key point marking and feature size annotation processing on the left atrial anterior-posterior diameter, aortic valve annulus diameter, aortic sinus diameter and ascending aortic diameter on the first ventricular mid-systolic frame image.

[0196] The output module 206 is used to output the first diastolic frame image, the first systolic frame image, and the first ventricular mid-systolic frame image, which have completed the key point marking and feature size annotation processing, as the processing result of the feature size of this cardiac ultrasound video measurement.

[0197] The present invention provides a processing device based on the characteristic size of cardiac ultrasound video measurement, which can execute the method steps in the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0198] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the receiving module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0199] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0200] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions described above can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The computer-readable storage medium described above can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs), etc.

[0201] Figure 3 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be the aforementioned terminal device or server, or it can be a terminal device or server connected to the aforementioned terminal device or server that implements the method of the embodiments of the present invention. Figure 3 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the methods and processes provided in the above embodiments of the present invention. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0202] exist Figure 3The system bus mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0203] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0204] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to execute the methods and processes provided in the above embodiments.

[0205] This invention also provides a chip for executing instructions, which is used to execute the methods and processes provided in the above embodiments.

[0206] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing characteristic dimensions based on cardiac ultrasound video. First, frame images are extracted from the cardiac ultrasound video. Then, based on a keypoint detection model, keypoint detection is performed on each frame image to obtain sixteen keypoints and eight characteristic dimensions corresponding to eight cross-sectional structures (left ventricle, right ventricle, interventricular septum, left ventricular posterior wall, left atrium, aortic valve annulus, aortic sinus, and ascending aorta). Based on the left ventricular characteristic dimension (left ventricular diameter), diastolic and systolic frames are selected from multiple frames. The characteristic dimensions of the aortic valve annulus and aortic sinus (aortic diameter) are then used to select diastolic and systolic frames. Based on the annular diameter and aortic sinus diameter, mid-systolic frames are selected from multiple images. Key points and characteristic dimensions are marked on the diastolic frames for the right ventricular diameter, interventricular septal thickness, left ventricular diameter, and left ventricular posterior wall thickness. Key points and characteristic dimensions are marked on the systolic frames for the left ventricular diameter. Key points and characteristic dimensions are marked on the mid-systolic frames for the left atrial anteroposterior diameter, aortic annular diameter, aortic sinus diameter, and ascending aortic diameter. Finally, the diastolic, systolic, and mid-systolic frames with marked key points and characteristic dimensions are output as the processing result. This invention solves the problem of requiring manual intervention in conventional processing methods, improves measurement efficiency, and ensures the stability of measurement quality.

[0207] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0208] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0209] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A processing method for measuring a characteristic dimension based on a cardiac ultrasound video, characterized in that, The method comprises: receiving a cardiac ultrasound video as a corresponding first video; The first video is subjected to video frame image extraction processing to generate a corresponding first frame image sequence; the first frame image sequence comprises a plurality of first frame images P i The first frame images are sequentially sorted and composed, and each first frame image corresponds to a first frame timestamp T i , and the frame image index i≥1. Based on a preset key point detection model, each of the first frame images P i The key point detection processing obtains corresponding sixteen key points and eight feature sizes; the eight feature sizes include a first left ventricular internal diameter d vs,i , a first aortic valve annulus diameter d avr,i , and a first aortic sinus diameter d as,i ; According to all the first left ventricular internal diameter d vs,i The first diastolic frame image and the first systolic frame image corresponding to the first frame image sequence are obtained by diastolic and systolic frame image screening processing; and according to all the first aortic valve ring diameter d avr,i And the first aortic sinus diameter d as,i The first ventricular systolic mid-frame image corresponding to the first frame image sequence is obtained by ventricular systolic mid-frame image screening processing. performing key point marking and feature size labeling processing on the right ventricular internal diameter, the interventricular septal thickness, the left ventricular internal diameter and the left ventricular posterior wall thickness on the first diastolic phase frame image; and performing key point marking and feature size labeling processing on the left ventricular internal diameter on the first systolic phase frame image; and performing key point marking and feature size labeling processing on the left atrial anteroposterior diameter, the aortic valve annulus diameter, the aortic sinus diameter and the ascending aorta internal diameter on the first ventricular systolic phase frame image; outputting the first diastolic phase frame image, the first systolic phase frame image and the first ventricular systolic phase frame image on which the key point marking and feature size labeling processing are completed as the processing result of measuring the feature size of the cardiac ultrasound video.

2. The processing method for measuring the feature size based on the cardiac ultrasound video according to claim 1, characterized in that the key point detection model is realized based on a Keypoint-RCNN model; the key point detection model comprises a first feature extraction network, a first region proposal network, a first focus region alignment network, a first full connection network, a second full connection network, a third full connection network, a first key point head network and a first fusion module; the first feature extraction network is connected with the first region proposal network; the first region proposal network is connected with the first focus region alignment network; the first focus region alignment network is connected with the first full connection network and the first key point head network respectively; the first full connection network is connected with the second full connection network and the third full connection network respectively; the first fusion module is connected with the first key point head network, the second full connection network and the third full connection network respectively; the first feature extraction network is a Resnet50 network; the first key point head network comprises a plurality of convolutional neural networks; the first feature extraction network is used for performing backbone feature extraction processing on the model input image to generate a corresponding first feature map; the first region proposal network is used for performing target region pre-recognition processing on the first feature map to generate a plurality of first pre-frames; the first pre-frame comprises a first pre-frame offset position and a first pre-frame size; the first pre-frame offset position is the offset position of the first pre-frame in the first feature map; the target region comprises sixteen types of target regions, which are two key regions of the left ventricle, two key regions of the aortic valve annulus, two key regions of the aortic sinus, two key regions of the right ventricle, two key regions of the interventricular septum, two key regions of the left ventricular posterior wall, two key regions of the left atrium and two key regions of the ascending aorta; the number of the first pre-frames is greater than 16; the first focus region alignment network is used for performing pre-frame merging processing on all the first pre-frames to generate sixteen first final frames, and extracting the sub-feature maps of each first final frame on the first feature map as corresponding first sub-feature maps; the first final frame comprises a first final frame offset position and a first final frame size; The first final selection box offset position is an offset position of the first final selection box in the first feature map; sixteen first sub-feature maps correspond to region features of sixteen key points, and specifically, left ventricular first key point region feature maps, left ventricular second key point region feature maps, aortic valve ring first key point region feature maps, aortic valve ring second key point region feature maps, aortic sinus first key point region feature maps, aortic sinus second key point region feature maps, right ventricular first key point region feature maps, right ventricular second key point region feature maps, interventricular septum first key point region feature maps, interventricular septum second key point region feature maps, left ventricular posterior wall first key point region feature maps, left ventricular posterior wall second key point region feature maps, left atrium first key point region feature maps, left atrium second key point region feature maps, ascending aorta first key point region feature maps, and ascending aorta second key point region feature maps; The first full connection network is used for performing global vector conversion on each first sub-feature map to generate a corresponding first global vector, and performing full connection calculation on each first global vector to generate a corresponding first full connection vector; the second full connection network is used for performing regression prediction on an offset position of the current first sub-feature map in the model input image according to the first full connection vector corresponding to each first sub-feature map to generate a corresponding first sub-feature map offset position; and the third full connection network is used for performing key point classification prediction on the current first sub-feature map according to the first full connection vector corresponding to each first sub-feature map to generate sixteen first type labels; the value of the first sub-feature map type label includes 1 or 0; the sixteen first type labels correspond to sixteen key point types, and specifically, left ventricular first key point types, left ventricular second key point types, aortic valve ring first key point types, aortic valve ring second key point types, aortic sinus first key point types, aortic sinus second key point types, right ventricular first key point types, right ventricular second key point types, interventricular septum first key point types, interventricular septum second key point types, left ventricular posterior wall first key point types, left ventricular posterior wall second key point types, left atrium first key point types, left atrium second key point types, ascending aorta first key point types, and ascending aorta second key point types; Only one label in the sixteen first type labels has a value of 1, and the rest are all 0; The first key point head network is used for key point position identification on the sixteen first sub-feature maps to obtain sixteen first sub-feature map key points; the first sub-feature map key point corresponds to a first key point offset position; the sixteen first sub-feature map key points include a first left ventricular key point, a second left ventricular key point, a first aortic valve ring key point, a second aortic valve ring key point, a first aortic sinus key point, a second aortic sinus key point, a first right ventricular key point, a second right ventricular key point, a first interventricular septum key point, a second interventricular septum key point, a first left ventricular posterior wall key point, a second left ventricular posterior wall key point, a first left atrial key point, a second left atrial key point, a first ascending aorta key point, and a second ascending aorta key point; The first fusion module is used for key point absolute position prediction according to the first terminal bounding box offset position corresponding to each first sub-feature map and the first key point offset position corresponding to each first sub-feature map to generate a corresponding first predicted key point image coordinate; and selecting a key point type corresponding to a first type label with a value of 1 from the sixteen first type labels corresponding to each first sub-feature map as a corresponding first predicted key point type; and composing a corresponding output feature vector from the first predicted key point image coordinate and the first predicted key point type corresponding to each first sub-feature map; and composing a corresponding model output tensor from the sixteen output feature vectors obtained and outputting.

3. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 2, characterized in that, The preset key point detection model is used to detect the key points of each first frame image P i The key point detection model is used to detect the key points of each first frame image P i The key point detection model is used to detect the key points of each first frame image P i The key point detection model is used to detect the key points of each first frame image P i The key point detection model is used The first frame image P i inputting the key point detection model for processing to generate a corresponding first model output tensor; the first model output tensor includes sixteen first output feature vectors; the first output feature vector includes the first predicted key point image coordinates and the first predicted key point type; According to the first predicted key point image coordinates of the first and second left ventricular key points, the length of the straight line connecting segment between the first and second left ventricular key points is calculated, denoted as the corresponding first left ventricular internal diameter d vs,i ; and according to the first predicted key point image coordinates of the first and second aortic valve annulus key points, the length of the straight line connecting segment between the first and second aortic valve annulus key points is calculated, denoted as the corresponding first aortic valve annulus diameter d avr,i ; and according to the first predicted key point image coordinates of the first and second aortic sinus key points, the length of the straight line connecting segment between the first and second aortic sinus key points is calculated, denoted as the corresponding first aortic sinus diameter d as,i ; and according to the first predicted key point image coordinates of the first and second right ventricular key points, the length of the straight line connecting segment between the first and second right ventricular key points is calculated, denoted as the corresponding first right ventricular internal diameter d vd,i ; and according to the first predicted key point image coordinates of the first and second interventricular septum key points, the length of the straight line connecting segment between the first and second interventricular septum key points is calculated, denoted as the corresponding first interventricular septum thickness d IS,i ; and according to the first predicted key point image coordinates of the first and second left ventricular posterior wall key points, the length of the straight line connecting segment between the first and second left ventricular posterior wall key points is calculated, denoted as the corresponding first left ventricular posterior wall thickness d lvpw,i ; and according to the first predicted key point image coordinates of the first and second left atrial key points, the length of the straight line connecting segment between the first and second left atrial key points is calculated, denoted as the corresponding first left atrial anteroposterior diameter d la,i ; and according to the first predicted key point image coordinates of the first and second ascending aorta key points, the length of the straight line connecting segment between the first and second ascending aorta key points is calculated, denoted as the corresponding first ascending aorta internal diameter d aa,i ; The sixteen key points are composed of the sixteen first sub-feature map key points corresponding to the sixteen first output feature vectors obtained; the sixteen key points include the first left ventricular key point, the second left ventricular key point, the first aortic valve ring key point, the second aortic valve ring key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricular key point, the second right ventricular key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricular posterior wall key point, the second left ventricular posterior wall key point, the first left atrial key point, the second left atrial key point, the first ascending aorta key point, and the second ascending aorta key point; corresponding to the eight characteristic dimensions; the eight characteristic dimensions include the first left ventricular internal diameter d vs,i , the first aortic valve annulus diameter d avr,i , the first aortic sinus diameter d as,i , the first right ventricular internal diameter d vd,i , the first interventricular septal thickness d IS,i , the first left ventricular posterior wall thickness d lvpw,i , the first left atrial anteroposterior diameter d la,i , the first ascending aorta internal diameter d aa,i .

4. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 1, characterized in that, said according to all the first left ventricular internal diameter d vs,i The diastolic and systolic frame image screening processing is performed on the first frame image sequence to obtain corresponding first diastolic frame image and first systolic frame image, specifically including: A two-dimensional coordinate system is constructed with a left ventricular internal diameter as a vertical axis Y and time as a horizontal axis X, denoted as a corresponding left ventricular internal diameter-time coordinate system; In the left ventricular internal diameter-time coordinate system, each of the first left ventricular internal diameters d vs,i For the longitudinal axis coordinate of the mapping point, the current first left ventricular internal diameter d vs,i The corresponding first frame timestamp T i The corresponding first mapping point is obtained by performing mapping point marking processing for the mapping point horizontal axis coordinate. The first to last first drawing points are sequentially connected to obtain a corresponding first curve; and the first curve is subjected to median filtering processing to obtain a corresponding second curve; The second curve is subjected to a most significant peak point screening process to obtain a corresponding first most significant peak point; and the abscissa coordinate of the first most significant peak point is extracted as a corresponding first time point; and the first frame timestamp T i The first frame image P i as the corresponding first diastolic frame image; flipping up and down the second curve to obtain a corresponding third curve; and performing a most significant peak point screening processing on the third curve to obtain a corresponding second most significant peak point; and extracting the abscissa coordinate of the second most significant peak point as a corresponding second time point; and taking the first frame timestamp T i corresponding to the first frame image P i as the corresponding first systolic frame image.

5. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 4, characterized in that, The most significant peak point screening processing specifically includes: The second curve or the third curve of the current most significant peak point screening processing is taken as a corresponding current curve; and each peak point of the current curve is marked as a corresponding first peak point; traversing each of the first peak points; when traversing, recording the first peak point being currently traversed as a corresponding current peak point; and drawing a straight line parallel to the horizontal axis X through the current peak point on the left ventricular internal diameter-time coordinate system as a corresponding first straight line; and recording the intersection point of the first straight line on the left side of the current peak point with any falling edge of the current curve as a corresponding first falling edge intersection point, and recording the intersection point of the first straight line on the right side of the current peak point with any rising edge of the current curve as a corresponding first rising edge intersection point, and respectively counting the number of the first falling edge intersection points and the first rising edge intersection points to generate a corresponding first falling edge intersection point number and a first rising edge intersection point number; and identifying whether the first falling edge intersection point number is 0, if yes, recording the starting position of the current curve as a corresponding first starting position, if not, recording the first falling edge intersection point on the left side of the current peak point as the corresponding first starting position; and identifying whether the first rising edge intersection point number is 0, if yes, recording the ending position of the current curve as a corresponding first ending position, if not, recording the first rising edge intersection point on the right side of the current peak point as the corresponding first ending position; and recording the minimum valley point between the first ending position and the current peak point on the current curve as a corresponding first valley point, recording the minimum valley point between the current peak point and the first ending position as a corresponding second valley point, and recording the larger value of the first and second valley points as a corresponding current valley point; and calculating the vertical axis coordinate difference between the current peak point and the current valley point to generate a corresponding first peak-valley difference; and when the traversal ends, selecting the maximum value from all the first peak-valley differences as a corresponding maximum peak-valley difference, and recording the first peak point corresponding to the maximum peak-valley difference as a corresponding current most significant peak point; if the current curve is the second curve, outputting the current most significant peak point as a corresponding first most significant peak point; if the current curve is the third curve, outputting the current most significant peak point as a corresponding second most significant peak point.

6. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 3, characterized in that, said according to all the first aortic valve ring diameter d avr,i and the first aortic sinus diameter d as,i The first frame image sequence is subjected to a ventricular systolic mid-frame image screening process to obtain a corresponding first ventricular systolic mid-frame image, specifically including: each of the first aortic valve annulus diameters d avr,i The two first predicted key point image coordinates corresponding to the first and second aortic valve annulus key points are marked as the first and second coordinates s 1,i , s 2,i ; and a vector from the first aortic valve annulus key point to the second aortic valve annulus key point is generated according to the first and second coordinates s 1,i , s 2,i , and is marked as the first vector s 1->2,i ; s 1->2,i = s 2,i - s 1,i ; each of the first aortic sinus portion diameters d as,i The two first predicted key point image coordinates of the corresponding first and second aortic sinus portion key points are marked as the corresponding third and fourth coordinates s 3,i , s 4,i ; and a vector from the first aortic sinus portion key point to the second aortic sinus portion key point is generated according to the third and fourth coordinates s 3,i , s 4,i , and is marked as the corresponding second vector s 3->4,i ; s 3->4,i = s 4,i - s 3,i ; The first aortic valve annulus diameter d is the same as that of the frame image index i. avr,i The diameter d of the first aortic sinus as,i The first vector s 1->2,i and the second vector s 3->4,i Form the corresponding first data group; performing rationality analysis on each of the first data sets to generate a corresponding first analysis result; and performing discrete data set identification on the first data sets with the first analysis result being reasonable to obtain a plurality of first discrete data sets; the first analysis result includes reasonable and unreasonable; corresponding to the first data group with the respective first analysis result being unreasonable in the first frame image sequence P i corresponding to the respective first discrete data group in the first frame image P i are all recorded as the corresponding first abnormal frame image; and the abnormal frame image data correction processing is performed on the respective first abnormal frame image; If the abnormal frame image data correction processing is successful, each of the first frame images P i The corresponding first aortic valve annulus diameter d avr,i And the first aortic sinus diameter d as,i The sum of the first aortic sinus diameter d i The maximum sum data is selected as the corresponding maximum sum data from all the obtained first sum data; and the first frame image P i The corresponding first mid-ventricular systolic phase frame image is output.

7. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 6, characterized in that, the performing rationality analysis on each of the first data sets to generate a corresponding first analysis result specifically includes: calculating the absolute value of the diameter difference between the first aortic valve annulus diameter d avr,i and the first aortic sinus diameter d as,i to generate a corresponding first absolute diameter difference; and identifying whether the first absolute diameter difference is lower than a preset absolute diameter difference threshold; if yes, setting a corresponding first check bit as success, and if not, setting the corresponding first check bit as failure; performing vector angle estimation on the first vector s of the first data set and the second vector s of the second data set to generate a corresponding first angle, 1->2,i and the second vector s 3->4,i performing vector angle estimation on the first vector s of the first data set and the second vector s of the second data set to generate a corresponding first angle, ; and identifying whether the first angle is smaller than a preset small angle threshold; if yes, setting a corresponding second check bit as success, and if not, setting the corresponding second check bit as failure; the first aortic annulus diameter d of the first data set avr,i the first aortic sinus diameter d of the first data set as,i whether the corresponding straight line segments have intersection points is identified; if yes, the corresponding third check bit is set as failed, and if not, the corresponding third check bit is set as successful; identifying the third check bit; if the third check bit is failure, setting a corresponding fourth check bit as failure; if the third check bit is success, identifying the first aortic valve annulus diameter d avr,i whether a corresponding straight line segment is in the first aortic sinus diameter d as,i identifying a left side of a corresponding straight line segment; if yes, setting a corresponding fourth check bit as success, if no, setting a corresponding fourth check bit as failure; if the obtained first, second, third and fourth test bits are all successful, setting the corresponding first analysis result as reasonable; if the obtained first, second, third or fourth test bits are failed, setting the corresponding first analysis result as unreasonable.

8. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 6, characterized in that, the performing discrete data set identification on the first data sets with the first analysis result being reasonable to obtain a plurality of first discrete data sets specifically includes: The first analysis result is reasonable for the first data set is recorded as a corresponding second data set; the first aortic valve annulus diameter d of all the second data sets avr,i constitute a corresponding first diameter set; and the number of the first aortic valve annulus diameter d in the first diameter set is counted to generate a corresponding first number N1; and the mean and standard deviation of the first diameter set are calculated to obtain a corresponding first mean μ1 and first standard deviation σ1; and a corresponding first range is constructed according to the first mean μ1 and the first standard deviation σ1; and the second data set whose first aortic valve annulus diameter d exceeds the first range is recorded as a corresponding first discrete data set; wherein, avr,i avr,i , the lower limit of the first range is and the upper limit is and α1 is a preset coefficient;​​ The diameter d of the first aortic sinus of all the second data groups as,i Form a corresponding second diameter set; and determine the diameter d of the first aortic sinus in the second diameter set. as,i The number of diameters is statistically analyzed to generate a corresponding second number N2; the mean and standard deviation of the second diameter set are calculated to obtain the corresponding second mean μ2 and second standard deviation σ2; a corresponding second amplitude limiting range is constructed based on the second mean μ2 and the second standard deviation σ2; and the diameter d of the first aortic sinus is... as,i The second data group that exceeds the second amplitude limit is denoted as the corresponding first discrete data group; wherein... , The lower limit of the second limiting range is The upper limit is α2 is a preset coefficient.

9. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 6, characterized in that, The abnormal frame image data correction processing of each first abnormal frame image specifically includes: Any first abnormal frame image in the first frame image sequence is taken as a corresponding current abnormal frame image, and the previous and next non-abnormal frame images of the current abnormal frame image in the first frame image sequence are taken as corresponding previous and next normal frame images; using the first left ventricular internal diameter d vs,i , the first aortic annulus diameter d avr,i , the first aortic sinus diameter d as,i , the first right ventricular internal diameter d vd,i , the first interventricular septal thickness d IS,i , the first left ventricular posterior wall thickness d lvpw,i , the first left atrial anteroposterior diameter d la,i , and the first ascending aorta internal diameter d aa,i , the first left ventricular internal diameter d vs,i , the first aortic annulus diameter d avr,i , the first aortic sinus diameter d as,i , the first right ventricular internal diameter d vd,i , the first interventricular septal thickness d IS,i , the first left ventricular posterior wall thickness d lvpw,i , the first left atrial anteroposterior diameter d la,i , and the first ascending aorta internal diameter d aa,i are reset by linear interpolation. The first predicted key point image coordinates of the first left ventricular key point, the second left ventricular key point, the first aortic valve ring key point, the second aortic valve ring key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricular key point, the second right ventricular key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricular posterior wall key point, the second left ventricular posterior wall key point, the first left atrial key point, the second left atrial key point, the first ascending aorta key point and the second ascending aorta key point of the previous and next normal frame images are used for linear interpolation reset of the first predicted key point image coordinates of the first left ventricular key point, the second left ventricular key point, the first aortic valve ring key point, the second aortic valve ring key point, the first aortic sinus key point, the second aortic sinus key point, the first right ventricular key point, the second right ventricular key point, the first interventricular septum key point, the second interventricular septum key point, the first left ventricular posterior wall key point, the second left ventricular posterior wall key point, the first left atrial key point, the second left atrial key point, the first ascending aorta key point and the second ascending aorta key point of the current abnormal frame image.

10. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 3, characterized in that, The key point marking and feature size labeling processing of the right ventricular internal diameter, the interventricular septal thickness, the left ventricular internal diameter and the left ventricular posterior wall thickness on the first diastolic frame image specifically includes: On the first diastolic frame image, the color of two pixel points corresponding to the two first predicted key point image coordinates of the first and second right ventricular key points is set as a preset first color; and a corresponding first line segment is obtained by connecting line segment drawing between the two pixel points corresponding to the first and second right ventricular key points based on the first color; and the first right ventricular internal diameter d vd,i is obtained as a product of the first diastolic frame image and a preset unit scale; and the display annotation information of the first line segment is set as the corresponding second right ventricular internal diameter; On the first diastolic frame image, the color of two pixel points corresponding to the two first predicted key point image coordinates of the first and second interventricular septum key points is set as a preset second color; and a corresponding second line segment is obtained by connecting line segment drawing between the two pixel points corresponding to the first and second interventricular septum key points based on the second color; and the first interventricular septum thickness d IS,i is the product of the unit scale and the first interventricular septum thickness d ; and the display label information of the second line segment is set as the corresponding second interventricular septum thickness. On the first diastolic frame image, the color of two pixel points corresponding to the two first predicted key point image coordinates of the first and second left ventricular key points is set as a third preset color; and a third line segment is drawn based on the third color between the two pixel points corresponding to the first and second left ventricular key points to obtain a corresponding third line segment; and the first left ventricular internal diameter d vs,i is obtained as a corresponding second left ventricular internal diameter; and the display label information of the third line segment is set as the corresponding second left ventricular internal diameter; On the first diastolic phase frame image, the color of two pixel points corresponding to the two first prediction key point image coordinates of the first and second left ventricular posterior wall key points is set as a fourth preset color; and a fourth line segment is drawn based on the fourth color between the two pixel points corresponding to the first and second left ventricular posterior wall key points; and the first left ventricular posterior wall thickness d lvpw,i is obtained by connecting the first diastolic phase frame image; and the display label information of the fourth line segment is set as the corresponding second left ventricular posterior wall thickness.

11. The method of claim 3, wherein, The key point marking and feature size labeling processing of the left ventricular internal diameter on the first systolic frame image specifically includes: On the first systolic frame image, the color of two pixel points corresponding to the two first predicted key point image coordinates of the first and second left ventricular key points is set as a preset fifth color; and a corresponding fifth line segment is obtained by connecting line segment drawing between the two pixel points corresponding to the first and second left ventricular key points based on the fifth color; and the first left ventricular internal diameter d vs,i is obtained by multiplying the first left ventricular internal diameter d by a preset unit scale. The display label information of the fifth line segment is set as the corresponding third left ventricular internal diameter.

12. The processing method for measuring a feature size based on a cardiac ultrasound video according to claim 3, characterized in that, The key point marking and feature size labeling processing of the left atrial anteroposterior diameter, the aortic valve ring diameter, the aortic sinus diameter and the ascending aorta internal diameter on the first ventricular systolic mid-phase frame image specifically includes: In the first ventricular systolic mid-phase frame image, the color of two pixel points corresponding to the two first prediction key point image coordinates of the first and second left atrial key points is set as a preset sixth color; and a corresponding sixth line segment is obtained by connecting the two pixel points corresponding to the first and second left atrial key points based on the sixth color; and the first left atrial anteroposterior diameter d la,i is obtained as a product of the preset unit scale and the first left atrial anteroposterior diameter; and the display label information of the sixth line segment is set as the corresponding second left atrial anteroposterior diameter; In the first ventricular systole mid-stage frame image, the color of two pixel points corresponding to the two first prediction key point image coordinates of the first and second aortic valve ring key points is set as a preset seventh color; and a corresponding seventh line segment is obtained by connecting line segment drawing between the two pixel points corresponding to the first and second aortic valve ring key points based on the seventh color; and the first aortic valve ring diameter d avr,i is obtained, and the display label information of the seventh line segment is set as the corresponding second aortic valve ring diameter; In the first ventricular systolic mid-phase frame image, the color of two pixel points corresponding to the two first prediction key point image coordinates of the first and second aortic sinus key points is set as a preset eighth color; and a corresponding eighth line segment is obtained by connecting line segment drawing between the two pixel points corresponding to the first and second aortic sinus key points based on the eighth color; and the first aortic sinus diameter d as,i is obtained, and the display label information of the eighth line segment is set as the corresponding second aortic sinus diameter; On the first ventricular systole mid-stage frame image, the color of two pixel points corresponding to the two first prediction key point image coordinates of the first and second ascending aorta key points is set as a preset ninth color; and a corresponding ninth line segment is obtained by connecting line segment drawing between the two pixel points corresponding to the first and second ascending aorta key points based on the ninth color; and the first ascending aorta inner diameter d aa,i is obtained, and the display label information of the ninth line segment is set as the corresponding second ascending aorta inner diameter.

13. An apparatus for implementing the processing method for measuring a feature size based on a cardiac ultrasound video according to any one of claims 1 to 12, characterized in that, The device includes a receiving module, an image framing module, a key point detection module, a framed image screening module, an image labeling module and an output module; The receiving module is used to receive a heart ultrasound video as a corresponding first video; The image framing module is configured to perform video frame image extraction processing on the first video to generate a corresponding first frame image sequence; the first frame image sequence comprises a plurality of first frame images P i sequenced in chronological order, each of the first frame images corresponds to a first frame timestamp T i , and a frame image index i≥1. The key point detection module is configured to perform key point detection processing on each of the first frame images P i to obtain corresponding sixteen key points and eight feature sizes; the eight feature sizes include a first left ventricular internal diameter d vs,i , a first aortic valve annulus diameter d avr,i , and a first aortic sinus diameter d as,i . The frame image screening module is configured to screen all the first left ventricular internal diameters d vs,i The diastolic and systolic frame image screening processing is performed on the first frame image sequence to obtain corresponding first diastolic frame images and first systolic frame images; and all the first aortic valve annulus diameters d avr,i and the first aortic sinus diameters d as,i The mid-systolic frame image screening processing is performed on the first frame image sequence to obtain corresponding first mid-systolic frame images; The image labeling module is configured to perform key point labeling and feature size labeling on the right ventricular internal diameter, the interventricular septal thickness, the left ventricular internal diameter, and the left ventricular posterior wall thickness on the first diastolic frame image; perform key point labeling and feature size labeling on the left ventricular internal diameter on the first systolic frame image; and perform key point labeling and feature size labeling on the left atrial anteroposterior diameter, the aortic valve annulus diameter, the aortic sinus diameter, and the ascending aorta internal diameter on the first ventricular systolic midframe image. The output module is configured to output the first diastolic frame image, the first systolic frame image, and the first ventricular systolic midframe image on which the key point labeling and the feature size labeling are completed as processing results of measuring feature sizes of the current cardiac ultrasound video.

14. An electronic device, comprising: Comprise: a memory, a processor, and a transceiver; the processor is configured to be coupled with the memory, read and execute instructions in the memory to implement the method of any one of claims 1-12; the transceiver is coupled with the processor, and the transceiver is controlled by the processor to perform message transmission and reception.

15. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, when the computer instructions are executed by a computer, the computer executes the method of any one of claims 1-12.

Images