Method and device for screening diastolic and systolic images based on cardiac ultrasound video
Through image semantic segmentation model and significance threshold processing, the end-systolic and end-systolic images in cardiac ultrasound videos are automatically screened, solving the problem of manual screening dependence in the prior art and improving the accuracy and stability of ejection fraction calculation.
Patent Information
- Application Number
- CN202210018424.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-07
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-01-07
AI Technical Summary
In existing cardiac ultrasound examination, the calculation of ejection fraction relies on manual screening of end-systolic and end-diastolic images, resulting in unstable accuracy and low accuracy, affecting the calculation results of ejection fraction.
The image semantic segmentation model was used to perform left ventricular segmentation on cardiac ultrasound video to generate an area-time curve, and peak and valley points were identified and screened through significance thresholds, and systolic and diastolic end images were automatically screened to reduce manual intervention.
It improves the accuracy and accuracy of image screening, ensures the quality stability of ejection fraction calculation, and reduces artificial errors.
Smart Images

Figure CN114419500B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for screening diastolic and systolic images based on cardiac ultrasound video. Background Art
[0002] Ejection fraction refers to the stroke volume, that is, the percentage of blood ejected from one ventricle in one heartbeat to the volume of the ventricle at the end of diastole, and is one of the important indicators for judging the health of the heart. The calculation of ejection fraction is related to the ventricular volume at the end of systole and diastole. Conventional cardiac ultrasound examination equipment outputs a set of video data, that is, cardiac ultrasound video, in real time. When the operator plays this set of cardiac ultrasound videos on the device, he often selects multiple frames of end-systolic and end-diastolic images from them based on personal experience as reference images for analyzing the ejection fraction. This mode of operation relies too much on personal manual experience and proficiency in manual operation of the equipment, and cannot effectively guarantee the stability of the screening quality of end-systolic and end-diastolic images. The screened images will be used as the basis for further calculation of the ejection fraction, and may also affect the calculation accuracy of the ejection fraction when the image screening quality is poor. Summary of the invention
[0003] The purpose of the present invention is to address the defects of the prior art and provide a method, device, electronic device and computer-readable storage medium for screening diastolic and systolic images based on cardiac ultrasound video, using an image semantic segmentation model to improve the left ventricle recognition accuracy, by identifying the significant peak points and valley points of the left ventricular area-time curve to ensure the recognition accuracy of the left ventricular end-systolic and end-diastolic extreme points, and by further identifying the effective time intervals of the significant peak points and valley points to improve the recognition accuracy of the left ventricular end-systolic and end-diastolic extreme points. Through the present invention, the manual intervention link in the traditional practice can be eliminated, the screening precision and accuracy of the end-systolic and end-diastolic images can be improved, and the stability of the image screening quality can be guaranteed.
[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present invention provides a method for screening diastolic and systolic images based on cardiac ultrasound video, the method comprising:
[0005] Obtain cardiac ultrasound video;
[0006] Performing video image frame processing on the cardiac ultrasound video to generate a plurality of frame images;
[0007] Performing left ventricle semantic segmentation processing on each of the frame images based on an image semantic segmentation model, thereby obtaining a corresponding left ventricle segmentation image in each of the frame images;
[0008] Performing image area statistics on each of the left ventricle segmentation images to generate corresponding left ventricle area parameters;
[0009] Performing area-time curve conversion processing according to all the left ventricular area parameters to generate a corresponding first area-time curve;
[0010] Performing significance threshold confirmation processing according to all the left ventricular area parameters to generate a corresponding first significance threshold;
[0011] Performing significant peak point and significant valley point identification processing on existing peak points and valley points of the first area-time curve according to the first significant threshold;
[0012] Screening the significant peak points and significant valley points on the first area-time curve for effective peak points and effective valley points;
[0013] The framed images corresponding to the effective valley points on the first area-time curve are used as the end-systolic screening images, and the framed images corresponding to the effective peak points are used as the end-diastolic screening images, and all the screening images are sorted according to the preset cardiac cycle sorting rules to generate a corresponding screening image sequence.
[0014] Preferably, performing image area statistics on each of the left ventricle segmented images to generate corresponding left ventricle area parameters specifically includes:
[0015] The total number of pixels of each of the left ventricle segmentation images is counted as the corresponding left ventricle area parameter.
[0016] Preferably, performing area-time curve conversion processing according to all the left ventricular area parameters to generate a corresponding first area-time curve specifically includes:
[0017] All the left ventricular area parameters are sorted according to the time sequence of the corresponding frame images to obtain a first area parameter sequence; and the first area parameter sequence is converted into an area-time curve with the area parameter as the vertical axis and the time as the horizontal axis to generate the first area-time curve.
[0018] Preferably, performing significance threshold confirmation processing according to all the left ventricular area parameters to generate a corresponding first significance threshold specifically includes:
[0019] All the left ventricular area parameters are sorted in order from small to large to obtain a second area parameter sequence; the left ventricular area parameter at the first preset sampling position with a forward sort in the second area parameter sequence is used as a smaller boundary value, and the left ventricular area parameter at the second preset sampling position with a backward sort is used as a larger boundary value; and half of the difference between the larger boundary value and the smaller boundary value is used as the corresponding first significance threshold.
[0020] Preferably, the step of performing significant peak point and significant valley point identification processing on existing peak points and valley points of the first area-time curve according to the first significant threshold specifically includes:
[0021] Taking each existing peak point on the first area-time curve as a peak point to be evaluated; performing a significance feature evaluation process on each peak point to be evaluated to generate a corresponding significance evaluation parameter; and taking the peak point to be evaluated whose significance evaluation parameter exceeds the first significance threshold as a first peak point;
[0022] Performing curve inversion processing on the first area-time curve to generate a corresponding second area-time curve; using each existing peak point on the second area-time curve as a peak point to be evaluated; performing significance feature evaluation processing on each of the peak points to be evaluated to generate a corresponding significance evaluation parameter; using the peak point to be evaluated whose significance evaluation parameter exceeds the first significance threshold as a second peak point; and according to the curve inversion correspondence relationship between the second area-time curve and the first area-time curve, using the existing valley points in the first area-time curve corresponding to the inversion of each of the second peak points as first valley points;
[0023] Each of the first peak points on the first area-time curve is used as the significant peak point, and each of the first valley points is used as the significant valley point.
[0024] Furthermore, the performing of a significance feature evaluation process on each of the peak points to be evaluated to generate corresponding significance evaluation parameters specifically includes:
[0025] The area-time curve where the peak point to be evaluated is located is recorded as the current area-time curve;
[0026] On the current area-time curve, a line passing through the peak point to be evaluated and parallel to the horizontal axis of time is recorded as a first parallel line;
[0027] The first intersection point of the first parallel line and the falling edge of the left side curve of the peak point to be evaluated is recorded as the first left side intersection point, and the curve segment from the first left side intersection point to the peak point to be evaluated is recorded as the first left side curve segment; and the first intersection point of the first parallel line and the rising edge of the right side curve of the peak point to be evaluated is recorded as the first right side intersection point, and the curve segment from the peak point to be evaluated to the first right side intersection point is recorded as the first right side curve segment;
[0028] The valley point with the lowest curve amplitude in the first left curve segment is used as the first reference valley point; the valley point with the lowest curve amplitude in the first right curve segment is used as the second reference valley point; and the valley point with a relatively high curve amplitude is selected from the first and second reference valley points as the third reference valley point;
[0029] The curve amplitude difference between the peak point to be evaluated and the third reference valley point is used as the significance evaluation parameter corresponding to the peak point to be evaluated.
[0030] Preferably, the screening of significant peak points and significant valley points on the first area-time curve for effective peak points and effective valley points specifically includes:
[0031] The significant peak point on the first area-time curve is recorded as a peak point to be screened; the peak point filtering process is performed on the peak point to be screened on the first area-time curve, specifically: when the time interval between two adjacent peak points to be screened is lower than a preset effective extreme point interval threshold, the peak point to be screened with a lower curve amplitude is filtered out from all current peak points to be screened; the peak point filtering process is continuously performed on the remaining peak points to be screened on the first area-time curve until the time interval between any pair of adjacent peak points to be screened on the first area-time curve is not lower than the effective extreme point interval threshold;
[0032] The significant valley points on the first area-time curve are recorded as valley points to be screened; the valley points to be screened on the first area-time curve are subjected to valley point filtering processing, specifically: when the time interval between two adjacent valley points to be screened is lower than the effective extreme point interval threshold, the valley points to be screened with a higher curve amplitude are filtered out from all current valley points to be screened; the valley point filtering processing is continuously performed on the remaining valley points to be screened on the first area-time curve until the time interval between any pair of adjacent valley points to be screened on the first area-time curve is not lower than the effective extreme point interval threshold;
[0033] The last remaining peak point to be screened on the first area-time curve is used as the effective peak point, and the last remaining valley point to be screened is used as the effective valley point.
[0034] A second aspect of the embodiments of the present invention provides a device for implementing the method described in the first aspect, including: an acquisition module, an image preprocessing module, a saliency processing module, and an image screening module;
[0035] The acquisition module is used to acquire cardiac ultrasound video;
[0036] The image preprocessing module is used to perform video image framing processing on the cardiac ultrasound video to generate a plurality of framed images; and perform left ventricle semantic segmentation processing on each of the framed images based on an image semantic segmentation model, so as to obtain a corresponding left ventricle segmented image in each of the framed images;
[0037] The significance processing module is used to perform image area statistics on each of the left ventricle segmentation images to generate corresponding left ventricle area parameters; and perform area-time curve conversion processing according to all the left ventricle area parameters to generate a corresponding first area-time curve; and perform significance threshold confirmation processing according to all the left ventricle area parameters to generate a corresponding first significance threshold;
[0038] The image screening module is used to identify the significant peak points and significant valley points of the existing peak points and valley points of the first area-time curve according to the first significance threshold; and to screen the significant peak points and significant valley points on the first area-time curve for effective peak points and effective valley points; and to use the framed images corresponding to the effective valley points on the first area-time curve as the end-systolic screening images, and the framed images corresponding to the effective peak points as the end-diastolic screening images, and to sort all the screened images according to the preset cardiac cycle sorting rules to generate a corresponding screening image sequence.
[0039] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;
[0040] The processor is used to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;
[0041] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.
[0042] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.
[0043] The embodiment of the present invention provides a method, device, electronic device and computer-readable storage medium for screening diastolic and systolic images based on cardiac ultrasound video. First, the image semantic segmentation model is used to perform left ventricle segmentation on the framed images of cardiac ultrasound video to obtain left ventricle segmented images at different time points; then the corresponding area-time curve is constructed based on the correspondence between the area of the left ventricle image and time, and the mean of the size boundary values is taken from the area of the left ventricle image as the significance threshold; then the significant peak points and valley points of the area-time curve are identified based on the significance threshold; then the significant peak points and valley points are screened for effective points based on the effective extreme point interval threshold; finally, the framed images corresponding to the effective valley points and peak points are used as the screening results of the end-systolic and end-diastolic images. Through the present invention, the manual intervention link in the traditional practice is eliminated, the screening precision and accuracy of the end-systolic and end-diastolic images are improved, and the stability of the image screening quality is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 A schematic diagram of a method for screening diastolic and systolic images based on cardiac ultrasound video provided in Embodiment 1 of the present invention;
[0045] Figure 2 A schematic diagram of an area-time curve provided in Example 1 of the present invention;
[0046] Figure 3 A module structure diagram of a device for screening diastolic and systolic images based on cardiac ultrasound video provided in Embodiment 2 of the present invention;
[0047] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] Embodiment 1 of the present invention provides a method for screening diastolic and systolic images based on cardiac ultrasound video, such as Figure 1 As shown in the schematic diagram of a method for screening diastolic and systolic images based on cardiac ultrasound video provided in the first embodiment of the present invention, the method mainly comprises the following steps:
[0050] Step 1: Obtain cardiac ultrasound video.
[0051] Here, the cardiac ultrasound video refers to ultrasound video data output by a cardiac ultrasound examination device.
[0052] Step 2: Perform video image frame processing on the cardiac ultrasound video to generate multiple frame images.
[0053] Here, an image frame extraction operation is performed on the cardiac ultrasound video according to a set video frame sampling rate. The framed image is the extracted single-frame image, and each framed image corresponds to a time point parameter.
[0054] Step 3, performing left ventricle semantic segmentation processing on each frame image based on the image semantic segmentation model, so as to obtain a corresponding left ventricle segmentation image in each frame image;
[0055] Among them, the network structure of the image semantic segmentation model can adopt a variety of neural network structures for realizing image semantic segmentation, including the DeepLabV3 network structure; the DeepLabV3 network structure can refer to the article "Rethinking Atrous Convolution for Semantic Image Segmentation" published by authors Liang-ChiehChen, George Papandreou, Florian Schroff and Hartwig Adam, which will not be further described here; the pixel values of all pixels of the left ventricle segmentation image are unified first pixel values.
[0056] Here, the first pixel value is conventionally selected from other color pixel values that have an obvious color contrast effect with the frame image.
[0057] Here, based on the image semantic segmentation model, left ventricle semantic segmentation is performed on each frame image to output an output image with a size consistent with the size of the frame image, and the pixel values of the pixel points of the output image only include two values: a preset foreground pixel value and a background pixel value; in the output image, the foreground image composed of all pixel points whose pixel values are foreground pixel values is the model left ventricle segmentation image; in the frame image, the pixel value of each pixel point corresponding to the model left ventricle segmentation image is set to the first pixel value, so that a corresponding left ventricle segmentation image can be obtained in each frame image.
[0058] Before performing left ventricular semantic segmentation processing on each frame image based on the image semantic segmentation model, the image semantic segmentation model needs to be trained. When training the image semantic segmentation model, the original cardiac ultrasound images are extracted from the training data set storing a large number of cardiac ultrasound images as training images, and then the segmented images corresponding to each original cardiac ultrasound image that have been manually or machine-segmented with the left ventricle shape are extracted as comparison images; and the image semantic segmentation model is trained with the training data pairs consisting of multiple sets of training images and comparison images until the model error converges.
[0059] Step 4, performing image area statistics on each left ventricle segmentation image to generate corresponding left ventricle area parameters;
[0060] Specifically, the method includes: counting the total number of pixels of each left ventricle segmentation image as the corresponding left ventricle area parameter.
[0061] Here, the number of pixels in the left ventricle segmentation image is used as a parameter representing the area size, that is, the left ventricle area parameter.
[0062] Step 5, performing area-time curve conversion processing according to all left ventricular area parameters to generate a corresponding first area-time curve;
[0063] Specifically, the method includes: sorting all left ventricular area parameters according to the time sequence of the corresponding frame images to obtain a first area parameter sequence; and performing area-time curve conversion processing on the first area parameter sequence to generate a first area-time curve with the area parameter as the vertical axis and the time as the horizontal axis.
[0064] Here, a two-dimensional area-time coordinate space is constructed with the area parameter as the vertical axis and the time as the horizontal axis, and then the corresponding coordinate points of the left ventricular area parameter of the first area parameter sequence are recorded in the area-time coordinate space, and then all the recorded coordinate points are connected in sequence to obtain the first area-time curve.
[0065] Step 6, performing significance threshold confirmation processing according to all left ventricular area parameters to generate a corresponding first significance threshold;
[0066] Specifically, the method includes: sorting all left ventricular area parameters in ascending order to obtain a second area parameter sequence; using the left ventricular area parameter at the first preset sampling position with a higher order in the second area parameter sequence as the smaller boundary value, and using the left ventricular area parameter at the second preset sampling position with a lower order as the larger boundary value; and using half of the difference between the larger boundary value and the smaller boundary value as the corresponding first significance threshold.
[0067] Here, the ratio of the first preset sampling position to the total number of left ventricular area parameters of the second area parameter sequence is a set ratio, that is, a first ratio, and the ratio of the second preset sampling position to the total number of left ventricular area parameters of the second area parameter sequence is also a set ratio, that is, a second ratio. The current step does not directly select the minimum and maximum values as boundary values in order to avoid errors caused by sudden extreme value points. The first significance threshold will be used as a significance feature for determining peak points in subsequent steps.
[0068] For example, the second area parameter sequence has 100 left ventricular area parameters, the first ratio is a%, and the second ratio is b%; then, the first preset sampling position is the position of the ath left ventricular area parameter in the second area parameter sequence, and the second preset sampling position is the position of the bth left ventricular area parameter; the first significance threshold = (larger boundary value - smaller boundary value) / 2 = (bth left ventricular area parameter - ath left ventricular area parameter) / 2.
[0069] Step 7, performing significant peak point and significant valley point identification processing on the existing peak points and valley points of the first area-time curve according to the first significant threshold;
[0070] Here, the first area-time curve is not a smooth curve. There are many local maximum and minimum values on the curve. Most of these local maximum and minimum values are noise points, not the actual extreme points at the end of systole and the end of diastole. In order to improve the recognition accuracy of the extreme points at the end of systole and the end of diastole, the current step performs extreme point significance analysis on the first area-time curve, and uses the first significance threshold as the basis for peak point significance judgment to identify each significant peak point and significant valley point;
[0071] Specifically, the method comprises: step 71, taking each existing peak point on the first area-time curve as a peak point to be evaluated; performing a significance feature evaluation process on each peak point to be evaluated to generate a corresponding significance evaluation parameter; and taking the peak point to be evaluated whose significance evaluation parameter exceeds a first significance threshold as a first peak point;
[0072] Here, the first significance threshold is used to judge whether the peak point to be evaluated is a significant peak point; before using the first significance threshold to judge the significant peak point, it is necessary to evaluate the significance evaluation parameters of each peak point to be evaluated;
[0073] Furthermore, a significant feature evaluation process is performed on each peak point to be evaluated to generate corresponding significant evaluation parameters, specifically including:
[0074] Step A1, recording the area-time curve where the peak point to be evaluated is located as the current area-time curve;
[0075] Here, the area-time curve where the peak point to be evaluated is located, that is, the current area-time curve, is actually the first area-time curve;
[0076] Step A2: on the current area-time curve, a line parallel to the horizontal axis of time passing through the peak point to be evaluated is recorded as the first parallel line;
[0077] For example, the current area-time curve is Figure 2 As shown in the schematic diagram of the area-time curve provided in the first embodiment of the present invention, taking the peak point P to be evaluated as an example, the first parallel line obtained by drawing a parallel line on the time axis through point P is as follows: Figure 2 As shown;
[0078] Step A3, record the first intersection point of the first parallel line with the falling edge of the left side curve of the peak point to be evaluated as the first left side intersection point, and record the curve segment from the first left side intersection point to the peak point to be evaluated as the first left side curve segment; and record the first intersection point of the first parallel line with the rising edge of the right side curve of the peak point to be evaluated as the first right side intersection point, and record the curve segment from the peak point to be evaluated to the first right side intersection point as the first right side curve segment;
[0079] For example Figure 2 As shown, taking the peak point P to be evaluated as an example, the first left intersection point PL is the intersection point of the first parallel line and the falling edge of the first curve on the left side of the peak point to be evaluated, and the first left curve segment is the curve segment of PL-P; the first right intersection point PR is the first intersection point of the first parallel line and the rising edge of the curve on the right side of the peak point to be evaluated, and the first left curve segment is the curve segment of P-PR;
[0080] It should be noted that if the first parallel line has no intersection with the falling edge of the left side curve of the peak point to be evaluated, that is, when the first left intersection is empty, the first left curve segment is the curve segment from the starting position of the curve to the peak point to be evaluated; if the first parallel line has no intersection with the rising edge of the right side curve of the peak point to be evaluated, that is, when the first right intersection is empty, then the first right curve segment is the curve segment from the peak point to be evaluated to the end position of the curve;
[0081] Step A4, taking the valley point with the lowest curve amplitude in the first left curve segment as the first reference valley point; and taking the valley point with the lowest curve amplitude in the first right curve segment as the second reference valley point; and selecting the curve point with a relatively high amplitude from the first and second reference valley points as the third reference valley point;
[0082] For example Figure 2As shown, taking the peak point P to be evaluated as an example, the first left curve segment, that is, the PL-P curve segment, includes two valley points V1 and V2, among which the valley point with the lowest curve amplitude is V1, so the first reference valley point is V1; the first right curve segment, that is, the P-PR curve segment, includes two valley points V3 and V4, among which the valley point with a relatively high curve amplitude is V4, so the second reference valley point is V4; of the first reference valley point V1 and the second reference valley point V4, the first reference valley point V1 has a relatively high curve amplitude, so the third reference valley point should be V1;
[0083] Step A5, taking the curve amplitude difference between the peak point to be evaluated and the third reference valley point as a significance evaluation parameter corresponding to the peak point to be evaluated;
[0084] For example Figure 2 As shown, taking the peak point P to be evaluated as an example, the significance evaluation parameter corresponding to the peak point P to be evaluated should be the amplitude of point P minus the amplitude of point V1;
[0085] Here, conventionally, when evaluating the significance of a curve or waveform, the amplitude difference from the peak point to the baseline is used as a significance evaluation parameter. However, in actual situations, the area-time curve is an irregular curve, and there is a high possibility of baseline drift, and the baseline drift is also difficult to be an ideal linear drift structure. In this case, the error of the peak point significance evaluation parameter obtained will be very large. To solve this problem, the embodiment of the present invention selects a relative baseline point, that is, a third reference valley point, for each peak point to be evaluated through the above steps A1-A4. By selecting the relative baseline point, the error effect of the baseline drift on the significance feature evaluation can be reduced, and the accuracy of the significance feature evaluation can be improved. Finally, the amplitude difference between the peak point to be evaluated and the third reference valley point is calculated through the above step A5 to obtain a more accurate significance evaluation parameter.
[0086] Step 72, performing curve inversion processing on the first area-time curve to generate a corresponding second area-time curve; using each existing peak point on the second area-time curve as a peak point to be evaluated; performing significance feature evaluation processing on each peak point to be evaluated to generate a corresponding significance evaluation parameter; using the peak point to be evaluated whose significance evaluation parameter exceeds the first significance threshold as the second peak point; and according to the curve inversion correspondence relationship between the second area-time curve and the first area-time curve, using the existing valley points in the first area-time curve corresponding to the inversion of each second peak point as the first valley point;
[0087] Here, the current step is actually to use the first significance threshold to judge the significance valley points of each existing valley point of the first area-time curve. Before judging the significance valley point, it is necessary to evaluate the significance evaluation parameters of each valley point to be evaluated. When evaluating the significance evaluation parameters of each valley point to be evaluated, the embodiment of the present invention first inverts the first area-time curve to obtain a second area-time curve, and the peak point in the second area-time curve is also the valley point in the first area-time curve. Then, the above steps A1-A5 are used to perform significance feature evaluation processing on each peak point of the second area-time curve, and the obtained significance evaluation parameters are actually the significance evaluation parameters of each valley point in the first area-time curve.
[0088] Step 73: taking each first peak point on the first area-time curve as a significant peak point, and taking each first valley point as a significant valley point.
[0089] Step 8, screening the significant peak points and significant valley points on the first area-time curve for effective peak points and effective valley points;
[0090] Here, after filtering out most of the noise points with weak significance through the aforementioned steps, there may still be some local extreme points with strong significance caused by environmental interference on the first area-time curve. The current step further filters these interference extreme points by time interval;
[0091] Specifically, it includes: step 81, recording the significant peak point on the first area-time curve as the peak point to be screened; performing peak point filtering processing on the peak point to be screened on the first area-time curve;
[0092] Further, peak point filtering processing is performed on the peak points to be screened on the first area-time curve, specifically including: when the time interval between two adjacent peak points to be screened is lower than a preset effective extreme point interval threshold, the peak point to be screened with a lower curve amplitude is filtered out from all current peak points to be screened; the peak point filtering processing is continuously performed on the remaining peak points to be screened on the first area-time curve until the time interval between any pair of adjacent peak points to be screened on the first area-time curve is not lower than the effective extreme point interval threshold;
[0093] Here, on the first area-time curve after the peak point filtering process is completed, the peak-to-peak distance between adjacent valid peak points is greater than the valid extreme value point interval threshold, and the valid extreme value point interval threshold is a preset interval threshold;
[0094] It should be noted that the embodiment of the present invention also supports another implementation method when performing peak point filtering processing on the peak points to be screened on the first area-time curve, and the steps include:
[0095] Step B1, on the first area-time curve, taking the first peak point to be screened as the first reference point;
[0096] Step B2, taking the next peak point to be screened after the first reference point as the second reference point;
[0097] Step B3, calculating the time interval between the first and second reference points to generate a first time interval;
[0098] Step B4, determining whether the first time interval is greater than the effective extreme point interval threshold; if the first time interval is greater than the effective extreme point interval threshold, taking the second reference point as the new first reference point; if the first time interval is less than or equal to the effective extreme point interval threshold, filtering out the peak point to be screened with a smaller amplitude from all current peak points to be screened in the first and second reference points, and taking the peak point to be screened with a larger amplitude as the new first reference point;
[0099] Step B5, determining whether the new first reference point is the last peak point to be screened on the first area-time curve, if yes, stopping the current peak point filtering process, if no, going to step B2;
[0100] Based on the above implementation, the last peak point to be screened on the first area-time curve can also be used as the first reference point, the previous peak point to be screened can be used as the second reference point, and the traversal can be performed from the first reference point forward; Based on the above implementation, the peak point to be screened with the maximum amplitude on the first area-time curve can also be used as the first reference point, and the traversal can be performed both forward and backward from the first reference point;
[0101] Step 82, recording the significant valley points on the first area-time curve as valley points to be screened; performing valley point filtering processing on the valley points to be screened on the first area-time curve;
[0102] Further, the valley value points to be screened on the first area-time curve are subjected to valley value point filtering processing, specifically including: when the time interval between two adjacent valley value points to be screened is lower than the effective extreme value point interval threshold, the valley value point to be screened with a higher curve amplitude is filtered out from all current valley value points to be screened; the remaining valley value points to be screened on the first area-time curve are continuously subjected to valley value point filtering processing until the time interval between any pair of adjacent valley value points to be screened on the first area-time curve is not lower than the effective extreme value point interval threshold;
[0103] Here, on the first area-time curve after the valley point filtering process is completed, the valley spacing between adjacent valid valley points is greater than the valid extreme point spacing threshold;
[0104] It should be noted that the embodiment of the present invention also supports another implementation method when performing valley point filtering processing on the valley points to be screened on the first area-time curve, and the steps include:
[0105] Step C1, on the first area-time curve, taking the first valley point to be screened as the third reference point;
[0106] Step C2, taking the next peak point to be screened after the third reference point as the fourth reference point;
[0107] Step C3, calculating the time interval between the third and fourth reference points to generate a second time interval;
[0108] Step C4, determining whether the second time interval is greater than the effective extreme point interval threshold; if the second time interval is greater than the effective extreme point interval threshold, taking the fourth reference point as the new third reference point; if the second time interval is less than or equal to the effective extreme point interval threshold, filtering out the valley value points to be screened with larger amplitudes from all the current valley value points to be screened among the third and fourth reference points, and taking the valley value points to be screened with smaller amplitudes as the new third reference point;
[0109] Step C5, determining whether the new third reference point is the last valley point to be screened on the first area-time curve, if yes, stopping the current valley point filtering process, if no, going to step C2;
[0110] Based on the above implementation, the last valley value point to be screened on the first area-time curve can also be used as the third reference point, the valley value point to be screened before the third reference point can be used as the fourth reference point, and traversal can be performed forward from the third reference point; Based on the above implementation, the valley value point to be screened with the minimum amplitude on the first area-time curve can also be used as the third reference point, and traversal can be performed forward and backward from the third reference point at the same time;
[0111] Step 83, taking the last remaining peak point to be screened on the first area-time curve as the effective peak point, and taking the last remaining valley point to be screened as the effective valley point.
[0112] Here, the effective peak point obtained on the first area-time curve should correspond to the end time point of the diastole of each heartbeat, and the effective valley point should correspond to the end time point of the systole of each heartbeat.
[0113] Step 9, using the framed images corresponding to the effective valley points on the first area-time curve as the end-systolic screening images, and the framed images corresponding to the effective peak points as the end-diastolic screening images, and sorting all the screening images according to the preset cardiac cycle sorting rules to generate the corresponding screening image sequence.
[0114] Here, the frame image corresponding to the effective valley point, that is, the end-systolic screening image, is the frame image with the most obvious systolic characteristics of each heartbeat, and the frame image corresponding to the effective peak point, that is, the end-diastolic screening image, is the frame image with the most obvious diastolic characteristics of each heartbeat; the cardiac cycle sorting rules can be defined according to actual needs, including at least the following rules: sorting all end-systolic images in chronological order, or sorting all end-diastolic images in chronological order, or sorting each group of end-systolic images + end-diastolic images in chronological order with each systolic period first and diastolic period last.
[0115] After obtaining the screening image sequence, the real-time ejection fraction of the current heartbeat can be calculated based on each group of end-systolic screening images + end-diastolic screening images; the mean end-systolic ventricular volume can be calculated based on the end-systolic screening image sequence, the mean end-diastolic ventricular volume can be calculated based on the end-diastolic screening image sequence, and then the statistical ejection fraction can be calculated based on the mean end-systolic ventricular volume and the mean end-diastolic ventricular volume.
[0116] Figure 3 The module structure diagram of a device for screening diastolic and systolic images based on cardiac ultrasound video provided in the second embodiment of the present invention, the device can be a terminal device or server for implementing the method of the embodiment of the present invention, or a device for implementing the method of the embodiment of the present invention connected to the above terminal device or server, for example, the device can be a device or chip system of the above terminal device or server. Figure 3 As shown, the device includes: an acquisition module 201, an image preprocessing module 202, a saliency processing module 203 and an image screening module 204.
[0117] The acquisition module 201 is used to acquire cardiac ultrasound video.
[0118] The image preprocessing module 202 is used to perform video image framing processing on the cardiac ultrasound video to generate multiple frame images; and perform left ventricle semantic segmentation processing on each frame image based on the image semantic segmentation model, so as to obtain a corresponding left ventricle segmentation image in each frame image.
[0119] The significance processing module 203 is used to perform image area statistics on each left ventricular segmentation image to generate the corresponding left ventricular area parameters; and perform area-time curve conversion processing based on all left ventricular area parameters to generate the corresponding first area-time curve; and perform significance threshold confirmation processing based on all left ventricular area parameters to generate the corresponding first significance threshold.
[0120] The image screening module 204 is used to identify the significant peak points and significant valley points of the existing peak points and valley points of the first area-time curve according to the first significance threshold; and to screen the significant peak points and significant valley points on the first area-time curve for effective peak points and effective valley points; and to use the framed images corresponding to the effective valley points on the first area-time curve as the end-systolic screening images, and the framed images corresponding to the effective peak points as the end-diastolic screening images, and to sort all the screened images according to the preset cardiac cycle sorting rules to generate a corresponding screening image sequence.
[0121] An embodiment of the present invention provides a device for screening diastolic and systolic images based on cardiac ultrasound video, which can execute the method steps in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.
[0122] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the acquisition module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above-mentioned module is determined. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.
[0123] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0124] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in accordance with the embodiment of the present invention is generated in whole or in part. The above computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the above computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) mode to another website site, computer, server or data center. The above computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).
[0125] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device may be the aforementioned terminal device or server, or may be a terminal device or server connected to the aforementioned terminal device or server to implement the method of the embodiment of the present invention. Figure 4As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303. Various instructions may be stored in the memory 302 to complete various processing functions and implement the methods and processing procedures provided in the above embodiments of the present invention. Preferably, the electronic device involved in the embodiment of the present invention also includes: a power supply 304, a system bus 305 and a communication port 306. The system bus 305 is used to realize communication connection between components. The above-mentioned communication port 306 is used for connection and communication between the electronic device and other peripherals.
[0126] exist Figure 4 The system bus mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.
[0127] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0128] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, in which instructions are stored. When the storage medium is run on a computer, the computer executes the method and processing process provided in the above embodiments.
[0129] An embodiment of the present invention further provides a chip for executing instructions, and the chip is used to execute the method and processing process provided in the above embodiment.
[0130] The embodiment of the present invention provides a method, device, electronic device and computer-readable storage medium for screening diastolic and systolic images based on cardiac ultrasound video, uses an image semantic segmentation model to improve the left ventricle recognition accuracy, ensures the recognition accuracy of the left ventricular end-systolic and end-diastolic extreme points by identifying the significant peak points and valley points of the left ventricular area-time curve, and improves the recognition accuracy of the left ventricular end-systolic and end-diastolic extreme points by further identifying the effective time intervals of the significant peak points and valley points. Through the present invention, the manual intervention link in the traditional practice is eliminated, the screening precision and accuracy of the end-systolic and end-diastolic images are improved, and the stability of the image screening quality is guaranteed.
[0131] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0132] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0133] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for screening diastolic and systolic images based on cardiac ultrasound video, characterized in that: The method comprises: Obtain cardiac ultrasound video; Performing video image frame processing on the cardiac ultrasound video to generate a plurality of frame images; Performing left ventricle semantic segmentation processing on each of the frame images based on an image semantic segmentation model, thereby obtaining a corresponding left ventricle segmentation image in each of the frame images; Performing image area statistics on each of the left ventricle segmentation images to generate corresponding left ventricle area parameters; Performing area-time curve conversion processing according to all the left ventricular area parameters to generate a corresponding first area-time curve; Performing significance threshold confirmation processing according to all the left ventricular area parameters to generate a corresponding first significance threshold; Performing significant peak point and significant valley point identification processing on existing peak points and valley points of the first area-time curve according to the first significant threshold; Screening the significant peak points and significant valley points on the first area-time curve for effective peak points and effective valley points; The framed images corresponding to the effective valley points on the first area-time curve are used as the end-systolic screening images, and the framed images corresponding to the effective peak points are used as the end-diastolic screening images, and all the screening images are sorted according to a preset cardiac cycle sorting rule to generate a corresponding screening image sequence; The step of performing significant peak point and significant valley point identification processing on the existing peak points and valley points of the first area-time curve according to the first significant threshold specifically includes: Taking each existing peak point on the first area-time curve as a peak point to be evaluated; performing a significance feature evaluation process on each peak point to be evaluated to generate a corresponding significance evaluation parameter; and taking the peak point to be evaluated whose significance evaluation parameter exceeds the first significance threshold as a first peak point; Performing curve inversion processing on the first area-time curve to generate a corresponding second area-time curve; using each existing peak point on the second area-time curve as a peak point to be evaluated; performing significance feature evaluation processing on each of the peak points to be evaluated to generate a corresponding significance evaluation parameter; using the peak point to be evaluated whose significance evaluation parameter exceeds the first significance threshold as a second peak point; and according to the curve inversion correspondence relationship between the second area-time curve and the first area-time curve, using the existing valley points in the first area-time curve corresponding to the inversion of each of the second peak points as first valley points; Taking each of the first peak points on the first area-time curve as the significant peak point, and each of the first valley points as the significant valley point; The performing a significant feature evaluation process on each of the peak points to be evaluated to generate a corresponding significant evaluation parameter specifically includes: The area-time curve where the peak point to be evaluated is located is recorded as the current area-time curve; On the current area-time curve, a line parallel to the horizontal axis of time passing through the peak point to be evaluated is recorded as a first parallel line; The first intersection point of the first parallel line and the falling edge of the left side curve of the peak point to be evaluated is recorded as the first left side intersection point, and the curve segment from the first left side intersection point to the peak point to be evaluated is recorded as the first left side curve segment; and the first intersection point of the first parallel line and the rising edge of the right side curve of the peak point to be evaluated is recorded as the first right side intersection point, and the curve segment from the peak point to be evaluated to the first right side intersection point is recorded as the first right side curve segment; The valley point with the lowest curve amplitude in the first left curve segment is used as the first reference valley point; the valley point with the lowest curve amplitude in the first right curve segment is used as the second reference valley point; and the valley point with a relatively high curve amplitude is selected from the first and second reference valley points as the third reference valley point; The curve amplitude difference between the peak point to be evaluated and the third reference valley point is used as the significance evaluation parameter corresponding to the peak point to be evaluated.
2. The method for screening diastolic and systolic images based on cardiac ultrasound video according to claim 1, characterized in that: The step of performing image area statistics on each of the left ventricle segmentation images to generate corresponding left ventricle area parameters specifically includes: The total number of pixels of each of the left ventricle segmentation images is counted as the corresponding left ventricle area parameter.
3. The method for screening diastolic and systolic images based on cardiac ultrasound video according to claim 1, characterized in that: The step of performing area-time curve conversion processing according to all the left ventricular area parameters to generate a corresponding first area-time curve specifically includes: All the left ventricular area parameters are sorted according to the time sequence of the corresponding frame images to obtain a first area parameter sequence; and the first area parameter sequence is converted into an area-time curve with the area parameter as the vertical axis and the time as the horizontal axis to generate the first area-time curve.
4. The method for screening diastolic and systolic images based on cardiac ultrasound video according to claim 1, characterized in that: The step of performing significance threshold confirmation processing according to all the left ventricular area parameters to generate a corresponding first significance threshold specifically includes: All the left ventricular area parameters are sorted in order from small to large to obtain a second area parameter sequence; the left ventricular area parameter at the first preset sampling position with a forward sort in the second area parameter sequence is used as a smaller boundary value, and the left ventricular area parameter at the second preset sampling position with a backward sort is used as a larger boundary value; and half of the difference between the larger boundary value and the smaller boundary value is used as the corresponding first significance threshold.
5. The method for screening diastolic and systolic images based on cardiac ultrasound video according to claim 1, characterized in that: The screening of the significant peak points and the significant valley points on the first area-time curve for effective peak points and effective valley points specifically includes: The significant peak point on the first area-time curve is recorded as a peak point to be screened; the peak point filtering process is performed on the peak point to be screened on the first area-time curve, specifically: when the time interval between two adjacent peak points to be screened is lower than a preset effective extreme point interval threshold, the peak point to be screened with a lower curve amplitude is filtered out from all current peak points to be screened; the peak point filtering process is continuously performed on the remaining peak points to be screened on the first area-time curve until the time interval between any pair of adjacent peak points to be screened on the first area-time curve is not lower than the effective extreme point interval threshold; The significant valley points on the first area-time curve are recorded as valley points to be screened; the valley points to be screened on the first area-time curve are subjected to valley point filtering processing, specifically: when the time interval between two adjacent valley points to be screened is lower than the effective extreme point interval threshold, the valley points to be screened with a higher curve amplitude are filtered out from all current valley points to be screened; the valley point filtering processing is continuously performed on the remaining valley points to be screened on the first area-time curve until the time interval between any pair of adjacent valley points to be screened on the first area-time curve is not lower than the effective extreme point interval threshold; The last remaining peak point to be screened on the first area-time curve is used as the effective peak point, and the last remaining valley point to be screened is used as the effective valley point.
6. A device for implementing the method steps of screening diastolic and systolic images based on cardiac ultrasound video according to any one of claims 1 to 5, characterized in that: The device comprises: an acquisition module, an image preprocessing module, a saliency processing module and an image screening module; The acquisition module is used to acquire cardiac ultrasound video; The image preprocessing module is used to perform video image framing processing on the cardiac ultrasound video to generate a plurality of framed images; and perform left ventricle semantic segmentation processing on each of the framed images based on an image semantic segmentation model, so as to obtain a corresponding left ventricle segmented image in each of the framed images; The significance processing module is used to perform image area statistics on each of the left ventricle segmentation images to generate corresponding left ventricle area parameters; and perform area-time curve conversion processing according to all the left ventricle area parameters to generate a corresponding first area-time curve; and perform significance threshold confirmation processing according to all the left ventricle area parameters to generate a corresponding first significance threshold; The image screening module is used to identify the significant peak points and significant valley points of the existing peak points and valley points of the first area-time curve according to the first significance threshold; and to screen the significant peak points and significant valley points on the first area-time curve for effective peak points and effective valley points; and to use the framed images corresponding to the effective valley points on the first area-time curve as the end-systolic screening images, and the framed images corresponding to the effective peak points as the end-diastolic screening images, and to sort all the screened images according to the preset cardiac cycle sorting rules to generate a corresponding screening image sequence.
7. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method steps described in any one of claims 1 to 5; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when executed by a computer, enable the computer to execute the method according to any one of claims 1 to 5.
