Method, device and equipment for determining cardiac cycle and readable storage medium
By applying a three-dimensional convolutional neural network model in fetal heart video for frame prediction and combining correction of cardiac change characteristics, the problem of low accuracy in cardiac cycle determination is solved, and higher prediction accuracy is achieved.
Patent Information
- Application Number
- CN202311751612.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-19
- Publication Date
- 2025-06-20
AI Technical Summary
In the prior art, there is a problem of low accuracy in determining the dynamic cycle of fetal heart video, mainly because the process of manually selecting specific phase frames is easily affected by subjective factors of the doctor.
The three-dimensional convolutional neural network model is used to predict the fetal heart video frame, extract the timing characteristics of the video segment, and correct the characteristics of the cardiac change, and finally obtain a more accurate cardiac cycle.
Through model prediction and correction of results, the accuracy of the cardiac cycle is significantly improved and the dependence on doctors' subjective judgments is reduced.
Smart Images
Figure CN120168002A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video processing technologies, and particularly to a method, apparatus, device, and readable storage medium for determining a cardiac cycle. Background Art
[0002] The morphology of the fetal heart changes significantly at different phases. When an obstetrician uses ultrasound for fetal heart screening, it is generally necessary to observe or measure certain structures of the fetal heart at specific phases. For example, chamber size, blood vessel inner diameter, etc.
[0003] The current clinical operation process generally freezes a segment of fetal heart video first, then manually selects frames at specific phases frame by frame to determine the cardiac cycle, and then initiates the measurement. Among them, the process of manually selecting frames at specific phases is easily affected by the subjective factors of the doctor, resulting in inaccurate determination of the cardiac cycle.
[0004] In summary, the problem of low accuracy in determining the cardiac cycle is a technical problem that needs to be urgently solved by those skilled in the art at present. Summary of the Invention
[0005] The purpose of this application is to provide a method, apparatus, device, and readable storage medium for determining a cardiac cycle, which can predict the prediction results of the phase states of each frame in the fetal heart video through a model, and correct the prediction results. Based on the corrected results, a more accurate cardiac cycle of the fetal heart video can be obtained.
[0006] To solve the above technical problems, this application provides the following technical solutions:
[0007] A method for determining a cardiac cycle includes:
[0008] Read the fetal heart video frame by frame, and determine each current frame and several frames before the current frame as a video segment;
[0009] Use a trained three-dimensional convolutional neural network model to extract the temporal features of each video segment, and perform state prediction processing on the last frame of each video segment based on the temporal features to obtain the prediction results of the phase states of each frame in the fetal heart video; the phase state is the systolic phase or the diastolic phase;
[0010] Combine the cardiac change characteristics to correct the prediction results to obtain corrected results;
[0011] Use the corrected results to obtain the cardiac cycle of the fetal heart video.
[0012] Exemplarily, the cardiac change characteristics include continuity and periodicity. The combining the cardiac change characteristics to correct the prediction results to obtain corrected results includes:
[0013] Combined with the cardiac cycle continuity, correct the prediction result to obtain the corrected result;
[0014] and / or,
[0015] Combined with the duration of the cardiac cycle, correct the prediction result to obtain the corrected result.
[0016] Exemplarily, the step of combining the cardiac cycle continuity to correct the prediction result to obtain the corrected result includes:
[0017] Use a sliding window to read the prediction result, move the sliding window frame by frame, and each time the sliding window is moved, obtain the phase state of each frame within the current sliding window, and form a phase state sequence with the phase states of each frame; different numerical values are used in the phase state sequence to represent the diastolic phase and the systolic phase;
[0018] Perform convolution calculation on each phase state sequence using a preset low-pass filter kernel and then take the average, and determine the smoothed phase state corresponding to the middle frame as the average value; the middle frame is the frame at the middle position among the frames corresponding to the phase state sequence;
[0019] Obtain a filtering result based on each of the smoothed phase states;
[0020] Extract the end frame of the systolic phase and the end frame of the diastolic phase from the filtering result;
[0021] Correct the phase states in the prediction result that do not match the end frame of the systolic phase and the end frame of the diastolic phase to obtain the corrected result.
[0022] Exemplarily, the preset low-pass filter kernel is a sign function, and the length of the sign function is 2k, where the signs from 1 to k are -1, and the signs from k + 1 to 2k are 1, and k is less than the number of frames corresponding to half of the cardiac cycle.
[0023] Exemplarily, the step of combining the duration of the cardiac cycle to correct the prediction result to obtain the corrected result includes:
[0024] Based on the prediction result, calculate the systolic duration of each systolic phase and the diastolic duration of each diastolic phase;
[0025] Calculate the mean value of the systolic durations and determine the systolic duration variance based on the calculated mean value, calculate the mean value of the diastolic durations and determine the diastolic duration variance based on the calculated mean value;
[0026] Based on the systolic duration variance and the diastolic duration variance, eliminate the outliers in the prediction result;
[0027] After eliminating the outliers, determine a continuous prediction sequence from the prediction result;
[0028] Determine the cardiac cycle duration using the continuous prediction sequence, and correct the prediction result after removing outliers based on the cardiac cycle duration to obtain the corrected result.
[0029] Exemplarily, determining the cardiac cycle duration using the continuous prediction sequence, and correcting the prediction result after removing outliers based on the cardiac cycle duration to obtain the corrected result includes:
[0030] Calculate the target systolic average duration and the target diastolic average duration of the continuous prediction sequence;
[0031] Use the target systolic average duration and the target diastolic average duration to fit the time points of the end-diastolic frame and the end-systolic frame in the fetal heart rate video;
[0032] Use the time points to correct the phase states in the prediction result after removing outliers that do not match the time points to obtain the corrected result.
[0033] Exemplarily, after obtaining the cardiac cycle of the fetal heart rate video using the corrected result, it further includes:
[0034] Divide the intermediate phase of the fetal heart rate video based on the cardiac cycle;
[0035] And / or
[0036] Use the corrected result to calculate the average duration of the cardiac cycle;
[0037] Use the average duration to calculate the number of cycles within one minute, and determine the number of cycles as the estimated fetal heart rate value.
[0038] A cardiac cycle determination device includes:
[0039] A video segmentation module, configured to read the fetal heart rate video frame by frame, and determine each current frame and several frames before the current frame as a video segment;
[0040] A model prediction module, configured to use a trained three-dimensional convolutional neural network model to extract the temporal features of each video segment, and perform state prediction processing on the last frame of each video segment based on the temporal features to obtain the prediction result of the phase state of each frame in the fetal heart rate video; the phase state is the systolic phase or the diastolic phase;
[0041] A prediction correction module, configured to correct the prediction result in combination with the cardiac change characteristics to obtain a corrected result;
[0042] A fetal heart rate analysis module, configured to obtain the cardiac cycle of the fetal heart rate video using the corrected result.
[0043] An electronic device, comprising:
[0044] a memory for storing a computer program;
[0045] a processor for implementing the steps of the above-mentioned cardiac cycle determination method when executing the computer program.
[0046] A readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned cardiac cycle determination method are implemented.
[0047] Applying the method provided in the embodiments of the present application, reading the fetal heart video frame by frame, and determining each current frame and several frames before the current frame as a video segment; using the trained three-dimensional convolutional neural network model to extract the temporal features of each video segment, and performing state prediction processing on the last frame of each video segment based on the temporal features to obtain the prediction results of the phase states of each frame in the fetal heart video; the phase state is the systolic phase or the diastolic phase; combining the cardiac change characteristics, correcting the prediction results to obtain the corrected results; using the corrected results to obtain the cardiac cycle of the fetal heart video.
[0048] In the present application, first, the fetal heart video is read frame by frame in sequence, and each current frame and several frames before the current frame are determined as a video segment. The trained three-dimensional convolutional neural network model is used to perform state prediction processing on the last frame of each video segment, so that the prediction results of the phase states of each frame in the fetal heart video can be obtained. Among them, the phase state is the systolic phase or the diastolic phase. Since the input to the three-dimensional convolutional neural network model is a video segment, the model can extract the temporal features in the video segment, and thus perform phase state prediction based on the temporal features. Compared with predicting based on a single-frame image, the accuracy of the prediction results can be improved based on the temporal features. In addition, when making a prediction, only the last frame of the video frame is predicted to belong to the systolic phase or the diastolic phase, and the classification items are simple, so the prediction ability can be quickly learned during the training process, and the prediction accuracy can be improved. To avoid incorrect model prediction results, therefore, the cardiac change characteristics are also combined to correct the prediction results to obtain the corrected results. Finally, based on the corrected results, a more accurate cardiac cycle can be obtained.
[0049] Correspondingly, the embodiments of the present application also provide a cardiac cycle determination device, device, and readable storage medium corresponding to the above-mentioned cardiac cycle determination method, which have the above technical effects and will not be elaborated here. Description of the Drawings
[0050] To more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the related art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.
[0051] Figure 1 It is a flowchart of an implementation of a cardiac cycle determination method in an embodiment of the present application;
[0052] Figure 2 It is a schematic diagram of a filtering kernel in an embodiment of the present application;
[0053] Figure 3 It is a square wave corresponding to a prediction result in an embodiment of the present application;
[0054] Figure 4 It is another square wave corresponding to a prediction result in an embodiment of the present application;
[0055] Figure 5 It is a schematic diagram of a triangular wave in an embodiment of the present application;
[0056] Figure 6 It is a schematic diagram of a cardiac cycle determination method in an embodiment of the present application;
[0057] Figure 7 It is a schematic structural diagram of a cardiac cycle determination device in an embodiment of the present application;
[0058] Figure 8 It is a schematic structural diagram of an electronic device in an embodiment of the present application;
[0059] Figure 9 It is a specific structural diagram of an electronic device in an embodiment of the present application. Detailed implementation manners
[0060] To enable those skilled in the art to better understand the solutions of the present application, the following will further elaborate on the present application in conjunction with the accompanying drawings and specific implementation manners. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.
[0061] Please refer to Figure 1 , Figure 1 It is a flowchart of a cardiac cycle determination method in an embodiment of the present application. This method can be applied to electronic devices such as ultrasonic devices. The following will be described by taking an ultrasonic device as an example. The method includes the following steps:
[0062] S101. Read the fetal heart rate video frame by frame, and determine each current frame and several frames before the current frame as a video segment.
[0063] Optionally, the ultrasonic device can obtain the fetal heart rate video in real time during ultrasonic scanning, or can call back the pre-stored fetal heart rate video. Further, after reading the fetal heart rate video, the fetal heart rate video can be directly used as the analysis object for video segment division; to reduce the computational complexity, several video frames can also be extracted at intervals from the fetal heart rate video (for example, extract one video frame every other frame), and the analysis object can be obtained by arranging them in order and then perform video segment division.
[0064] In this embodiment, the number of frames N of each video segment can be specified in advance (N is an adjustable parameter greater than 1, which can be set and adjusted according to actual needs, for example, N can be 8). Then, during the process of reading the fetal heart rate video frame by frame in sequence, the current frame and the first N - 1 frames before the current frame can be determined as a video segment.
[0065] In this embodiment, during the process of reading the fetal heart rate video to obtain video segments, the entire fetal heart rate video can be read. In the case of a long fetal heart rate video, a part of the fetal heart rate video can also be read.
[0066] There can be only one frame difference between two adjacent video segments. Of course, in practical applications, there can also be multiple frame differences. In this embodiment, an example is given where there is only one frame difference between adjacent video segments and the entire fetal heart rate video is read.
[0067] For example, if the total number of fetal heart rate video frames is M frames, then the first N frames can be read in sequence and these N frames can be determined as a video segment. Then continue to read the (N + 1)-th frame, and then determine the frames from the 2nd to the (N + 1)-th frame as another video segment. Continue to read the (N + 2)-th frame, and then determine the frames from the 3rd to the (N + 2)-th frame as another video segment, and so on, until the M-th frame of the fetal heart rate video is read.
[0068] In practical applications, when reading the fetal heart rate video, a sliding window can be used for reading. Specifically, the length of the sliding window can be set to N, and each time it slides frame by frame. All the frames corresponding to the sliding window after each movement are determined as a video segment, and each time it slides only one video frame.
[0069] S102. Use the trained three-dimensional convolutional neural network model to extract the temporal features of each video segment, and perform state prediction processing on the last frame of each video segment based on the temporal features to obtain the prediction results of the phase states of each frame in the fetal heart rate video.
[0070] Among them, the phase state is the systolic phase or the diastolic phase. In this embodiment, the categories of fetal heart sections include the four-chamber view, the left ventricular outflow tract view, the right ventricular outflow tract view, the three-vessel view, and the three-vessel trachea view. The phase state of the section is only divided into two categories: the systolic phase and the diastolic phase. The image state of the section between the end-diastolic phase and the end-systolic phase of the fetal heart is defined as the systolic phase, and the image state of the section between the end-systolic phase and the end-diastolic phase is defined as the diastolic phase.
[0071] In this embodiment, a three-dimensional convolutional neural network model can be pre-trained, so as to use the three-dimensional convolutional neural network model to perform state prediction processing on the last frame of each video segment, so as to obtain the prediction results of the phase states of each frame in the fetal heart video. The prediction processing process can specifically be to extract all the image features and temporal features of the currently input video segment, and then predict whether the last frame of the video segment is in the diastolic phase or the systolic phase based on the image features and temporal features.
[0072] It should be noted that in this embodiment, since the input to the three-dimensional convolutional neural network model (also simply referred to as the model) is a video segment of N frames, that is, the first N - 1 frames in the fetal heart video cannot predict the fetal heart phase, so the prediction results will not have the corresponding prediction results for the first N - 1 frames. Given that the number of frames in the fetal heart video is much larger than N, therefore, the loss of N - 1 frames has little impact on the overall processing, and the prediction results of the lost N frames can be corrected subsequently.
[0073] The prediction results specifically include the phase state of the last frame in each video segment, such as the systolic phase or the diastolic phase. Since the video segment reads the fetal heart video frame by frame, and each current frame and several frames before the current frame are determined as a video segment, therefore, the number of frames corresponding to the phase state predictions in the prediction results is the same as the number of video segments input into the three-dimensional convolutional neural network model for state prediction processing.
[0074] If the fetal heart video has a total of M frames and each video segment has N frames, the prediction results can include the prediction results of the phase states of each frame corresponding to the Nth frame to the Mth frame in the fetal heart video. If different numerical values are used to represent different prediction results, the prediction results can specifically be a sequence composed of the numerical values corresponding to the phase states of each frame from the Nth frame to the Mth frame. If the prediction results are represented graphically in sequence, a square wave diagram can be obtained.
[0075] For example, in practical applications, a sequence can be used to represent the prediction results. 1 can be used to represent the diastolic phase and 0 can be used to represent the systolic phase. Of course, a signal wave can also be used to represent the prediction results. For example, the presence of a signal can be used to represent the diastolic phase and the absence of a signal can be used to represent the systolic phase.
[0076] Process of training a three-dimensional convolutional neural network model: According to the steps of step S101 above, read the fetal heart video to obtain a number of video segments; then, label each video segment with the phase state label of the last frame; use the labeled video segments as training samples and test samples. Use the training samples to train the three-dimensional convolutional neural network model, and use the test samples to test the three-dimensional convolutional neural network model, and finally obtain a three-dimensional convolutional neural network model that can predict the phase state of the last frame in the input video segment. For the settings of the number of iterations, loss function, etc. during the training process, the training process of related models can be referred to. That is to say, in this embodiment, it is only necessary to ensure that the samples used for training are video frames with phase state labels for the last frame, and during the training process, the three-dimensional convolutional neural network model can learn the ability to predict the phase state of the last frame of the video segment.
[0077] Exemplarily, the three-dimensional convolutional neural network model can at least include multiple feature extraction units (the number of feature extraction units can be greater than or equal to the number of video frames in the video segment), a fusion unit, a stacking unit, etc. Specifically, the temporal features of each video frame in the video segment can be extracted in parallel by the feature extraction units of the three-dimensional convolutional neural network model, and then these temporal features can be integrated by means of a fusion unit, a stacking unit, etc., and based on the integrated result, state prediction processing is performed on the last frame of the video segment to obtain the prediction result of the phase state of the current frame. It should be noted that in this embodiment, the three-dimensional convolutional neural network model can be various 3D convolutional neural network classification models, such as 3D resnet18.
[0078] In specific implementation, video segments can be input into the trained three-dimensional convolutional neural network model one by one, and the output of the three-dimensional convolutional neural network model is the prediction of the phase state of the last frame in the currently input video segment. After completing the prediction of each last frame in all video segments corresponding to the fetal heart video, the prediction results of the phase states of each frame in the fetal heart video can be obtained.
[0079] S103. Combine the cardiac motion change characteristics to correct the prediction result to obtain a corrected result.
[0080] Cardiac cycle: When the heart relaxes, the internal pressure decreases, and the blood in the vena cava flows back into the heart. When the heart contracts, the internal pressure increases, and the blood is pumped into the artery. Each contraction and relaxation of the heart constitutes a cardiac cycle. In a cardiac cycle, first, the two atria contract, and the contraction of the right atrium is slightly earlier than that of the left atrium. After the atria begin to relax, the two ventricles contract, and the contraction of the left ventricle is slightly earlier than that of the right ventricle. In the later stage of ventricular relaxation, the atria begin to contract again.
[0081] Since the cardiac motion changes have certain characteristics, the prediction result can be corrected based on these change characteristics to obtain a corrected result.
[0082] For example, in an ideal state, the turning point from diastole to systole is the end of diastole, and the turning point from the end of systole to the end of diastole is the end of systole. However, during actual model prediction, there may be frames with incorrect predictions, and these incorrect predictions need to be masked. In addition, according to clinical prior knowledge, during the fetal heartbeat, the cardiac cycle is generally stable within a relatively short period of time, and the systole and diastole are in an alternating state. Therefore, some prediction results that do not conform to the duration and alternating changes of the cardiac cycle can be corrected.
[0083] In a specific embodiment of the present application, the cardiac motion change characteristics include continuity and periodicity. Combining the cardiac cycle change characteristics, the prediction result is corrected to obtain a corrected result, including:
[0084] Combining cardiac continuity, the prediction result is corrected to obtain a corrected result;
[0085] And / or, combining the duration of the cardiac cycle, the prediction result is corrected to obtain a corrected result.
[0086] Among them, cardiac continuity has the following characteristics: during the cardiac cycle, after the end of systole is diastole, and after the end of diastole is systole. If several frames before and after a certain frame in the prediction result are all diastole, and this frame is judged as systole, there is obviously an incorrect prediction and it needs to be corrected.
[0087] Among them, the duration of the cardiac cycle is relatively stable. If several consecutive frames are all diastole and the cumulative duration significantly exceeds the duration of one diastole, then there may be frames with incorrect predictions among these frames and they need to be corrected. The correction can be based on the duration of a single cycle. For example, the other consecutive diastole frames outside one diastole are corrected to systole.
[0088] Among them, "and / or" means that in actual applications, when correcting the prediction result, the correction can be based on at least one of the characteristics of cardiac continuity and the duration of the cardiac cycle. For example, the correction can be performed separately using cardiac continuity, or separately using the duration of the cardiac cycle; it can also be performed by combining cardiac continuity and the duration of the cardiac cycle. When the two are combined for correction, the correction can be first performed based on cardiac continuity and then the result of the first correction is secondarily corrected using the duration of the cardiac cycle. Of course, the correction can also be first performed based on the duration of the cardiac cycle and then the result of the first correction is secondarily corrected using cardiac continuity.
[0089] In a specific embodiment of the present application, combining cardiac continuity, the prediction result is corrected to obtain a corrected result, including:
[0090] Read the prediction results using a sliding window, move the sliding window frame by frame, and each time the sliding window is moved, obtain the phase states of each frame within the current sliding window, and form a phase state sequence with the phase states of each frame; different numerical values are used in the phase state sequence to represent the diastolic and systolic phases;
[0091] Perform convolution calculation on each phase state sequence using a preset low-pass filter kernel and then take the average, and determine the smoothed phase state corresponding to the middle frame as the average value; the middle frame is the frame at the middle position among the frames corresponding to the phase state sequence.
[0092] Obtain the filtering result based on each smoothed phase state;
[0093] Extract the end-systolic frame and end-diastolic frame from the filtering result;
[0094] Correct the phase states in the prediction result that do not match the end-systolic frame and end-diastolic frame to obtain the correction result.
[0095] For ease of description, the above steps will be combined and described below.
[0096] The end-diastolic frame (which can be simply referred to as the end-diastolic frame) and the end-systolic frame (which can be simply referred to as the end-systolic frame) are not only the phases most commonly used in clinical practice for fetal heart rate measurement, but also the most commonly used frames for demarcating a single cardiac cycle during continuous heartbeats. Using a three-dimensional convolutional neural network, the systolic and diastolic phases of the fetal heart are predicted. Ideally, the turning point from the diastolic phase to the systolic phase is the end-diastolic phase, and the turning point from the end-systolic phase to the end-diastolic phase is the end-systolic phase. However, there may be frames with incorrect predictions during actual model prediction, so these incorrect predictions need to be masked.
[0097] In this embodiment, these incorrect predictions can be masked based on filtering. Specifically, the prediction results can be read first based on the method of moving the sliding window frame by frame, so as to read the phase states of each frame corresponding to the current sliding window, and then the phase states of these frames are formed into a phase state sequence. That is, each time the sliding window is moved, a phase state sequence can be obtained.
[0098] Different numerical values in this phase state sequence represent the diastolic and systolic phases. For example, 1 can be used to represent the diastolic phase and 0 can be used to represent the systolic phase.
[0099] For ease of calculation, the length of this phase state sequence (i.e., the length / size of the sliding window) can be kept consistent with the length of the preset low-pass filter kernel. When the symbolic data of the filter kernel, that is, the length is 2k, the continuous 2k-frame time series state sequence is convolved with the filter kernel to obtain the smoothed phase state of the middle frame of the 2k frames.
[0100] It should be noted that among them, the middle frame refers to the phase state corresponding to this frame being in the central position of the time sequence state sequence. For example, when the length of the time sequence state sequence is 2k, the middle frame refers to k or k + 1. In practical applications, the k-th frame (or the (k + 1)-th frame) in the continuous 2k frames can be uniformly determined as the middle frame.
[0101] Since the prediction results are read frame by frame, the time sequence state sequence can be regarded as the time sequence state sequence of this middle frame. That is, by reading the time sequence states corresponding to each frame in the current sliding window, therefore, by reading the prediction results, multiple time sequence state sequences of a specific length can be obtained. Among them, the specific length is the length of the sliding window. For example, when the length of the sliding window is 8, a total of 8 frames of time sequence states can be read each time, and these 8 frames of time sequence states are spliced into a time sequence state sequence with a total of 8 values. The upper limit of the number of time sequence state sequences can be the difference between the total number of frames corresponding to the prediction results and the specific length.
[0102] For the method based on filtering technology, the cross-correlation calculation filtering is performed on the prediction results of the three-dimensional network using a preset low-pass filter kernel. The filtering calculation process is to perform convolution calculation on each time sequence state sequence using the preset low-pass filter kernel and then take the average, and determine the average value as the smoothed time sequence state corresponding to the middle frame. That is to say, perform convolution calculation on each time sequence state sequence with the preset low-pass filter kernel, and then take the average of the obtained results, and the obtained average value can be determined as the smoothed time sequence state corresponding to the middle frame.
[0103] After filtering each time sequence state sequence, all the smoothed time sequence states can be spliced to obtain the filtering result.
[0104] Since different time sequence states are represented by different values, after filtering, the positions of the peaks and valleys presented in this filtering result are the positions of the changes in the time sequence states. Therefore, the end-systolic frame and the end-diastolic frame can be extracted based on this filtering result.
[0105] Based on this end-systolic frame and end-diastolic frame, the time sequence states in the prediction results that do not match them can be modified to obtain the corrected result. For example, if the time sequence state of a certain end-systolic frame in the prediction result is diastole, the time sequence state corresponding to this frame can be corrected to systole; if the time sequence state of a certain end-diastolic frame in the prediction result is systole, the time sequence state corresponding to this frame can be corrected to diastole; for the time sequence states that are diastole between adjacent end-diastolic frames and end-systolic frames, they are corrected to systole; for the time sequence states that are systole between adjacent end-systolic frames and end-diastolic frames, they are corrected to diastole.
[0106] In a specific implementation manner of the present application, the preset low-pass filter kernel is a sign function, and the length of the sign function is 2k, where the signs of 1 to k are -1, and the signs of k + 1 to 2k are 1, and k is less than the number of frames corresponding to half a cardiac cycle.
[0107] The specific implementation of the sign function is as follows Figure 2 As shown, it consists of k consecutive 1s and k -1s. Wherein, k is an adjustable parameter. Generally, the lower the sampling rate, the smaller k is, the smaller N is, and the range of 2k does not exceed half a cardiac cycle, such as k ≥ 4. Based on this sign function, it is possible to effectively extract the frame at the transformation moment from the systolic phase to the diastolic phase or from the diastolic phase to the systolic phase in the fetal heart prediction result.
[0108] Among them, convolution calculation refers to the process of performing convolution calculation on each phase state sequence with the filter kernel to obtain the convolution result.
[0109] For example, if a phase state sequence is 1, 1, 1, 1, 1, -1, -1, -1, and the filter kernel is: -1, -1, -1, -1, 1, 1, 1, 1, the convolution calculation is performed on the two to get 6. Divide this value by the filter kernel length 8 to get 0.75, which is used as the filter value of the frame (the middle frame) corresponding to the middle position of the convolution kernel in this phase state sequence. Since the filter kernel length is even, the fourth frame is uniformly taken to correspond to this filter value (i.e., the smoothed phase state). Then the sliding window moves one bit backward to get another phase state sequence: 1, 1, 1, 1, -1, -1, -1, -1. At this time, the convolution calculation is performed to get 8. That is, every time the sliding window moves one bit backward, a filter value of one frame can be calculated, and finally the filter results of all prediction results can be obtained. The filter result presents a triangular wave shape. The more accurate the prediction result, the more regular the triangular wave shape. During the filtering process, the filter results corresponding to 2k - 1 frames may be lost. Given that the number of frames in a fetal heart video is much larger than 2k - 1, the results of the lost 2k - 1 frames can be ignored, and the lost frames can also be inferred based on the filter results later.
[0110] For example: If 1 represents the diastolic phase and 0 represents the systolic phase, the prediction result obtained based on the model can be a square wave composed of 01, as Figure 3 shown; for the convenience of calculation, 0 can be replaced by -1. Then, a square wave composed of +1 and -1 is obtained, as Figure 4 shown, and then the cross-correlation calculation is performed on this square wave with the filter kernel. If so, a triangular wave can be obtained, as Figure 5As shown, at the peak and trough of the triangular wave are the jump positions corresponding to the prediction results. After filtering, the end-diastolic phase shows a peak value, and the end-systolic phase shows a trough value. Subsequently, a high threshold can be set to extract the frames corresponding to the peak value as the end-diastolic phase, and a low threshold can be set to extract the frames corresponding to the trough value as the end-systolic phase. In this way, most of the frames with prediction errors will be eliminated / corrected in this process.
[0111] That is to say, after filtering the prediction results output by the initial model, the phase states of each frame can be re-determined based on the end-diastolic and end-systolic phases found corresponding to the filtered triangular wave.
[0112] In a specific implementation manner of the present application, in combination with the duration of the cardiac cycle, the prediction results are corrected to obtain corrected results, including:
[0113] Based on the prediction results, calculate the systolic duration of each systolic phase and the diastolic duration of each diastolic phase;
[0114] Calculate the mean of the systolic durations and determine the systolic duration variance based on the calculated mean, calculate the mean of the diastolic durations and determine the diastolic duration variance based on the calculated mean;
[0115] Based on the systolic duration variance and the diastolic duration variance, eliminate the outliers in the prediction results;
[0116] After eliminating the outliers, determine a continuous prediction sequence from the prediction results;
[0117] Use the continuous prediction sequence to determine the cardiac cycle duration, and correct the prediction results after eliminating the outliers based on the cardiac cycle duration to obtain corrected results.
[0118] For ease of description, the above steps will be combined and described below.
[0119] Based on the prediction results obtained from the model or further based on the corrected results after filtering the prediction results, the results of the end-systolic and end-diastolic frames of a fetal cardiac ultrasound video may still not be accurate enough. Specifically, there will be some frames with prediction errors or omissions.
[0120] Generally, during the fetal heart beating process, the cardiac cycle is generally basically stable within a relatively short time period, and the systolic and diastolic phases are in an alternating state. Therefore, in this embodiment, statistical information is used to improve the prediction result or the corrected result based on filtering correction. Specifically, the prediction result can be used as a candidate frame. First, determine the durations (i.e., lengths) of all predicted diastolic and systolic phases, respectively calculate the mean and variance of the systolic and diastolic phases, and exclude the frames with too large variance as outliers. Among them, outliers can be determined based on a threshold. For example, frames with systolic duration variance and / or diastolic duration variance exceeding a preset threshold are determined as outliers. The size of the threshold can be determined or adjusted according to actual needs and will not be elaborated here one by one.
[0121] Since outliers are removed, there will be frames with missing phase states, resulting in discontinuous prediction results. To better clarify the cardiac cycle duration, from the prediction result after removing outliers, find the sequence corresponding to the continuous segment without frame missing phase states as the continuous prediction sequence.
[0122] Among them, the cardiac cycle duration is determined using the continuous prediction sequence, and the prediction result after removing outliers is corrected based on the cardiac cycle duration to obtain the corrected result, including:
[0123] Calculate the target systolic average duration and the target diastolic average duration of the continuous prediction sequence;
[0124] Use the target systolic average duration and the target diastolic average duration to fit the time points of the end-diastolic frame and the end-systolic frame in the fetal heart video;
[0125] Use the time points to correct the phase states that do not match the time points in the prediction result after removing outliers to obtain the corrected result.
[0126] Among them, the target systolic average duration refers to the average duration of the systolic phase determined based on the continuous prediction sequence; the target diastolic average duration refers to the average duration of the diastolic phase determined based on the continuous prediction sequence. Specifically, the duration of the frames corresponding to adjacent multiple diastolic phases in the continuous prediction sequence can be determined as the duration of the diastolic phase, the sum of the durations of multiple diastolic phases is calculated and then the average value is calculated, and this average value is determined as the target diastolic average duration; the duration of the frames corresponding to adjacent multiple systolic phases in the continuous prediction sequence can be determined as the duration of the systolic phase, the sum of the durations of multiple systolic phases is calculated and then the average value is calculated, and this average value is determined as the target diastolic average duration.
[0127] That is to say, the longest continuous prediction segment after excluding outliers is selected. Then, the average lengths of the diastolic and systolic periods are calculated in this segment, and the time points of other end-diastolic frames and end-systolic frames in the video are fitted based on the average length information. These time points can be used to determine which frame is the end-diastolic frame and which frame is the end-systolic frame. For the frame whose previous adjacent time point is the end-diastolic frame and the next time point is the end-systolic frame, it is the systolic period; for the frame whose previous adjacent time point is the end-systolic frame and the next time point is the end-diastolic frame, it is the diastolic period.
[0128] Based on this, after the time points of the end-systolic frame and the end-diastolic frame are determined, the phase states of each frame in the prediction result can be modified based on these end-systolic frames and end-diastolic frames. The correction process can refer to the specific process of correcting the prediction result based on the end-systolic frame and the end-diastolic frame determined by the continuity feature as described above, and will not be elaborated here one by one.
[0129] After correction, the phase states of each frame in the final prediction can be obtained.
[0130] S104. Use the correction result to obtain the cardiac cycle of the fetal heart video.
[0131] In the correction result, the phase state of each frame has been shown. It is clear that an adjacent systolic period and a diastolic period can be regarded as a cardiac cycle. Therefore, based on the prediction results of the systolic and diastolic periods, each cardiac cycle in the fetal heart video can be determined.
[0132] In a specific embodiment of the present application, after using the correction result to obtain the cardiac cycle of the fetal heart video, it further includes: dividing the middle phase based on the cardiac cycle for the fetal heart video.
[0133] Among them, using the correction result to subdivide the fetal phases includes:
[0134] Using the correction result to demarcate all cardiac cycles in the fetal heart video;
[0135] Dividing the middle phase for the cardiac cycles obtained after demarcation.
[0136] Based on the correction result, that is, the results of the end-systolic and end-diastolic periods of the fetal heart, the cardiac cycles in the fetal echocardiogram video can be directly demarcated. Within any cardiac cycle obtained after demarcation, according to clinical knowledge, a more detailed middle phase division is further carried out. After obtaining the middle phase, the characteristics of the middle phase can be compared with the ultrasound image to find abnormal fetal heart activities.
[0137] In a specific embodiment of the present application, after obtaining the cardiac cycle of the fetal heart video using the correction result, it further includes: calculating the average duration of the cardiac cycle using the correction result; calculating the number of cycles within one minute using the average duration, and determining the number of cycles as the estimated fetal heart rate value.
[0138] Among them, estimating the fetal heart rate using the correction result includes:
[0139] Calculating the average duration of all predicted cardiac cycles of the fetal heart video using the correction result;
[0140] Calculating the number of cycles within one minute using the average duration;
[0141] Determining the number of cycles as the estimated fetal heart rate value.
[0142] After obtaining the average cardiac cycle of the fetal heart by averaging all predicted cycles of the entire video segment, calculate the number of cycles within one minute as the estimated fetal heart rate value.
[0143] The changes in intraventricular pressure, ventricular volume, blood flow, and valve activity at each phase of the cardiac cycle are centered around the systolic and diastolic activities of the ventricle. The entire cardiac cycle proceeds through 8 intermediate phases.
[0144] Isovolumetric contraction phase: The ventricle begins to contract when it reaches the peak of the R wave of the electrocardiogram. The strong contraction of the ventricular muscle causes a sharp increase in intraventricular pressure. When it exceeds the atrial pressure, the blood in the left and right ventricles respectively pushes the left and right atrioventricular valves to close them. Since the papillary muscles and chordae tendineae tighten the atrioventricular valves and prevent them from flipping into the atrium, and the circular muscles at the atrioventricular junction contract to narrow the diameter of the atrioventricular junction, both can prevent the backflow of ventricular blood into the atrium. At this time, the intraventricular pressure rises sharply, but when it does not exceed the aortic pressure (about 80 mmHg at the end of diastole) and the pulmonary artery pressure (about 8 - 10 mmHg at the end of diastole), the semilunar valves remain closed. During this short period (on average 0.05 seconds in the human body), both the atrioventricular valves and the semilunar valves are closed, the length from the apex to the base of the ventricle decreases, the ventricle becomes rounder, the ventricular muscle tension increases, and the ventricular volume remains unchanged, so it is called the isovolumetric contraction phase.
[0145] Rapid ejection phase: The ventricular muscle continues to contract, the tension increases, the intraventricular pressure rises sharply, quickly exceeding the aortic pressure and the pulmonary artery pressure, and both semilunar valves are pushed open, and blood is ejected into the aorta and the pulmonary artery and quickly reaches the maximum rate. At the end of the rapid ejection phase, the ventricular pressure reaches its peak (about 120 - 130 mmHg in the left ventricle and about 24 - 25 mmHg in the right ventricle). This phase lasts for an average of 0.09 seconds, accounting for about 1 / 3 of the systolic phase of the heart, and the ejected blood volume accounts for 80 - 85% of each stroke volume.
[0146] Reduced ejection period: During this period, the contractile force of the ventricle and the intraventricular pressure begin to decrease, and the ejection velocity slows down. At this time, the intraventricular pressure is slightly lower than the aortic pressure (the difference is a few millimeters of mercury), but because the total energy of ventricular contraction (pressure energy plus kinetic energy) is still higher than the total energy level in the aorta, blood continues to be ejected from the ventricle, lasting for an average of 0.13 seconds. Then it enters the ventricular diastole.
[0147] Protodiastole: The ventricle begins to relax, ejection stops, and the intraventricular pressure drops rapidly. The left ventricular pressure was already slightly lower than the aortic pressure, and the right ventricular pressure quickly drops below the pulmonary artery pressure. At this time, the semilunar valves on both sides close rapidly to prevent blood from flowing back into the ventricle. The period from the start of ventricular diastole to the closure of the semilunar valves is called protodiastole, lasting about 0.04 seconds.
[0148] Isovolumic relaxation period: When the semilunar valves close, the intraventricular pressure is still higher than the atrial pressure. The atrioventricular valves are still closed. When the intraventricular pressure continues to drop below the atrial pressure, the atrioventricular valves open. During this short period from the closure of the semilunar valves to the opening of the atrioventricular valves, the intraventricular pressure drops rapidly while the ventricular volume remains basically unchanged, which is called the isovolumic relaxation period, lasting about 0.08 seconds.
[0149] Rapid filling period: After the atrioventricular valves open, the ventricular volume expands rapidly. At this time, the intraventricular pressure is even lower than the atrial pressure, and the blood accumulated in the atrium and large veins rushes into the ventricle rapidly, lasting about 0.11 seconds. About 2 / 3 of the blood in the ventricle is filled during this period.
[0150] Reduced filling period: (Late diastole) As the ventricle fills rapidly with blood, the speed of blood flowing from the veins back into the ventricle through the atrium gradually slows down, the atrioventricular pressure difference decreases, and the ventricular volume further increases. This period is called the reduced filling period, lasting about 0.19 seconds. Then the atrium begins to contract.
[0151] Atrial systole: At the end of ventricular diastole, the atrium begins to contract, the atrial pressure rises to eject the remaining blood into the ventricle, further increasing the ventricular filling degree, and the ventricular pressure also shows a small increase. The relaxation of the atrium reduces the atrial pressure, which helps to close the atrioventricular valves. Therefore, the atrioventricular valves already have a tendency to close before ventricular contraction. By the start of the next isovolumic contraction, one cardiac cycle is completed.
[0152] For example: Please refer to Figure 6 , first use the trained three-dimensional convolutional neural network model to predict the fetal heart phase of the fetal heart video, and obtain the 01 square wave as shown in Figure 3 , and then through filtering processing, the triangular wave as shown in Figure 5 can be obtained; candidate frames are extracted from the triangular wave, and then fitting processing is performed based on the duration information of the period to obtain the correction result (the final result shown in the figure).
[0153] In this embodiment, Figures 2 to 6 the unit of the abscissa of the coordinates involved in Figures 2 to 6 is frame, and the unit of the ordinate is the numerical value.
[0154] Applying the method provided by the embodiment of the present application, the fetal heart rate video is read frame by frame, and each current frame and several frames before the current frame are determined as a video segment; using the trained three-dimensional convolutional neural network model, the temporal features of each video segment are extracted, and based on the temporal features, state prediction processing is performed on the last frame of each video segment to obtain the prediction results of the phase states of each frame in the fetal heart rate video; the phase state is the systolic phase or the diastolic phase; combining the cardiac motion change features, the prediction results are corrected to obtain the corrected results; using the corrected results, the cardiac cycle of the fetal heart rate video is obtained.
[0155] In the present application, first, the fetal heart rate video is read frame by frame in order, and each current frame and several frames before the current frame are determined as a video segment. Using the trained three-dimensional convolutional neural network model to perform state prediction processing on the last frame of each video segment, the prediction results of the phase states of each frame in the fetal heart rate video can be obtained. Among them, the phase state is the systolic phase or the diastolic phase. Since the input to the three-dimensional convolutional neural network model is a video segment, the model can extract the temporal features in the video segment, and thus perform phase state prediction based on the temporal features. Compared with predicting based on a single-frame image, the accuracy of the prediction results can be improved based on the temporal features. In addition, when making a prediction, only the last frame of the video frame is predicted to belong to the systolic phase or the diastolic phase, and the classification items are simple, and the prediction ability can be quickly learned during the training process, improving the prediction accuracy. To avoid incorrect model prediction results, therefore, the cardiac motion change features are also combined to correct the prediction results, so as to obtain the corrected results. Finally, based on the corrected results, a more accurate cardiac cycle can be obtained.
[0156] Corresponding to the above method embodiment, the embodiment of the present application also provides a video processing device, and the video processing device described below can be correspondingly referred to the video processing method described above.
[0157] See Figure 7 As shown in Figure 7 , the device includes the following modules:
[0158] A video segmentation module 101, configured to read the fetal heart rate video frame by frame, and determine each current frame and several frames before the current frame as a video segment;
[0159] A model prediction module 102, configured to use the trained three-dimensional convolutional neural network model to extract the temporal features of each video segment, and perform state prediction processing on the last frame of each video segment based on the temporal features to obtain the prediction results of the phase states of each frame in the fetal heart rate video; the phase state is the systolic phase or the diastolic phase;
[0160] The prediction correction module 103 is used to combine the cardiac motion change features to correct the prediction result and obtain a corrected result.
[0161] The fetal heart rate analysis module 104 is used to obtain the cardiac cycle of the fetal heart rate video by using the corrected result.
[0162] Applying the device provided by the embodiment of the present application, the fetal heart rate video is read frame by frame, and each current frame and several frames before the current frame are determined as a video segment; the trained three-dimensional convolutional neural network model is used to extract the temporal features of each video segment, and based on the temporal features, state prediction processing is performed on the last frame of each video segment to obtain the prediction result of the phase state of each frame in the fetal heart rate video; the phase state is the systolic phase or the diastolic phase; combining the cardiac motion change features, the prediction result is corrected to obtain a corrected result; using the corrected result, the cardiac cycle of the fetal heart rate video is obtained.
[0163] In the present application, first, the fetal heart rate video is read frame by frame in sequence, and each current frame and several frames before the current frame are determined as a video segment. The trained three-dimensional convolutional neural network model is used to perform state prediction processing on the last frame of each video segment, so that the prediction result of the phase state of each frame in the fetal heart rate video can be obtained. Among them, the phase state is the systolic phase or the diastolic phase. Since the input to the three-dimensional convolutional neural network model is a video segment, the model can extract the temporal features in the video segment, and thus perform phase state prediction based on the temporal features. Compared with predicting based on a single-frame image, the accuracy of the prediction result can be improved based on the temporal features. In addition, when making a prediction, only the last frame of the video frame is predicted to belong to the systolic phase or the diastolic phase, and the classification items are simple, so the prediction ability can be quickly learned during the training process, and the prediction accuracy can be improved. To avoid incorrect prediction results of the model, therefore, the cardiac motion change features are also combined to correct the prediction result, so as to obtain a corrected result. Finally, based on the corrected result, a more accurate cardiac cycle can be obtained.
[0164] In a specific embodiment of the present application, the cardiac motion change features include continuity and periodicity. The prediction correction module includes:
[0165] The continuity correction sub-module is specifically used to combine the cardiac continuity to correct the prediction result and obtain a corrected result.
[0166] In a specific embodiment of the present application, the cardiac motion change features include continuity and periodicity. The prediction correction module includes:
[0167] The periodicity correction sub-module is used to combine the duration of the cardiac cycle to correct the prediction result and obtain a corrected result.
[0168] In a specific embodiment of the present application, the continuity correction sub-module includes:
[0169] A sequence reading unit for reading the prediction results using a sliding window, moving the sliding window frame by frame, and obtaining the phase states of each frame within the current sliding window each time the sliding window is moved, and constructing a phase state sequence from the phase states of each frame; different numerical values are used in the phase state sequence to represent the diastolic and systolic phases;
[0170] A convolution calculation unit for performing convolution calculation on each phase state sequence using a preset low-pass filter kernel and then taking the average, and determining the smoothed phase state corresponding to the middle frame as the average value; the middle frame is the frame at the middle position among the frames corresponding to the phase state sequence;
[0171] A filtering result acquisition unit for obtaining a filtering result based on each smoothed phase state;
[0172] An end-frame extraction unit for extracting the end frame of the systolic phase and the end frame of the diastolic phase from the filtering result;
[0173] A phase correction unit for correcting the phase states in the prediction results that do not match the end frame of the systolic phase and the end frame of the diastolic phase to obtain a correction result.
[0174] In a specific embodiment of the present application, the preset low-pass filter kernel is a sign function, the length of the sign function is 2k, where the signs of 1 to k are -1, and the signs of k + 1 to 2k are 1, and k is less than the number of frames corresponding to half of the cardiac cycle.
[0175] In a specific embodiment of the present application, the periodic correction sub-module includes:
[0176] A duration calculation unit for calculating the systolic duration of each systolic phase and the diastolic duration of each diastolic phase based on the prediction results;
[0177] A variance calculation unit for calculating the mean of the systolic durations and determining the systolic duration variance based on the calculated mean, calculating the mean of the diastolic durations and determining the diastolic duration variance based on the calculated mean;
[0178] An outlier removal unit for removing outliers in the prediction results based on the systolic duration variance and the diastolic duration variance;
[0179] A continuous prediction sequence determination unit for determining a continuous prediction sequence from the prediction results after removing the outliers;
[0180] A phase correction unit for using the continuous prediction sequence to determine the cardiac cycle duration and correcting the prediction results after removing the outliers based on the cardiac cycle duration to obtain a correction result.
[0181] In a specific embodiment of the present application, the phase correction unit includes:
[0182] A target duration calculation subunit, configured to calculate the target systolic average duration and the target diastolic average duration of a continuous prediction sequence;
[0183] An end-frame time point determination subunit, configured to fit the time points of the diastolic end frame and the systolic end frame in the fetal heart rate video by using the target systolic average duration and the target diastolic average duration;
[0184] A correction subunit, configured to correct the phase states in the prediction result that do not match the time points after removing outliers by using the time points, so as to obtain a correction result.
[0185] In a specific embodiment of the present application, it further includes:
[0186] An intermediate phase division module, configured to divide the intermediate phases of the fetal heart rate video based on the cardiac cycle after obtaining the cardiac cycle of the fetal heart rate video by using the correction result.
[0187] In a specific embodiment of the present application, it further includes:
[0188] A heart rate estimation module, configured to calculate the average duration of the cardiac cycle by using the correction result; calculate the number of cycles within one minute by using the average duration, and determine the number of cycles as the fetal heart rate estimation value.
[0189] Corresponding to the above method embodiment, an embodiment of the present application further provides an electronic device, and an electronic device described below can be mutually referred to with a video processing method described above.
[0190] See Figure 8 As shown, the electronic device includes:
[0191] A memory 332, configured to store a computer program;
[0192] A processor 322, configured to implement the steps of the video processing method in the above method embodiment when executing the computer program.
[0193] Specifically, please refer to Figure 9 , Figure 9FIG. 0 is a schematic structural diagram of an electronic device provided in this embodiment. The electronic device may vary significantly due to different configurations or performances, and may include one or more central processing units (CPUs) 322 (for example, one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. Among them, the memory 332 may be a transient storage or a persistent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the data processing device. Further, the processor 322 may be configured to communicate with the memory 332 and execute a series of instruction operations in the memory 332 on the electronic device 301.
[0194] The electronic device 301 may further include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.
[0195] The steps in the video processing method described above may be implemented by the structure of the electronic device.
[0196] Corresponding to the above method embodiment, the embodiment of the present application also provides a readable storage medium. The following-described readable storage medium may be referred to in correspondence with the above-described video processing method.
[0197] A readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video processing method in the above method embodiment are implemented.
[0198] The readable storage medium may specifically be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc., various readable storage media that can store program codes.
[0199] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments may be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts may be referred to the description of the method part.
[0200] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.
[0201] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.
[0202] Finally, it should also be noted that in this article, relationships such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "including", "comprising" or any other variant are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.
[0203] Specific examples are used in this article to elaborate on the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.
Claims
1. A method for determining a cardiac cycle, characterized in that, Including: Read the fetal heart rate video frame by frame, and determine each current frame and several frames before the current frame as a video segment; Using the trained three-dimensional convolutional neural network model, extract the temporal features of each video segment, and perform state prediction processing on the last frame of each video segment based on the temporal features to obtain the prediction results of the phase states of each frame in the fetal heart rate video; the phase state is the systolic phase or the diastolic phase; Combined with the cardiac motion change characteristics, correct the prediction results to obtain the corrected results; Using the corrected results, obtain the cardiac cycle of the fetal heart rate video.
2. The method according to claim 1, characterized in that, The cardiac motion change characteristics include continuity and periodicity. The combining the cardiac motion change characteristics and correcting the prediction results to obtain the corrected results includes: Combined with cardiac continuity, correct the prediction results to obtain the corrected results; And / or Combined with the duration of the cardiac cycle, correct the prediction results to obtain the corrected results.
3. The method according to claim 2, characterized in that, The combining the cardiac continuity and correcting the prediction results to obtain the corrected results includes: Use a sliding window to read the prediction results, move the sliding window frame by frame, and each time the sliding window is moved, obtain the phase states of each frame within the current sliding window, and form a phase state sequence with the phase states of each frame; different numerical values are used in the phase state sequence to represent the diastolic phase and the systolic phase; Use a preset low-pass filter kernel to perform convolution calculation on each phase state sequence and then take the average, and determine the smoothed phase state corresponding to the middle frame as the average value; the middle frame is the frame at the middle position of each frame corresponding to the phase state sequence; Based on each of the smoothed phase states, obtain a filtering result; Extract the end frame of the systolic phase and the end frame of the diastolic phase from the filtering result; Correct the phase states in the prediction results that do not match the end frame of the systolic phase and the end frame of the diastolic phase to obtain the corrected results.
4. The method according to claim 3, characterized in that, The preset low-pass filter kernel is a sign function, and the length of the sign function is 2k, where the signs from 1 to k are -1, and the signs from k + 1 to 2k are 1, and k is less than the number of frames corresponding to half of the cardiac cycle.
5. The method according to claim 2, characterized in that, The combining the duration of the cardiac cycle and correcting the prediction results to obtain the corrected results includes: Based on the prediction results, calculate the systolic duration of each systolic phase and the diastolic duration of each diastolic phase; Calculate the average value of the systolic durations and determine the systolic duration variance based on the calculated average value, calculate the average value of the diastolic durations and determine the diastolic duration variance based on the calculated average value; Based on the systolic duration variance and the diastolic duration variance, remove the outliers from the prediction results; After removing the outliers, determine a continuous prediction sequence from the prediction results; Use the continuous prediction sequence to determine the cardiac cycle duration, and based on the cardiac cycle duration, correct the prediction results after removing the outliers to obtain the corrected results.
6. The method according to claim 5, characterized in that, Use the continuous prediction sequence to determine the cardiac cycle duration, and based on the cardiac cycle duration, correct the prediction results after removing the outliers to obtain the corrected results, including: Calculate the target average systolic duration and the target average diastolic duration of the continuous prediction sequence; Fitting the time points of the end-diastolic frame and the end-systolic frame in the fetal heart rate video by using the target average systolic duration and the target average diastolic duration; Correcting the phase states in the prediction result after outlier removal that do not match the time points by using the time points to obtain the correction result.
7. The method according to any one of claims 1 to 6, characterized in that, After obtaining the cardiac cycle of the fetal heart rate video by using the correction result, it further includes: Dividing the intermediate phase of the fetal heart rate video based on the cardiac cycle; And / or Calculating the average duration of the cardiac cycle by using the correction result; Calculating the number of cycles within one minute by using the average duration and determining the number of cycles as the estimated fetal heart rate value.
8. A device for determining a cardiac cycle, characterized in that, It includes: A video segmentation module for sequentially reading the fetal heart rate video and determining each current frame and several frames before the current frame as a video segment; A model prediction module for using a trained three-dimensional convolutional neural network model to extract the temporal features of each video segment and performing state prediction processing on the last frame of each video segment based on the temporal features to obtain the prediction result of the phase state of each frame in the fetal heart rate video; the phase state is the systolic phase or the diastolic phase; A prediction correction module for correcting the prediction result by combining the cardiac change characteristics to obtain the correction result; A fetal heart rate analysis module for obtaining the cardiac cycle of the fetal heart rate video by using the correction result.
9. An electronic device, characterized in that, It includes: A memory for storing computer programs; A processor for implementing the steps of the cardiac cycle determination method according to any one of claims 1 to 7 when executing the computer program.
10. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, and when the computer program is executed by the processor, the steps of the cardiac cycle determination method according to any one of claims 1 to 7 are implemented.