Method, device and equipment for determining cardiac cycle time phase and medium

Abstract

The invention discloses a cardiac cycle time phase determination method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: determining a target mask of each heart chamber in a to-be-processed cardiac ultrasound image; determining a ventricular area and an atrium area in the to-be-processed cardiac ultrasound image by using contour information of each heart chamber determined based on the target mask; and determining a cardiac cycle time phase corresponding to the cardiac ultrasound image to be processed based on a change value corresponding to an area difference between the ventricular area and the atrial area. The corresponding contour information is determined by using the target mask of each heart chamber in the to-be-processed cardiac ultrasound image, and then the area difference between the ventricular area and the atrial area is calculated according to the contour information, so that the cardiac cycle time phase of the to-be-processed cardiac ultrasound image is determined based on the change value corresponding to the area difference. According to the scheme, the cardiac cycle time phase of the to-be-processed cardiac ultrasound image can be more intelligently determined in real time.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and medium for determining a cardiac cycle phase. Background Art

[0002] Currently, when determining the cardiac cycle phase of a cardiac ultrasound image, a doctor can turn on the PW (Pulsed-wave Doppler) mode and then manually obtain the cardiac cycle. In this process, the doctor needs to participate in the entire process, which requires a lot of manpower costs.

[0003] In addition, in the related art, in order to obtain cardiac cycle images, it is necessary to use the ultrasound video stored in the ultrasound equipment, which cannot be achieved in real time.

[0004] In summary, how to determine the cardiac cycle phase of the cardiac ultrasound image to be processed in real time and more intelligently is a problem that needs to be solved at present. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a cardiac cycle phase determination method, device, equipment and medium, which can determine the cardiac cycle phase of the cardiac ultrasound image to be processed in real time and more intelligently. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a method for determining a cardiac cycle phase, comprising:

[0007] determining a target mask of each heart chamber in the cardiac ultrasound image to be processed;

[0008] Determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask;

[0009] The cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on a change value corresponding to the area difference between the ventricular area and the atrial area.

[0010] Optionally, determining a target mask of each cardiac chamber in the cardiac ultrasound image to be processed includes:

[0011] The chamber structure segmentation model pre-built based on a preset convolutional neural network is used to process the cardiac ultrasound image to obtain the target mask of each cardiac chamber.

[0012] Optionally, the preset convolutional neural network includes a CNN backbone network and a feature decoder;

[0013] Accordingly, the chamber structure segmentation model constructed in advance based on a preset convolutional neural network is used to process the cardiac ultrasound image to be processed to obtain a target mask of each cardiac chamber, including:

[0014] Utilizing the CNN backbone network in the chamber structure segmentation model to extract high-level semantic features in the cardiac ultrasound image to be processed;

[0015] The high-level semantic features are decoded using the feature decoder to obtain a target mask for each cardiac chamber.

[0016] Optionally, the step of extracting high-level semantic features in the cardiac ultrasound image to be processed by using the CNN backbone network in the chamber structure segmentation model includes:

[0017] The CNN backbone network in the chamber structure segmentation model is used to extract the underlying semantic features in the cardiac ultrasound image to be processed; the underlying semantic features include edge feature information, contour feature information and grayscale feature information;

[0018] The underlying semantic features are subjected to feature fusion and encoding processing to obtain high-level semantic features.

[0019] Optionally, the feature decoder includes a first feature decoder and a second feature decoder;

[0020] Accordingly, the step of decoding the high-level semantic features using the feature decoder to obtain a target mask of each cardiac chamber includes:

[0021] Decoding the high-level semantic features using the first feature decoder to obtain location information and index information of each cardiac chamber;

[0022] Decoding the high-level semantic features using the second feature decoder to obtain a three-dimensional mask of each cardiac chamber;

[0023] A two-dimensional mask corresponding to the index information is selected from the three-dimensional mask, and the two-dimensional mask is modified based on the position information to obtain a target mask of each heart chamber.

[0024] Optionally, the position information is a minimum circumscribed rectangular frame corresponding to each of the cardiac chambers;

[0025] Accordingly, the two-dimensional mask is modified based on the position information to obtain a target mask of each cardiac chamber, including:

[0026] The two-dimensional mask is cropped using the minimum circumscribed rectangular frame, so that the mask in the two-dimensional mask located inside the minimum circumscribed rectangular frame is used as the target mask.

[0027] Optionally, the training set corresponding to the chamber structure segmentation model includes training images obtained after enhancing historical cardiac ultrasound images and corresponding training labels; the training labels include the boundary range of each cardiac chamber structure in the training images.

[0028] Optionally, before determining the ventricular area and the atrial area in the cardiac ultrasound image to be processed, the method further includes:

[0029] Using a computer vision algorithm to convert the target mask into initial contour information of each of the heart chambers;

[0030] The initial contour information is post-processed using a preset post-processing algorithm to obtain contour information of each of the heart chambers.

[0031] Optionally, the preset post-processing algorithm includes any one or more of an image denoising algorithm, an image smoothing algorithm and an image edge cutting algorithm.

[0032] Optionally, determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area includes:

[0033] Recording the area difference between the ventricular area and the atrial area in a preset area difference change graph; the preset area difference change graph is used to record the area difference between the ventricular area and the atrial area in different consecutive cardiac ultrasound images;

[0034] Determine the gradient change rate of the area difference corresponding to the cardiac ultrasound image to be processed in the current preset area difference change graph;

[0035] The cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on the gradient change rate.

[0036] Optionally, determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the gradient change rate includes:

[0037] If the gradient change rate is a positive value, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the diastolic phase;

[0038] If the gradient change rate is a negative value, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic phase;

[0039] If the gradient change rate is zero, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined using the frame number of the cardiac ultrasound image to be processed and the cardiac cycle length determined based on the preset area difference change map.

[0040] Optionally, determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed by using the frame number of the cardiac ultrasound image to be processed and the cardiac cycle length determined based on the preset area difference change graph includes:

[0041] Performing cycle fitting on the preset area difference variation graph to determine the cardiac cycle length;

[0042] Acquire a first target frame number corresponding to a previous extreme value point adjacent to a position point corresponding to the cardiac ultrasound image to be processed in the preset area difference change graph;

[0043] Predicting a second target frame number corresponding to a next extreme point adjacent to the previous extreme point in the preset area difference variation graph based on the cardiac cycle length and the first target frame number;

[0044] Determine the position type of the position point corresponding to the cardiac ultrasound image to be processed by using the comparison result between the frame number of the cardiac ultrasound image to be processed and the second target frame number and the type of the previous extreme point;

[0045] The cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on the position type.

[0046] Optionally, the determining the position type of the position point corresponding to the cardiac ultrasound image to be processed by using the comparison result between the frame number of the cardiac ultrasound image to be processed and the second target frame number and the type of the previous extreme point includes:

[0047] If the difference between the frame number of the cardiac ultrasound image to be processed and the second target frame number is within a preset difference range, the position type of the position point corresponding to the cardiac ultrasound image to be processed is determined according to the type of the previous extreme point; wherein, if the type of the previous extreme point is a maximum value, the position type is a trough position; if the type of the previous extreme point is a minimum value, the position type is a peak position.

[0048] Optionally, determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the position type includes:

[0049] If the position type is a peak position, determining that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is a diastolic phase;

[0050] If the position type is a trough position, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic phase.

[0051] Optionally, determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area includes:

[0052] Determine a difference between the area difference between the ventricle area and the atrial area and an area difference of a previous frame; the area difference of the previous frame is an area difference between the ventricle area and the atrial area in a previous frame of processed cardiac ultrasound image relative to the cardiac ultrasound image to be processed;

[0053] The cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on the difference.

[0054] Optionally, determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the difference includes:

[0055] If the difference is greater than zero, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the diastolic phase;

[0056] If the difference is less than zero, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic phase;

[0057] If the difference is equal to zero, the cardiac cycle phase of the last frame of processed cardiac ultrasound image is used as the cardiac cycle phase of the to-be-processed cardiac ultrasound image.

[0058] In a second aspect, the present application discloses a cardiac cycle phase determination device, comprising:

[0059] A mask determination module, used for determining a target mask of each cardiac chamber in the cardiac ultrasound image to be processed;

[0060] an area determination module, configured to determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask;

[0061] The cycle phase determination module is used to determine the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area.

[0062] In a third aspect, the present application discloses an electronic device, including:

[0063] Memory, used to store computer programs;

[0064] The processor is used to execute the computer program to implement the steps of the cardiac cycle phase determination method disclosed above.

[0065] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned method for determining the cardiac cycle phase are implemented.

[0066] It can be seen that the present application determines the target mask of each cardiac chamber in the cardiac ultrasound image to be processed; determines the ventricular area and the atrial area in the cardiac ultrasound image to be processed using the contour information of each cardiac chamber determined based on the target mask; and determines the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area. It can be seen that the present application first needs to determine the target mask of each cardiac chamber in the cardiac ultrasound image to be processed, and use the target mask to determine the contour information of each cardiac chamber, further calculate the ventricular area and the atrial area according to the contour information, and finally determine the cardiac cycle phase of the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area. Among them, the above-mentioned cardiac ultrasound image to be processed can be the current frame cardiac ultrasound image acquired in real time, or it can be a historical cardiac ultrasound image. If it is the current frame cardiac ultrasound image, it can achieve real-time determination of the cardiac cycle phase of the current frame cardiac ultrasound image. In addition, the present application determines the cardiac cycle phase of the cardiac ultrasound image to be processed by automatically calculating the ventricular-atrial area difference. The entire process does not require the involvement of a doctor, making it more intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0068] Figure 1 A flow chart of a cardiac cycle phase determination method disclosed in the present application;

[0069] Figure 2 A flowchart of a specific cardiac cycle phase determination method disclosed in the present application;

[0070] Figure 3 A schematic diagram of a network structure of a chamber structure segmentation model disclosed in this application;

[0071] Figure 4 A workflow diagram of a chamber structure segmentation model disclosed in this application;

[0072] Figure 5 This is a flow chart of the method for determining cardiac cycle phase based on gradient change rate disclosed in this application;

[0073] Figure 6 This is a flow chart of the method for dividing cardiac cycle phases according to gradient change rate disclosed in the present application;

[0074] Figure 7 This is a flow chart of a method for determining a cardiac cycle phase based on an area difference between two adjacent frames disclosed in the present application;

[0075] Figure 8 This is a schematic diagram of the structure of a cardiac cycle phase determination device disclosed in the present application;

[0076] Fig. 9 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0078] Currently, when determining the cardiac cycle phase of a cardiac ultrasound image, doctors need to participate in the entire cardiac cycle phase determination process, which requires a lot of manpower costs. In addition, in related technologies, in order to obtain cardiac cycle images, it is necessary to use ultrasound videos retained in ultrasound equipment, which cannot be achieved in real time. To this end, the embodiments of the present application disclose a cardiac cycle phase determination method, device, equipment and medium, which can determine the cardiac cycle phase of a cardiac ultrasound image to be processed in real time and more intelligently.

[0079] See also Figure 1 As shown, the embodiment of the present application discloses a method for determining a cardiac cycle phase, the method comprising:

[0080] Step S11: determining a target mask of each heart chamber in the cardiac ultrasound image to be processed.

[0081] In this embodiment, a cardiac ultrasound image to be processed is obtained, and a target mask of each cardiac chamber in the cardiac ultrasound image to be processed is determined. The cardiac ultrasound image to be processed may be a current frame cardiac ultrasound image acquired in real time, or a historical cardiac ultrasound image retained in an ultrasound device, and this embodiment does not limit this. The cardiac ultrasound image to be processed may be a single image or an image set consisting of multiple images. In the case where the cardiac ultrasound image to be processed is an image set, each cardiac ultrasound image to be processed may be processed separately.

[0082] Furthermore, the image format of the cardiac ultrasound image to be processed may be specifically an RGB format. The cardiac ultrasound image to be processed may also be a grayscale image output by an ultrasound device.

[0083] The cardiac ultrasound image to be processed may be a cardiac ultrasound image containing preset chambers, for example, may be an ultrasound image containing four complete chambers.

[0084] Step S12: Determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask.

[0085] In this embodiment, the target mask is used to determine the contour information of each heart chamber, and the ventricular area and the atrial area are further calculated based on the contour information; the ventricular area here includes the left ventricular area and the right ventricular area, which can be the sum of the left ventricular area and the right ventricular area; the atrial area includes the left atrium area and the right atrium area, which can be the sum of the left atrium area and the right atrium area.

[0086] It should be noted that before determining the ventricular area and atrial area in the cardiac ultrasound image to be processed, the method further includes: using a computer vision algorithm to convert the target mask into the initial contour information of each of the cardiac chambers; using a preset post-processing algorithm to post-process the initial contour information to obtain the contour information of each of the cardiac chambers. That is, after obtaining the target mask, this embodiment converts the target mask into the initial contour information of each cardiac chamber by using a traditional computer vision algorithm (Computer Vision, i.e., CV algorithm), and further, it is necessary to post-process the initial contour information by using a preset post-processing algorithm to obtain the contour information of each cardiac chamber. It is understandable that the initial contour directly converted from the target mask may have contour edge noise, rough edges, etc., so it is necessary to perform post-processing operations on it by using a preset post-processing algorithm to obtain the contour information of each cardiac chamber.

[0087] The above-mentioned preset post-processing algorithms include but are not limited to image denoising algorithms, image smoothing algorithms, and image edge cutting algorithms, etc. It should be noted that when using the image edge cutting algorithm for edge cutting, the initial contour information can be optimized in combination with the original image information so that the optimized contour information does not overflow the actual boundary range.

[0088] Step S13: determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area.

[0089] In this embodiment, when calculating the area difference between the ventricular area and the atrial area, the sum of the left ventricular area and the right ventricular area is subtracted from the sum of the left atrial area and the right atrial area. It can be understood that the ventricular area is usually larger than the atrial area. Therefore, in this embodiment, the ventricular area is subtracted from the atrial area, so that the obtained area difference is a positive value.

[0090] Further, after obtaining the area difference between the ventricular area and the atrial area, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined using the change value corresponding to the area difference. In this embodiment, it is mainly to determine whether the cardiac ultrasound image to be processed is in the diastolic phase or the systolic phase of the cardiac cycle; wherein, when in the diastolic phase, the ventricular area continues to increase and the atrial area continues to decrease; when in the systolic phase, the ventricular area continues to decrease and the atrial area continues to increase.

[0091] It can be seen that the present application determines the target mask of each cardiac chamber in the cardiac ultrasound image to be processed; determines the ventricular area and the atrial area in the cardiac ultrasound image to be processed using the contour information of each cardiac chamber determined based on the target mask; and determines the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area. It can be seen that the present application first needs to determine the target mask of each cardiac chamber in the cardiac ultrasound image to be processed, and use the target mask to determine the contour information of each cardiac chamber, further calculate the ventricular area and the atrial area according to the contour information, and finally determine the cardiac cycle phase of the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area. Among them, the above-mentioned cardiac ultrasound image to be processed can be the current frame cardiac ultrasound image acquired in real time, or it can be a historical cardiac ultrasound image. If it is the current frame cardiac ultrasound image, it can achieve real-time determination of the cardiac cycle phase of the current frame cardiac ultrasound image. In addition, the present application determines the cardiac cycle phase of the cardiac ultrasound image to be processed by automatically calculating the ventricular-atrial area difference. The entire process does not require the involvement of a doctor, making it more intelligent.

[0092] See also Figure 2 As shown, the embodiment of the present application discloses a specific method for determining the phase of the cardiac cycle. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0093] Step S21: Process the cardiac ultrasound image to be processed using a chamber structure segmentation model pre-constructed based on a preset convolutional neural network to obtain a target mask of each cardiac chamber.

[0094] In this embodiment, a chamber structure segmentation model for cardiac chamber structure segmentation is constructed in advance based on a preset convolutional neural network. Therefore, the chamber structure segmentation model is used to process the cardiac ultrasound image to be processed to obtain a target mask of each cardiac chamber.

[0095] Specifically, the training set corresponding to the above-mentioned chamber structure segmentation model includes training images obtained after enhancing the historical cardiac ultrasound images and corresponding training labels; the training labels include the boundary range of each cardiac chamber structure in the training images. That is, when training the model, a large number of training images with boundary range annotations of each cardiac chamber structure will be used for training. The training images can specifically be historical cardiac ultrasound images, and different types of enhancement methods will be used to enhance the historical cardiac ultrasound images. The enhancement methods include but are not limited to horizontal flipping, vertical flipping, mirror filling rotation, mosaic enhancement, affine transformation, random contrast enhancement, random brightness enhancement, image cropping, etc. In addition, in order to make the trained chamber structure segmentation model have stronger learning ability, generalization ability, etc., the model can be trained for different audiences. For example, if the fetal heart phase is to be judged, the training data can be fetal heart ultrasound images covering different gestational ages and different regions.

[0096] Further, the network structure of the chamber structure segmentation model can be found in Figure 3 As shown in Figure 3 It can be seen that the above preset convolutional neural network specifically includes a CNN backbone network and a feature decoder. Figure 4 As shown, step S21 may specifically include:

[0097] Step S211: Utilize the CNN backbone network in the chamber structure segmentation model to extract high-level semantic features in the cardiac ultrasound image to be processed.

[0098] In this embodiment, the preset convolutional neural network specifically includes a CNN (Convolutional Neural Networks) backbone network and a feature decoder, wherein the CNN backbone network is mainly used for image feature extraction. Therefore, when a cardiac ultrasound image to be processed is obtained and the cardiac ultrasound image to be processed is processed using the chamber structure segmentation model, the CNN backbone network in the chamber structure segmentation model is first used to extract high-level semantic features in the cardiac ultrasound image to be processed.

[0099] In a specific embodiment, the above-mentioned use of the CNN backbone network in the chamber structure segmentation model to extract high-level semantic features in the cardiac ultrasound image to be processed includes: using the CNN backbone network in the chamber structure segmentation model to extract the underlying semantic features in the cardiac ultrasound image to be processed; the underlying semantic features include edge feature information, contour feature information and grayscale feature information; and performing feature fusion and encoding processing on the underlying semantic features to obtain high-level semantic features. That is, the CNN backbone network first extracts the underlying semantic features in the cardiac ultrasound image to be processed, the underlying semantic features include but are not limited to edge feature information, contour feature information and grayscale feature information, and then performs feature fusion and encoding processing on these underlying semantic features in the high-level convolutional network to generate high-level semantic features.

[0100] Step S212: Decode the high-level semantic features using the feature decoder to obtain a target mask for each cardiac chamber.

[0101] In this embodiment, after the high-level semantic features of the cardiac ultrasound image to be processed are extracted using the CNN backbone network, the high-level semantic features are decoded using a feature decoder to obtain a target mask of each cardiac chamber.

[0102] In a specific embodiment, Figure 3As shown, the feature decoder includes a first feature decoder and a second feature decoder; accordingly, the use of the feature decoder to decode the high-level semantic features to obtain the target mask of each heart chamber includes: using the first feature decoder to decode the high-level semantic features to obtain the position information and index information of each heart chamber; using the second feature decoder to decode the high-level semantic features to obtain the three-dimensional mask of each heart chamber; selecting the two-dimensional mask corresponding to the index information from the three-dimensional mask, and correcting the two-dimensional mask based on the position information to obtain the target mask of each heart chamber. That is, the feature decoder for decoding high-level semantic features includes a first feature decoder and a second feature decoder, wherein the first feature decoder is used to decode the high-level semantic features to obtain the position information and index information of each heart chamber, and the second feature decoder is used to decode the high-level semantic features to obtain the three-dimensional mask of each heart chamber. It should be pointed out that the three-dimensional mask here is formed by stacking several layers of channels, each layer of channels can be understood as two-dimensional mask information, and among several layers of channels, only one channel has two-dimensional mask information that is valid, and the index information output by the first feature decoder can accurately locate the channel from several layers of channels; wherein, the index information refers to the index value of the channel dimension where the two-dimensional mask is located in the three-dimensional mask output by the network. Therefore, in this embodiment, information transmission is performed between the first feature decoder and the second feature decoder, that is, the index information output by the first feature decoder is used to filter out the effective two-dimensional mask from the three-dimensional mask output by the second feature decoder, and then the position information output by the first feature decoder is used to correct the two-dimensional mask to obtain the target mask of each heart chamber. That is to say, in this embodiment, one decoder outputs index information, and the other decoder outputs a three-dimensional mask. By combining the two decoders, it is possible to accurately and quickly locate the effective two-dimensional mask from the three-dimensional mask based on the index information, and filter out the remaining invalid two-dimensional mask information, thereby improving the accuracy of the target mask and reducing the amount of data calculation.

[0103] For example, assuming that the index information output by the first decoder is 8, the three-dimensional mask output by the second decoder is composed of 32 channels stacked together, and the index values ​​of these 32 channels are marked from 1 to 32 respectively; therefore, a two-dimensional mask with an index value of 8 is filtered out from the three-dimensional mask, and then the two-dimensional mask is corrected using the position information output by the first feature decoder to obtain the target mask of each heart chamber.

[0104] In a specific embodiment, the above-mentioned position information is the minimum circumscribed rectangular frame corresponding to each of the cardiac chambers; accordingly, the correction of the two-dimensional mask based on the position information to obtain the target mask of each cardiac chamber includes: using the minimum circumscribed rectangular frame to crop the two-dimensional mask so as to use the mask located inside the minimum circumscribed rectangular frame in the two-dimensional mask as the target mask. It can be understood that the expression form of the position information can be to generate a corresponding minimum circumscribed rectangular frame according to the actual structure of each cardiac chamber on the image, so as to determine the approximate position and boundary of each cardiac chamber, and then use the minimum circumscribed rectangular frame to crop the two-dimensional mask so as to use the mask located inside the minimum circumscribed rectangular frame in the two-dimensional mask as the target mask. That is, this embodiment removes the mask that exceeds the range of the minimum circumscribed rectangular frame, thereby reducing errors, so that the target mask obtained after the correction process is more in line with the actual structure of the cardiac chamber, and facilitates the subsequent measurement of the cardiac chamber area to be more accurate.

[0105] Step S22: Determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask.

[0106] Step S23: determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area.

[0107] For more specific processing procedures of the above steps S22 and S23, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0108] It can be seen that in the embodiment of the present application, a chamber structure segmentation model for cardiac chamber structure segmentation is pre-constructed based on a preset convolutional neural network, and the chamber structure segmentation model is used to process the cardiac ultrasound image to be processed to obtain the target mask of each cardiac chamber. In addition, the preset convolutional neural network specifically includes a CNN backbone network and a feature decoder, wherein the CNN backbone network is used to extract the underlying semantic features in the cardiac ultrasound image to be processed, and these underlying semantic features are feature fused and encoded in the high-level convolutional network to generate high-level semantic features; the feature decoder includes a first feature decoder and a second feature decoder, the first feature decoder is used to decode the high-level semantic features to obtain the position information and index information of each cardiac chamber, the position information is specifically the minimum circumscribed rectangular frame corresponding to each cardiac chamber, the second feature decoder is used to decode the high-level semantic features to obtain the three-dimensional mask of each cardiac chamber, and then use the index information to filter out the corresponding two-dimensional mask from the three-dimensional mask, and then remove the two-dimensional mask that exceeds the minimum circumscribed rectangular frame range, thereby reducing the error to obtain the target mask of each cardiac chamber. That is, the embodiment of the present application provides a cardiac cycle phase determination method based on the assistance of a CNN network model.

[0109] See also Figure 5 As shown, the embodiment of the present application discloses a specific method for determining the phase of the cardiac cycle. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0110] Step S31: determining a target mask of each cardiac chamber in the cardiac ultrasound image to be processed.

[0111] Step S32: Determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask.

[0112] Step S33: recording the area difference between the ventricular area and the atrial area in a preset area difference change graph; the preset area difference change graph is used to record the area difference between the ventricular area and the atrial area in different consecutive cardiac ultrasound images.

[0113] In this embodiment, after the area difference between the ventricular area and the atrial area is obtained, the area difference is recorded in a preset area difference change graph, wherein the preset area difference change graph is used to record the area difference between the ventricular area and the atrial area in different consecutive cardiac ultrasound images. It is understandable that change graphs can be drawn for the ventricular area and the atrial area of ​​different consecutive cardiac ultrasound images, respectively, and then the difference between the two is made to obtain the preset area difference change graph composed of the area difference between the ventricular area and the atrial area.

[0114] Step S34: determining the gradient change rate of the area difference corresponding to the cardiac ultrasound image to be processed in the current preset area difference change graph.

[0115] In this embodiment, the area difference is recorded in a preset area difference change map, and the gradient change rate of the area difference corresponding to the cardiac ultrasound image to be processed in the current preset area difference change map is determined, that is, the gradient change rate of the area difference in the current preset area difference change map is determined.

[0116] Step S35: determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the gradient change rate.

[0117] In this embodiment, the specific steps of determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed according to the specific value of the gradient change rate can be found in Figure 6 As shown in , step S35 may specifically include:

[0118] Step S351: If the gradient change rate is a positive value, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the diastolic phase.

[0119] That is, if the gradient change rate is a positive value, it means that the area difference between the ventricular area and the atrial area is increasing, that is, the ventricular area continues to increase and the atrial area continues to decrease, so that the area difference between the two continues to increase. Therefore, when the gradient change rate is a positive value, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the diastolic phase.

[0120] Step S352: If the gradient change rate is a negative value, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic phase.

[0121] In this embodiment, if the gradient change rate is a negative value, it means that the area difference between the ventricular area and the atrial area is in a decreasing state, that is, at this time, the ventricular area continues to decrease, and the atrial area continues to increase, so that the area difference between the two continues to decrease. Therefore, when the gradient change rate is a negative value, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic period.

[0122] Step S353: If the gradient change rate is zero, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined using the frame number of the cardiac ultrasound image to be processed and the cardiac cycle length determined based on the preset area difference change diagram.

[0123] In this embodiment, if the gradient change rate is zero, it means that the current phase is at the end of a cardiac cycle phase. Therefore, when the gradient change rate is zero, there are two situations: 1. The current ventricular area is the largest and the atrial area is the smallest, which is the end of the diastolic period; 2. The current ventricular area is the smallest and the atrial area is the largest, which is the end of the systolic period. Therefore, it is impossible to directly determine that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic period at this time, and it is necessary to further use the frame number of the cardiac ultrasound image to be processed and the cardiac cycle length determined based on the preset area difference change diagram to determine the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed.

[0124] In a specific embodiment, the above-mentioned determination of the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed by using the frame number of the cardiac ultrasound image to be processed and the cardiac cycle length determined based on the preset area difference change graph includes: performing period fitting on the preset area difference change graph to determine the cardiac cycle length; obtaining a first target frame number corresponding to the previous extreme point adjacent to the position point corresponding to the cardiac ultrasound image to be processed in the preset area difference change graph; predicting a second target frame number corresponding to the next extreme point adjacent to the previous extreme point in the preset area difference change graph based on the cardiac cycle length and the first target frame number; determining the position type of the position point corresponding to the cardiac ultrasound image to be processed by using the comparison result between the frame number of the cardiac ultrasound image to be processed and the second target frame number and the type of the previous extreme point; and determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the position type.

[0125] That is, firstly, the preset area difference change graph is subjected to cycle fitting to determine the length of the cardiac cycle. It should be noted that in order to ensure a better cycle fitting effect and a more accurate cardiac cycle length, the preset area difference change graph needs to include at least two cardiac cycles. Further, the first target frame number corresponding to the previous extreme point adjacent to the position point corresponding to the cardiac ultrasound image to be processed in the preset area difference change graph is obtained. The previous extreme point here corresponds to the end of the systolic period or the end of the diastolic period in the cardiac cycle. Then, the cardiac cycle length and the first target frame number can be used to predict the second target frame number corresponding to the next extreme point adjacent to the previous extreme point in the preset area difference change graph. The comparison result is obtained by comparing the frame number of the cardiac ultrasound image to be processed with the second target frame number. Finally, the position type of the position point corresponding to the cardiac ultrasound image to be processed is determined in combination with the comparison result and the type of the previous extreme point.

[0126] Furthermore, the above-mentioned method of determining the position type of the position point corresponding to the cardiac ultrasound image to be processed by using the comparison result between the frame number of the cardiac ultrasound image to be processed and the second target frame number and the type of the previous extreme point includes: if the difference between the frame number of the cardiac ultrasound image to be processed and the second target frame number is within a preset difference range, then determining the position type of the position point corresponding to the cardiac ultrasound image to be processed according to the type of the previous extreme point; wherein, if the type of the previous extreme point is a maximum value, then the position type is a trough position; if the type of the previous extreme point is a minimum value, then the position type is a peak position. It can be understood that when comparing the frame number of the cardiac ultrasound image to be processed with the second target frame number, it is mainly to determine whether the difference between the two is within the preset difference range. If the difference is within the preset difference range, the position type of the position point corresponding to the cardiac ultrasound image to be processed is determined according to the type of the previous extreme point. If the type of the previous extreme point is a maximum value, the position type of the position point corresponding to the cardiac ultrasound image to be processed is a trough position; if the type of the previous extreme point is a minimum value, the position type of the position point corresponding to the cardiac ultrasound image to be processed is a peak position. In this way, the above-mentioned determination of the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the position type includes: if the position type is a peak position, determining that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is a diastole; if the position type is a trough position, determining that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is a systole. That is to say, if the position type of the position point corresponding to the cardiac ultrasound image to be processed is a peak position, then the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the diastole period; if the position type of the position point corresponding to the cardiac ultrasound image to be processed is a trough position, then the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the systole period.

[0127] For example, assuming that the previous extreme point is a maximum value, the corresponding first target frame number is 100 frames, which means that the cardiac ultrasound image corresponding to the previous extreme point is at the end of the diastole period, and assuming that the cardiac cycle length is 30 frames. Then, based on the first target frame number and the cardiac cycle length, it can be predicted that the second target frame number corresponding to the next extreme point adjacent to the previous extreme point is 115 frames, which is a minimum value at this time, that is, the cardiac ultrasound image should be at the end of the systole period after half a cycle. The frame number of the cardiac ultrasound image to be processed is then compared with the second target frame number to determine whether the difference between the two is within the preset difference range, for example, within 2 frames. If the frame number of the cardiac ultrasound image to be processed is 114, then the difference between the two is 1, which is within the preset difference range. Therefore, it can be determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systole period.

[0128] In addition, in a specific embodiment, the above-mentioned last extreme point may only refer to the last maximum value or the last minimum value, and then only the frame number of the maximum value or the minimum value needs to be recorded each time. For example, taking the last maximum value as an example, the second target frame number corresponding to the next maximum value can be predicted based on the first target frame number of the last maximum value plus the length of the cardiac cycle. If the frame number of the cardiac ultrasound image to be processed is close to the second target frame number, and whether the difference between the two is within the preset difference range, then the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the diastolic period; if the frame number of the cardiac ultrasound image to be processed differs from the second target frame number by about half a cardiac cycle, then the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the systolic period. In addition, in the above process, the second target frame number corresponding to the next minimum value can also be predicted based on the first target frame number plus half a cardiac cycle length. The subsequent judgment steps are similar to the above content, so they are not repeated here.

[0129] For more specific processing procedures of the above steps S31 and S32, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0130] It can be seen that after obtaining the area difference between the ventricular area and the atrial area, the embodiment of the present application records the area difference in a preset area difference change map, wherein the preset area difference change map is used to record the area difference between the ventricular area and the atrial area in different consecutive cardiac ultrasound images, and determine the gradient change rate of the area difference corresponding to the cardiac ultrasound image to be processed in the current preset area difference change map. In the case where the gradient change rate is a positive value, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the diastole period, and in the case where the gradient change rate is a negative value, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the systole period, and in the case where the gradient change rate is zero, the frame number of the cardiac ultrasound image to be processed and the cardiac cycle length determined based on the preset area difference change map are used to determine the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed. That is, the embodiment of the present application provides a method for determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed according to the gradient change rate of the area difference in the preset area difference change map.

[0131] See also Figure 7 As shown, the embodiment of the present application discloses a specific method for determining the phase of the cardiac cycle. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0132] Step S41: determining a target mask of each cardiac chamber in the cardiac ultrasound image to be processed.

[0133] Step S42: Determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask.

[0134] Step S43: Determine the difference between the area difference between the ventricular area and the atrial area and the area difference of the previous frame; the area difference of the previous frame is the area difference between the ventricular area and the atrial area in the previous frame of processed cardiac ultrasound image relative to the cardiac ultrasound image to be processed.

[0135] In this embodiment, after obtaining the area difference between the ventricular area and the atrial area, the difference between the area difference and the area difference of the previous frame is determined. The area difference of the previous frame refers to the area difference between the ventricular area and the atrial area in the previous frame of processed cardiac ultrasound image relative to the cardiac ultrasound image to be processed.

[0136] Step S44: determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the difference.

[0137] In this embodiment, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on the difference between the area difference of the current frame and the area difference of the previous frame. Further, the above-mentioned determination of the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the difference includes: if the difference is greater than zero, then the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the diastole; if the difference is less than zero, then the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the systole; if the difference is equal to zero, then the cardiac cycle phase of the previous frame of processed cardiac ultrasound image is used as the cardiac cycle phase of the cardiac ultrasound image to be processed.

[0138] That is, in the first specific embodiment, if the difference between the area difference of the current frame and the area difference of the previous frame is greater than zero, it means that the area difference between the ventricular area and the atrial area of ​​the current frame is in an increasing state compared with the previous frame, that is, the ventricular area continues to increase and the atrial area continues to decrease. Therefore, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the diastole.

[0139] In the second specific embodiment, if the difference between the area difference of the current frame and the area difference of the previous frame is less than zero, it means that the area difference between the ventricular area and the atrial area of ​​the current frame is decreasing compared with the previous frame, that is, the ventricular area continues to decrease and the atrial area continues to increase. Therefore, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the contraction period.

[0140] In the third specific embodiment, if the difference between the area difference of the current frame and the area difference of the previous frame is equal to zero, it means that the current frame is at the end of a certain cardiac cycle phase, and the cardiac cycle phase of the previous processed cardiac ultrasound image is used as the cardiac cycle phase of the cardiac ultrasound image to be processed. That is, if the cardiac cycle phase of the previous processed cardiac ultrasound image is diastolic, the cardiac cycle phase of the cardiac ultrasound image to be processed is also diastolic, and if the cardiac cycle phase of the previous processed cardiac ultrasound image is systolic, the cardiac cycle phase of the cardiac ultrasound image to be processed is also systolic.

[0141] For more specific processing procedures of the above steps S41 and S42, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0142] It can be seen that after obtaining the current frame area difference between the current ventricular area and the atrial area, the embodiment of the present application determines the difference between the current frame area difference and the previous frame area difference. The previous frame area difference refers to the area difference between the ventricular area and the atrial area in the previous frame of processed cardiac ultrasound image relative to the cardiac ultrasound image to be processed. If the difference between the current frame area difference and the previous frame area difference is greater than zero, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the diastole. If the difference between the current frame area difference and the previous frame area difference is less than zero, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined to be the systole. If the difference between the current frame area difference and the previous frame area difference is equal to zero, the cardiac cycle phase of the previous frame of processed cardiac ultrasound image is used as the cardiac cycle phase of the cardiac ultrasound image to be processed. That is, the embodiment of the present application provides a method for determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the difference between the current ventricular area and the atrial area and the previous frame area difference.

[0143] See also Figure 8 As shown, the embodiment of the present application discloses a cardiac cycle phase determination device, the device comprising:

[0144] A mask determination module 11 is used to determine a target mask of each cardiac chamber in the cardiac ultrasound image to be processed;

[0145] An area determination module 12, configured to determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask;

[0146] The cycle phase determination module 13 is used to determine the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area.

[0147] Since the embodiments of the device part correspond to the above embodiments, please refer to the description of the embodiments of the method part for the embodiments of the device part, and will not be repeated here.

[0148] It can be seen that the present application determines the target mask of each cardiac chamber in the cardiac ultrasound image to be processed; determines the ventricular area and the atrial area in the cardiac ultrasound image to be processed using the contour information of each cardiac chamber determined based on the target mask; and determines the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area. It can be seen that the present application first needs to determine the target mask of each cardiac chamber in the cardiac ultrasound image to be processed, and use the target mask to determine the contour information of each cardiac chamber, further calculate the ventricular area and the atrial area according to the contour information, and finally determine the cardiac cycle phase of the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area. Among them, the above-mentioned cardiac ultrasound image to be processed can be the current frame cardiac ultrasound image acquired in real time, or it can be a historical cardiac ultrasound image. If it is the current frame cardiac ultrasound image, it can achieve real-time determination of the cardiac cycle phase of the current frame cardiac ultrasound image. In addition, the present application determines the cardiac cycle phase of the cardiac ultrasound image to be processed by automatically calculating the ventricular-atrial area difference. The entire process does not require the involvement of a doctor, making it more intelligent.

[0149] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the cardiac cycle phase determination method performed by the electronic device disclosed in any of the aforementioned embodiments.

[0150] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0151] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0152] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0153] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows, Unix, Linux, etc. In addition to including a computer program that can be used to complete the cardiac cycle phase determination method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. In addition to data transmitted from an external device received by the electronic device, the data 223 can also include data collected by its own input and output interface 25, etc.

[0154] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the method for determining the cardiac cycle phase disclosed in any of the aforementioned embodiments are implemented.

[0155] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0156] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0157] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0158] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0159] The above is a detailed introduction to a cardiac cycle phase determination method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in the field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for determining a cardiac cycle phase, characterized in that: include: determining a target mask of each heart chamber in the cardiac ultrasound image to be processed; Determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask; The cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on a change value corresponding to the area difference between the ventricular area and the atrial area.

2. The method for determining cardiac cycle phase according to claim 1, characterized in that: The step of determining a target mask of each cardiac chamber in the cardiac ultrasound image to be processed comprises: The chamber structure segmentation model pre-built based on a preset convolutional neural network is used to process the cardiac ultrasound image to obtain the target mask of each cardiac chamber.

3. The method for determining cardiac cycle phase according to claim 2, characterized in that: The preset convolutional neural network includes a CNN backbone network and a feature decoder; Accordingly, the chamber structure segmentation model constructed in advance based on a preset convolutional neural network is used to process the cardiac ultrasound image to be processed to obtain a target mask of each cardiac chamber, including: Utilizing the CNN backbone network in the chamber structure segmentation model to extract high-level semantic features in the cardiac ultrasound image to be processed; The high-level semantic features are decoded using the feature decoder to obtain a target mask for each cardiac chamber.

4. The method for determining cardiac cycle phase according to claim 3, characterized in that: The method of extracting high-level semantic features in the cardiac ultrasound image to be processed by using the CNN backbone network in the chamber structure segmentation model includes: The CNN backbone network in the chamber structure segmentation model is used to extract the underlying semantic features in the cardiac ultrasound image to be processed; the underlying semantic features include edge feature information, contour feature information and grayscale feature information; The underlying semantic features are subjected to feature fusion and encoding processing to obtain high-level semantic features.

5. The method for determining cardiac cycle phase according to claim 3, characterized in that: The feature decoder includes a first feature decoder and a second feature decoder; Accordingly, the step of decoding the high-level semantic features using the feature decoder to obtain a target mask of each cardiac chamber includes: Decoding the high-level semantic features using the first feature decoder to obtain location information and index information of each cardiac chamber; Decoding the high-level semantic features using the second feature decoder to obtain a three-dimensional mask of each cardiac chamber; A two-dimensional mask corresponding to the index information is selected from the three-dimensional mask, and the two-dimensional mask is modified based on the position information to obtain a target mask of each heart chamber.

6. The method for determining cardiac cycle phase according to claim 5, characterized in that: The position information is the minimum circumscribed rectangular frame corresponding to each of the cardiac chambers; Accordingly, the two-dimensional mask is modified based on the position information to obtain a target mask of each cardiac chamber, including: The two-dimensional mask is cropped using the minimum circumscribed rectangular frame, so that the mask in the two-dimensional mask located inside the minimum circumscribed rectangular frame is used as the target mask.

7. The method for determining cardiac cycle phase according to claim 2, characterized in that: The training set corresponding to the chamber structure segmentation model includes training images obtained after enhancing historical cardiac ultrasound images and corresponding training labels; the training labels include the boundary range of each cardiac chamber structure in the training images.

8. The method for determining cardiac cycle phase according to claim 1, characterized in that: Before determining the ventricular area and the atrial area in the cardiac ultrasound image to be processed, the method further includes: Using a computer vision algorithm to convert the target mask into initial contour information of each of the heart chambers; The initial contour information is post-processed using a preset post-processing algorithm to obtain contour information of each of the heart chambers.

9. The method for determining cardiac cycle phase according to claim 8, characterized in that: The preset post-processing algorithm includes any one or several of an image denoising algorithm, an image smoothing algorithm and an image edge cutting algorithm.

10. The method for determining the cardiac cycle phase according to any one of claims 1 to 9, characterized in that: The determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area comprises: Recording the area difference between the ventricular area and the atrial area in a preset area difference change graph; the preset area difference change graph is used to record the area difference between the ventricular area and the atrial area in different consecutive cardiac ultrasound images; Determine the gradient change rate of the area difference corresponding to the cardiac ultrasound image to be processed in the current preset area difference change graph; The cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on the gradient change rate.

11. The method for determining cardiac cycle phase according to claim 10, characterized in that: The step of determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the gradient change rate includes: If the gradient change rate is a positive value, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the diastolic phase; If the gradient change rate is a negative value, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic phase; If the gradient change rate is zero, the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined using the frame number of the cardiac ultrasound image to be processed and the cardiac cycle length determined based on the preset area difference change map.

12. The method for determining cardiac cycle phase according to claim 11, characterized in that: Determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed by using the frame number of the cardiac ultrasound image to be processed and the cardiac cycle length determined based on the preset area difference change graph includes: Performing cycle fitting on the preset area difference variation graph to determine the cardiac cycle length; Acquire a first target frame number corresponding to a previous extreme value point adjacent to a position point corresponding to the cardiac ultrasound image to be processed in the preset area difference change graph; Predicting a second target frame number corresponding to a next extreme point adjacent to the previous extreme point in the preset area difference variation graph based on the cardiac cycle length and the first target frame number; Determine the position type of the position point corresponding to the cardiac ultrasound image to be processed by using the comparison result between the frame number of the cardiac ultrasound image to be processed and the second target frame number and the type of the previous extreme point; The cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on the position type.

13. The method for determining cardiac cycle phase according to claim 12, characterized in that: The step of determining the position type of the position point corresponding to the cardiac ultrasound image to be processed by using the comparison result between the frame number of the cardiac ultrasound image to be processed and the second target frame number and the type of the previous extreme point comprises: If the difference between the frame number of the cardiac ultrasound image to be processed and the second target frame number is within a preset difference range, the position type of the position point corresponding to the cardiac ultrasound image to be processed is determined according to the type of the previous extreme point; wherein, if the type of the previous extreme point is a maximum value, the position type is a trough position; if the type of the previous extreme point is a minimum value, the position type is a peak position.

14. The method for determining cardiac cycle phase according to claim 12 or 13, characterized in that: The step of determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the position type includes: If the position type is a peak position, determining that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is a diastolic phase; If the position type is a trough position, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic phase.

15. The method for determining cardiac cycle phase according to any one of claims 1 to 9, characterized in that: The determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area comprises: Determine a difference between the area difference between the ventricle area and the atrial area and an area difference of a previous frame; the area difference of the previous frame is an area difference between the ventricle area and the atrial area in a previous frame of processed cardiac ultrasound image relative to the cardiac ultrasound image to be processed; The cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is determined based on the difference.

16. The method for determining cardiac cycle phase according to claim 15, characterized in that: The step of determining the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the difference comprises: If the difference is greater than zero, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the diastolic phase; If the difference is less than zero, it is determined that the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed is the systolic phase; If the difference is equal to zero, the cardiac cycle phase of the last frame of processed cardiac ultrasound image is used as the cardiac cycle phase of the to-be-processed cardiac ultrasound image.

17. A cardiac cycle phase determination device, characterized in that: include: A mask determination module, used for determining a target mask of each cardiac chamber in the cardiac ultrasound image to be processed; an area determination module, configured to determine the ventricular area and the atrial area in the cardiac ultrasound image to be processed by using the contour information of each of the cardiac chambers determined based on the target mask; The cycle phase determination module is used to determine the cardiac cycle phase corresponding to the cardiac ultrasound image to be processed based on the change value corresponding to the area difference between the ventricular area and the atrial area.

18. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the method for determining the cardiac cycle phase according to any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the method for determining the cardiac cycle phase according to any one of claims 1 to 16 are implemented.

Images