Cardiac cycle detection method and device, electronic device, and storage medium

Abstract

The present application provides a cardiac cycle detection method and apparatus, electronic device, and storage medium, comprising: segmenting each image in a coronary angiography image sequence to obtain a segmentation result; constructing an image description feature corresponding to each image in the image sequence based on multiple segmentation results corresponding to the image sequence; constructing a feature similarity sequence based on the multiple image description features corresponding to the image sequence; and determining the corresponding phase of the cardiac cycle for each image in the image sequence based on the feature similarity sequence and the multiple segmentation results corresponding to the image sequence. The present application scheme detects the cardiac cycle based on images generated by coronary angiography.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a cardiac cycle detection method and device, electronic equipment, and computer-readable storage medium. Background Art

[0002] The cardiac cycle refers to the series of events that occur during a heartbeat, including contraction and relaxation of the cardiac chambers. The entire cardiac cycle proceeds through eight phases: isovolumetric contraction, rapid ejection phase, slowed ejection phase, prediastole, isovolumetric relaxation phase, rapid filling phase, slowed filling phase, and atrial contraction. Detection of the cardiac cycle is crucial for determining surgical timing, assessing coronary artery disease, evaluating cardiac function, and guiding surgical procedures. Therefore, a solution for detecting the cardiac cycle based on coronary angiography images is urgently needed. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a cardiac cycle detection method and device, electronic equipment, and storage medium for detecting the cardiac cycle based on images generated by coronary angiography.

[0004] In one aspect, the present application provides a method for detecting a cardiac cycle, comprising:

[0005] Segment each image in the coronary angiography image sequence to obtain a segmentation result;

[0006] constructing an image description feature corresponding to each image in the image sequence based on a plurality of segmentation results corresponding to the image sequence;

[0007] constructing a feature similarity sequence based on a plurality of image description features corresponding to the image sequence;

[0008] Based on the feature similarity sequence and a plurality of segmentation results corresponding to the image sequence, a corresponding phase of each image in the image sequence in the cardiac cycle is determined.

[0009] In one embodiment, constructing an image description feature corresponding to each image in the image sequence based on the multiple segmentation results corresponding to the image sequence includes:

[0010] determining a plurality of feature points in the image based on a segmentation result of each image in the image sequence;

[0011] determining, based on a plurality of characteristic points of each image in the image sequence, a plurality of characteristic point velocities corresponding to each image;

[0012] An image description feature is constructed based on the speeds of a plurality of feature points and the positions of a plurality of feature points corresponding to each image in the image sequence.

[0013] In one embodiment, constructing image description features based on the velocities of a plurality of feature points and the positions of a plurality of feature points corresponding to each image in the image sequence includes:

[0014] The image description feature is constructed based on the speeds of a plurality of feature points corresponding to each image in the image sequence, the positions of a plurality of feature points, the local features and the global features of the image.

[0015] In one embodiment, before constructing the image description feature based on the velocities of the feature points, the positions of the feature points, the local features and the global features of the image corresponding to each image in the image sequence, the method further includes:

[0016] determining a local image where a coronary artery is located in each image in the image sequence based on a segmentation result corresponding to the image;

[0017] Performing convolution calculation on the local image to obtain local features corresponding to the image;

[0018] Perform convolution calculation on the image to obtain global features corresponding to the image.

[0019] In one embodiment, before constructing the image description feature based on the velocities of the feature points, the positions of the feature points, the local features and the global features of the image corresponding to each image in the image sequence, the method further includes:

[0020] determining a local image where a coronary artery is located in each image in the image sequence based on a segmentation result corresponding to the image;

[0021] constructing a local image similarity sequence based on the local image corresponding to each image in the image sequence, and using the local image similarity corresponding to each image in the local image similarity sequence as a local feature of the image;

[0022] Based on each image in the image sequence or the segmentation result of each image, a global image similarity sequence is constructed, and the global image similarity corresponding to each image in the global image similarity sequence is used as the global feature of the image.

[0023] In one embodiment, constructing a feature similarity sequence based on a plurality of image description features corresponding to the image sequence includes:

[0024] For each pair of adjacent images in the image sequence, calculating the similarity between the image description features of the adjacent images to obtain multiple similarities;

[0025] Based on the multiple similarities, the feature similarity sequence is constructed.

[0026] In one embodiment, constructing a feature similarity sequence based on a plurality of image description features corresponding to the image sequence includes:

[0027] For each image in the image sequence, calculating a similarity between an image description feature of the image and a reference description feature to obtain a plurality of similarities;

[0028] Based on the multiple similarities, the feature similarity sequence is constructed.

[0029] In one embodiment, the step of obtaining the reference description features includes:

[0030] Averaging processing is performed on the multiple image description features to obtain the benchmark description feature.

[0031] In one embodiment, determining the corresponding phase of each image in the image sequence in the cardiac cycle based on the feature similarity sequence and the plurality of segmentation results corresponding to the image sequence includes:

[0032] Determining the periodic stage of the image corresponding to each feature similarity based on the periodic variation pattern of the feature similarity in the feature similarity sequence;

[0033] Selecting from the image sequence a number of first images in which the area where the coronary artery is located is the largest, and a number of second images in which the area where the coronary artery is located is the smallest in the segmentation results;

[0034] determining that the plurality of first images are in end-diastole and the plurality of second images are in end-systole;

[0035] The time phase of each image in the image sequence is determined based on the cycle stages of the first images and the second images.

[0036] On the other hand, the present application provides a cardiac cycle detection device, comprising:

[0037] A segmentation module is used to segment each image in the coronary angiography image sequence to obtain a segmentation result;

[0038] A first construction module is configured to construct an image description feature corresponding to each image in the image sequence based on a plurality of segmentation results corresponding to the image sequence;

[0039] A second construction module is configured to construct a feature similarity sequence based on a plurality of image description features corresponding to the image sequence;

[0040] The determining module is configured to determine a corresponding phase of each image in the image sequence in the cardiac cycle based on the feature similarity sequence and a plurality of segmentation results corresponding to the image sequence.

[0041] In another aspect, the present application provides an electronic device, comprising:

[0042] processor;

[0043] a memory for storing processor-executable instructions;

[0044] Wherein, the processor is configured to execute the above-mentioned cardiac cycle detection method.

[0045] In addition, the present application provides a computer-readable storage medium, which stores a computer program. The computer program can be executed by a processor to complete the above-mentioned cardiac cycle detection method.

[0046] The present application scheme can generate image description features for each image based on the segmentation results of each image in the image sequence generated by coronary angiography, and then construct a feature similarity sequence based on the image description features that characterizes the periodic change law of the distribution of blood vessel and catheter positions. The feature similarity sequence can be used to determine the periodic stage of the image corresponding to the feature similarity in the cardiac cycle, and then accurately identify the time phase corresponding to each image in the image sequence based on the segmentation results and feature similarity sequence corresponding to the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments of the present application.

[0048] Figure 1 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application;

[0049] Figure 2 A schematic flow chart of a cardiac cycle detection method provided in one embodiment of the present application;

[0050] Figure 3 A graph showing a trend of feature similarity changes in a feature similarity sequence provided in an embodiment of the present application;

[0051] Figure 4 Provided for an embodiment of this application Figure 2 Detailed flow diagram of step 220;

[0052] Figure 5 A flowchart of a method for generating global features and local features provided in one embodiment of the present application;

[0053] Figure 6 A schematic flow chart of a method for generating global features and local features provided in another embodiment of the present application;

[0054] Figure 7 Provided for an embodiment of this application Figure 2 Detailed flow chart of step 240;

[0055] Figure 8 This is a block diagram of a cardiac cycle detection device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0057] Similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0058] like Figure 1 As shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12, Figure 1 In the example, a processor 11 is used. Processor 11 and memory 12 are connected via bus 10. Memory 12 stores instructions executable by processor 11. Processor 11 executes these instructions, enabling electronic device 1 to perform all or part of the method described in the following embodiments. In one embodiment, electronic device 1 can be a host, server, or server cluster, and is configured to perform the cardiac cycle detection method.

[0059] The memory 12 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0060] The present application also provides a computer-readable storage medium, which stores a computer program. The computer program can be executed by the processor 11 to complete the cardiac cycle detection method provided in the present application.

[0061] See also Figure 2 , is a flow chart of a cardiac cycle detection method provided in an embodiment of the present application, such as Figure 2 As shown, the method may include the following steps 210 to 240.

[0062] Step 210: Segment each image in the coronary angiography image sequence to obtain a segmentation result.

[0063] The image sequence generated by the coronary angiography includes multiple images acquired during the coronary angiography process, and the multiple images are arranged in the order of acquisition time. Here, the images can be DICOM (Digital Imaging and Communications in Medicine) images.

[0064] An electronic device implementing this solution can acquire an image sequence generated by coronary angiography in different ways in different application scenarios. In one scenario, if cardiac cycle detection is performed on images from a completed coronary angiography procedure, the image sequence generated by the coronary angiography can be read from a specified location in local storage. In another scenario, if images generated by a currently ongoing coronary angiography procedure are processed, the latest image can be acquired in real time and combined with multiple previously acquired images to form an image sequence.

[0065] For each image in the image sequence, the electronic device can input the image into a trained segmentation network, which then outputs a segmentation result for the image. The segmentation network can be a semantic segmentation network or an instance segmentation network, and the segmentation result can be a semantic segmentation result or an instance segmentation result, used to characterize the location of blood vessels and catheters used for coronary angiography in the image. After segmenting each image and obtaining a segmentation result, multiple segmentation results corresponding to the image sequence can be obtained.

[0066] Step 220: constructing an image description feature corresponding to each image in the image sequence based on the multiple segmentation results corresponding to the image sequence.

[0067] After obtaining multiple segmentation results, the position distribution of blood vessels and catheters in each image in the image sequence can be determined, and then image description features of the images can be constructed based on the position distribution. In one embodiment, the image description features can be feature parameters in the form of multidimensional vectors, feature maps, etc.

[0068] Step 230: construct a feature similarity sequence based on the multiple image description features corresponding to the image sequence.

[0069] After obtaining the image description features corresponding to each image in the image sequence, a feature similarity sequence can be constructed based on the similarities between different image description features. This feature similarity sequence includes multiple feature similarities and can characterize the periodic changes in the distribution of blood vessels and catheters in the image sequence.

[0070] See also Figure 3 , is a graph showing a trend of feature similarity changes in a feature similarity sequence provided by an embodiment of the present application, such as Figure 3 As shown in the graph, the horizontal axis represents the sequence number of the feature similarity, that is, the sequence number of the image in the image sequence; the vertical axis represents the magnitude of the feature similarity. The trend of the change of each feature similarity in the feature similarity sequence in the graph shows a periodicity. Figure 3 The three dots represent the same phase of the three cycles, and the three triangles also represent the same phase of the three cycles. The distance between each pair of dots represents the feature similarity of multiple images within a cycle. Similarly, the distance between each pair of triangles represents the feature similarity of multiple images within a cycle.

[0071] Step 240: Determine the corresponding phase of each image in the image sequence in the cardiac cycle based on the feature similarity sequence and the multiple segmentation results corresponding to the image sequence.

[0072] Based on the periodic variation patterns of the distribution of blood vessel and catheter positions in the image sequence, as represented by the feature similarity sequence, the cardiac cycle phase of the image corresponding to each feature similarity in the image sequence can be determined. For example, if the image sequence contains 60 frames, based on the periodic variation patterns exhibited by the feature similarity sequence, it can be determined that one cardiac cycle corresponds to 15 frames. The 60 frames can be divided into four cardiac cycles, and it can be assumed that within a cardiac cycle, the first frame is at the beginning of the cardiac cycle, and the 15th frame is at the end of the cardiac cycle.

[0073] It should be noted that the phases of the cardiac cycle corresponding to the start and end stages are still uncertain. Therefore, based on the phenomenon of cardiac chamber contraction and relaxation at each stage of the cardiac cycle, it can be determined that the area occupied by blood vessels (coronary arteries) in the segmentation results of the image during cardiac chamber contraction is smaller, while the area occupied by blood vessels in the segmentation results of the image during cardiac chamber relaxation is larger. Based on this rule, the phase corresponding to each image within a cardiac cycle can be determined.

[0074] Through the above measures, image description features can be generated for each image based on the segmentation results of each image in the image sequence generated by coronary angiography, and then a feature similarity sequence that characterizes the periodic change law of the distribution of blood vessel and catheter positions can be constructed based on the image description features. The feature similarity sequence can in turn determine the periodic stage of the image corresponding to the feature similarity in the cardiac cycle, and then the time phase corresponding to each image in the image sequence can be accurately identified based on the segmentation results and feature similarity sequence corresponding to the image.

[0075] In one embodiment, when constructing image description features based on the segmentation results, see Figure 4 , provided in one embodiment of the present application Figure 4 The detailed flow chart of step 220 is as follows: Figure 4 As shown, when executing step 220, the following steps 221 to 223 can be specifically executed.

[0076] Step 221: Based on the segmentation results of each image in the image sequence, determine a number of feature points in the image.

[0077] Here, the feature points can be configured as needed. For example, the feature points may include a catheter port, a blood vessel bifurcation port, and the like.

[0078] Because the segmentation results for each image can indicate the location of blood vessels and coronary angiography catheters within the image, a point at the catheter end can be used as the catheter port, and a point at the vessel bifurcation can be used as the vessel bifurcation. For each image, one catheter port and at least one vessel bifurcation can be identified. After identifying feature points, the position of each feature point within the image can be determined. This position can be represented by the pixel coordinates of the image coordinate system established by the feature point within the image.

[0079] Step 222: Determine the speeds of the feature points corresponding to each image according to the feature points of each image in the image sequence.

[0080] For any feature point, the feature point velocity of the feature point in the next frame can be determined based on the position of the feature point in the two frames before and after the feature point in the image sequence. According to the position of the feature point in the two frames before and after, the displacement of the feature point can be determined, and the feature point velocity can be obtained by dividing the displacement by the acquisition interval between the two frames before and after. Here, the image acquisition interval is determined by the device that acquires the image during coronary angiography and is a known quantity. In this case, based on the position of the same feature point in the n-1 frame and the n frame in the image sequence, the feature point velocity of the feature point in the n frame can be determined. The feature point velocity corresponding to the 1st frame in the image sequence can be considered to be 0.

[0081] By calculating each feature point in the image separately, we can get the speed of several feature points.

[0082] Step 223: Constructing image description features based on the velocities of a plurality of feature points and the positions of a plurality of feature points corresponding to each image in the image sequence.

[0083] After obtaining the velocities and positions of the feature points corresponding to each image in the image sequence, the velocities of the feature points can be normalized to obtain normalized velocities of the feature points, and the positions of the feature points can be normalized to obtain normalized positions of the feature points. Based on the normalized velocities and positions of the feature points, image description features can be constructed.

[0084] For example, there are 5 feature points in an image, corresponding to 5 feature point velocities and 5 feature point positions (the positions include horizontal and vertical coordinates). After normalization, 15 numerical values ​​can be obtained. These 15 numerical values ​​are arranged in a specified order to obtain a vector containing 15 elements as the image description feature of the image. Here, the specified order can indicate the arrangement order of the 5 feature points, according to which the normalized feature point velocities of the 5 feature points are arranged first, and then the normalized feature point positions of the 5 feature points are arranged.

[0085] In order to ensure that the image description features corresponding to each image in the image sequence are comparable in the future, the feature points identified in each image can be unified. Exemplarily, feature points identified in all images in the image sequence can be selected to construct the image description features. Alternatively, feature points identified in a specified proportion of images in the image sequence can be selected to construct the image description features. Here, the specified proportion can be configured as needed. Exemplarily, the specified proportion can be 60%, 70%, 80%, 90%, etc. In this case, if the feature points selected for constructing the image description features are not identified in any image, the elements corresponding to these feature points can be replaced with 0 values ​​in the image feature description corresponding to the image.

[0086] Through these measures, we can identify the feature points in each image based on the segmentation results, and then construct image description features based on the speed and position of the feature points. Because the position and speed of the feature points are closely related to the state of each stage of the cardiac cycle, this image description feature can help with subsequent cardiac cycle detection.

[0087] In one embodiment, during the process of executing step 223 to construct the image description feature, the image description feature may be constructed based on the speeds of several feature points corresponding to each image in the image sequence, the positions of several feature points, and the global features and local features of the image.

[0088] Among them, the global features represent the features of the entire image; the local features represent the features of the blood vessels (coronary arteries) as local areas in the image.

[0089] Exemplarily, both global features and local features are multidimensional vectors. After constructing a multidimensional vector from the velocities of several feature points and the positions of several feature points in the manner described above, the global features and local features can be superimposed on the multidimensional vector to obtain a new multidimensional vector as an image description feature.

[0090] Through this measure, the image description features can include richer feature information, which is helpful for subsequent cardiac cycle detection.

[0091] In one embodiment, see Figure 5 , which is a flow chart of a method for generating global features and local features provided in an embodiment of the present application, such as Figure 5 As shown, local features and global features may be generated for each image in the image sequence through the following steps 510 to 530 .

[0092] Step 510: For each segmentation result corresponding to each image in the image sequence, determine the local image where the coronary artery is located in the image.

[0093] The segmentation result corresponding to any image in the image sequence can represent the location of the coronary artery in the image. Based on the location, a local image containing the coronary artery can be cropped from the image. Here, the local image can be a rectangular image.

[0094] Step 520: Perform convolution calculation on the local image to obtain local features corresponding to the image.

[0095] The electronic device can input a local image into a convolutional neural network, perform convolution calculations on the local image through the convolutional neural network, and thus output local features corresponding to the image.

[0096] Step 530: Perform convolution calculation on the image to obtain global features corresponding to the image.

[0097] The electronic device can input the image into the convolutional neural network, perform convolution calculation on the image through the convolutional neural network, and output the global features corresponding to the image.

[0098] Through the above measures, global features and local features can be generated for each image in the image sequence.

[0099] In one implementation, see Figure 6 , which is a flow chart of a method for generating global features and local features provided by another embodiment of the present application, such as Figure 6As shown, local features and global features may be generated for each image in the image sequence through the following steps 610 to 630 .

[0100] Step 610: For each segmentation result corresponding to each image in the image sequence, determine the local image where the coronary artery is located in the image.

[0101] The segmentation result corresponding to any image in the image sequence can represent the location of the coronary artery in the image. Based on the location, a local image containing the coronary artery can be cropped from the image. Here, the local image can be a rectangular image.

[0102] Step 620: construct a local image similarity sequence based on the local image corresponding to each image in the image sequence, and use the local image similarity corresponding to each image in the local image similarity sequence as the local feature of the image.

[0103] For each pair of partial images of adjacent images in the image sequence, the electronic device may calculate the similarity between the two using an image similarity evaluation algorithm, and use this as the partial image similarity of the subsequent frame of the adjacent images. The image similarity evaluation algorithm may be a histogram comparison algorithm, mean square error, structural similarity index, peak signal-to-noise ratio, perceptual hashing algorithm, etc., and is not limited in this application.

[0104] After calculating the similarity between the partial image of the n-1th frame image and the partial image of the nth frame image in the image sequence, the similarity can be used as the partial image similarity of the nth frame image. For the first frame image in the image sequence, it can be considered that there is no corresponding partial image similarity.

[0105] The multiple local image similarities are arranged in the order of their corresponding images in the image sequence to form a local image similarity sequence. Furthermore, each local image similarity in the local image similarity sequence can be used as a local feature of the image corresponding to the local image similarity.

[0106] Step 630: Based on each image in the image sequence or the segmentation result of each image, a global image similarity sequence is constructed, and the global image similarity corresponding to each image in the global image similarity sequence is used as a global feature of the image.

[0107] For each pair of adjacent images in the image sequence, the electronic device can calculate the similarity of the pair of adjacent images themselves through an image similarity evaluation algorithm, or calculate the similarity of the segmentation results of the pair of adjacent images as the global image similarity corresponding to the latter frame image in the adjacent images.

[0108] After calculating the similarity between the n-1th and nth frames in the image sequence, this similarity can be used as the global image similarity of the nth frame. Alternatively, after calculating the similarity between the segmentation results of the n-1th and nth frames in the image sequence, this similarity can be used as the global image similarity of the nth frame. For the first frame in the image sequence, it can be assumed that it has no corresponding global image similarity.

[0109] The multiple global image similarities are arranged in the order of their corresponding images in the image sequence to form a global image similarity sequence. Furthermore, each global image similarity in the global image similarity sequence can be used as a global feature of the image corresponding to the global image similarity.

[0110] Through the above measures, global features and local features can be generated for images in the image sequence.

[0111] In one embodiment, when executing step 230 to construct a feature similarity sequence, the electronic device may calculate the similarity between the image description features of each pair of adjacent images in the image sequence, thereby obtaining multiple similarities. Here, if the image description features are multidimensional vectors, the similarity can be calculated using methods such as Euclidean distance and cosine distance; if the image description features are feature maps, the similarity can be calculated using an image similarity evaluation algorithm.

[0112] After calculating the similarity between the image description features of the n-1th and nth frames in the image sequence, this similarity can be used as the similarity corresponding to the nth frame. After obtaining multiple similarities, the multiple similarities can be arranged according to the order of their corresponding images in the image sequence to construct a feature similarity sequence.

[0113] Through the above measures, a feature similarity sequence can be constructed through multiple image description features, which can be used to subsequently characterize the changing trend of the image description features.

[0114] In one embodiment, when executing step 230 to construct a feature similarity sequence, the electronic device may calculate the similarity between the image description features of each image in the image sequence and the benchmark description features, thereby obtaining multiple similarities. Here, the benchmark description features can be obtained by converting the image description features of all images in the image sequence and used as a benchmark value for calculating similarity. Here, if the image description features are multidimensional vectors, the similarity can be calculated using methods such as Euclidean distance and cosine distance; if the image description features are feature maps, the similarity can be calculated using an image similarity evaluation algorithm.

[0115] After obtaining the multiple similarities, the multiple similarities may be arranged according to the arrangement order of their corresponding images in the image sequence, thereby constructing a feature similarity sequence.

[0116] Through the above measures, a feature similarity sequence can be constructed through multiple image description features, which can be used to subsequently characterize the changing trend of the image description features.

[0117] In one embodiment, if a feature similarity sequence is constructed using a baseline descriptive feature, the electronic device may perform averaging processing on multiple image descriptive features corresponding to the image sequence to obtain the baseline descriptive feature. Here, if the image descriptive feature is a vector, the average of the elements at the same position in the multiple image descriptive features may be calculated, thereby constructing the baseline descriptive feature based on the multiple averages. If the image descriptive feature is a feature map, the average of the pixel values ​​at the same position in the multiple image descriptive features may be calculated, thereby constructing the baseline descriptive feature based on the multiple averages.

[0118] For example, the image description feature is recorded as TensorFrame=[v1,v2,v3...v k ], the average of element v1, the average of element v2, the average of element v3, and so on in multiple image description features can be calculated. k The average of multiple elements is used to construct the benchmark description feature.

[0119] In one embodiment, see Figure 7 , provided in one embodiment of the present application Figure 2 The detailed flow chart of step 240 is as follows: Figure 7 As shown, when executing step 240, steps 241 to 244 may be specifically executed.

[0120] Step 241: Based on the periodic variation pattern of the feature similarities in the feature similarity sequence, determine the periodic stage of the image corresponding to each feature similarity.

[0121] Because the changing trend of feature similarities in the feature similarity sequence exhibits a periodic pattern, the electronic device can determine the number of cardiac cycles included in the image sequence, group images corresponding to each feature similarity in the image sequence into cardiac cycles, and determine the cardiac cycle phase of each image. In this case, the image corresponding to each cardiac cycle can be determined, and further, the cardiac cycle phase of each image can be determined.

[0122] Step 242: Select from the image sequence a number of first images in which the area where the coronary artery is located is the largest, and a number of second images in which the area where the coronary artery is located is the smallest in the segmentation results.

[0123] The electronic device may select, from multiple frames of images corresponding to at least one cardiac cycle, a first image in which the area where the coronary artery is located is the largest and a second image in which the area where the coronary artery is located is the smallest based on the image segmentation result.

[0124] From the multiple frames of images corresponding to one cardiac cycle, one first image and one second image can be selected. If the multiple frames of images corresponding to multiple cardiac cycles are selected, multiple first images and multiple second images can be obtained.

[0125] Step 243: Determine that the plurality of first images are in the end-diastole phase and the plurality of second images are in the end-systole phase.

[0126] Step 244: Determine the time phase of each image in the image sequence based on the cycle stages of the plurality of first images and the plurality of second images.

[0127] At the end of diastole in the cardiac cycle, blood vessels expand to their maximum extent, and accordingly, the area occupied by the blood vessels (coronary arteries) in the image is the largest. Therefore, it can be determined that the first image is at the end of diastole. At the end of systole in the cardiac cycle, blood vessels contract to their maximum extent, and accordingly, the area occupied by the blood vessels in the image is the smallest. Therefore, it can be determined that the second image is at the end of systole.

[0128] Since the first image is in end-diastole and the second image is in end-systole, the phases of other images in the same cardiac cycle as the first and second images can be determined based on their respective cycle phases. After determining the phases corresponding to each image in any cardiac cycle, the phases corresponding to each image in other cardiac cycles can be determined based on the cycle phases of each image in that cardiac cycle and the cycle phases of each image in other cardiac cycles (images in the same cycle phase of different cardiac cycles have the same corresponding phases). In this case, the phase of each image in the image sequence can be determined.

[0129] To obtain a more precise time phase, the time phase corresponding to the first image at the end of diastole can be determined as the slowed filling phase, and the time phase corresponding to the second image at the end of systole can be determined as the slowed ejection phase. Then, based on the order of the other six time phases (isovolumetric contraction, rapid ejection phase, prediastole, isovolumetric relaxation, rapid filling phase, and atrial contraction) relative to the slowed filling phase and slowed ejection phase in the entire cardiac cycle, as well as the proportion of each time phase in the entire cardiac cycle, the time phases corresponding to multiple images corresponding to the entire cardiac cycle can be determined. Similarly, after determining the time phases of each image in one cardiac cycle, the time phases of each image in other cardiac cycles can be determined.

[0130] Through the above measures, the cardiac cycle to which each image in the image sequence belongs can be determined based on the periodic change law of feature similarity, and then the first image and the second image can be screened out based on the characteristics of the end-diastolic and end-systolic vascular areas, and the time phases of the first image and the second image can be determined, and then the time phases of other images can be inferred.

[0131] This application solution, without the need for additional measurement equipment to collect physiological signals, can determine the phase of each image in the image sequence and complete cardiac cycle detection using only the image sequence generated by coronary angiography. In addition, this process uses deep learning algorithms for segmentation, effectively improving the robustness of the overall algorithm.

[0132] Figure 8 FIG. 1 is a block diagram of a cardiac cycle detection device according to an embodiment of the present invention. Figure 8 As shown, the device may include:

[0133] A segmentation module 810 is used to segment each image in the coronary angiography image sequence to obtain a segmentation result;

[0134] A first construction module 820 is configured to construct an image description feature corresponding to each image in the image sequence based on a plurality of segmentation results corresponding to the image sequence;

[0135] A second construction module 830 is configured to construct a feature similarity sequence based on a plurality of image description features corresponding to the image sequence;

[0136] The determination module 840 is configured to determine a corresponding phase of each image in the image sequence in the cardiac cycle based on the feature similarity sequence and a plurality of segmentation results corresponding to the image sequence.

[0137] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned cardiac cycle detection method, and will not be repeated here.

[0138] In several embodiments provided in this application, the disclosed devices and methods may also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a portion of code, and the module, program segment, or a portion of code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, may be implemented using a dedicated hardware-based system that performs the specified functions or actions, or may be implemented using a combination of dedicated hardware and computer instructions.

[0139] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0140] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A cardiac cycle detection method, characterized in that: include: Segment each image in the coronary angiography image sequence to obtain a segmentation result; Constructing an image description feature corresponding to each image in the image sequence based on a plurality of segmentation results corresponding to the image sequence; wherein constructing an image description feature corresponding to each image in the image sequence based on the plurality of segmentation results corresponding to the image sequence includes: determining a plurality of feature points in the image based on the segmentation result of each image in the image sequence; determining a plurality of feature point velocities corresponding to each image based on the plurality of feature points in each image in the image sequence; and constructing the image description feature based on the plurality of feature point velocities and positions of the plurality of feature points corresponding to each image in the image sequence; constructing a feature similarity sequence based on a plurality of image description features corresponding to the image sequence; Based on the feature similarity sequence and a plurality of segmentation results corresponding to the image sequence, a corresponding phase of each image in the image sequence in the cardiac cycle is determined.

2. The method according to claim 1, characterized in that The constructing of image description features based on the velocities of the plurality of feature points and the positions of the plurality of feature points corresponding to each image in the image sequence includes: The image description feature is constructed based on the speeds of a plurality of feature points corresponding to each image in the image sequence, the positions of a plurality of feature points, the local features and the global features of the image.

3. The method according to claim 2, characterized in that Before constructing the image description feature based on the velocities of the feature points, the positions of the feature points, the local features and the global features of the image corresponding to each image in the image sequence, the method further includes: determining a local image where a coronary artery is located in each image in the image sequence based on a segmentation result corresponding to the image; Performing convolution calculation on the local image to obtain local features corresponding to the image; Perform convolution calculation on the image to obtain global features corresponding to the image.

4. The method according to claim 2, characterized in that Before constructing the image description feature based on the velocities of the feature points, the positions of the feature points, the local features and the global features of the image corresponding to each image in the image sequence, the method further includes: determining a local image where a coronary artery is located in each image in the image sequence based on a segmentation result corresponding to the image; constructing a local image similarity sequence based on the local image corresponding to each image in the image sequence, and using the local image similarity corresponding to each image in the local image similarity sequence as a local feature of the image; Based on each image in the image sequence or the segmentation result of each image, a global image similarity sequence is constructed, and the global image similarity corresponding to each image in the global image similarity sequence is used as the global feature of the image.

5. The method according to claim 1, wherein The constructing of a feature similarity sequence based on a plurality of image description features corresponding to the image sequence includes: For each pair of adjacent images in the image sequence, calculating the similarity between the image description features of the adjacent images to obtain multiple similarities; Based on the multiple similarities, the feature similarity sequence is constructed.

6. The method according to claim 1, characterized in that The constructing of a feature similarity sequence based on a plurality of image description features corresponding to the image sequence includes: For each image in the image sequence, calculating a similarity between an image description feature of the image and a reference description feature to obtain a plurality of similarities; Based on the multiple similarities, the feature similarity sequence is constructed.

7. The method according to claim 6, characterized in that The step of obtaining the benchmark description feature includes: Averaging processing is performed on the multiple image description features to obtain the benchmark description feature.

8. The method according to claim 1, characterized in that Determining the corresponding phase of each image in the image sequence in the cardiac cycle based on the feature similarity sequence and the plurality of segmentation results corresponding to the image sequence includes: Determining the periodic stage of the image corresponding to each feature similarity based on the periodic variation pattern of the feature similarity in the feature similarity sequence; Selecting from the image sequence a number of first images in which the area where the coronary artery is located is the largest, and a number of second images in which the area where the coronary artery is located is the smallest in the segmentation results; determining that the plurality of first images are in end-diastole and the plurality of second images are in end-systole; The time phase of each image in the image sequence is determined based on the cycle stages of the first images and the second images.

9. A cardiac cycle detection device, characterized in that: include: A segmentation module is used to segment each image in the coronary angiography image sequence to obtain a segmentation result; a first construction module configured to construct an image description feature corresponding to each image in the image sequence based on a plurality of segmentation results corresponding to the image sequence; wherein the first construction module is specifically configured to: determine a plurality of feature points in the image based on the segmentation result of each image in the image sequence; determine a plurality of feature point velocities corresponding to each image based on the plurality of feature points in the image sequence; and construct the image description feature based on the plurality of feature point velocities and positions of the plurality of feature points corresponding to each image in the image sequence; A second construction module is configured to construct a feature similarity sequence based on a plurality of image description features corresponding to the image sequence; The determining module is configured to determine a corresponding phase of each image in the image sequence in the cardiac cycle based on the feature similarity sequence and a plurality of segmentation results corresponding to the image sequence.

10. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to execute the cardiac cycle detection method described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and the computer program can be executed by a processor to complete the cardiac cycle detection method according to any one of claims 1 to 8.

Images