Method and device for quantitatively analyzing coronary angiography based on angiography video

Abstract

An embodiment of the present invention relates to a method and apparatus for quantitative analysis of coronary angiography based on angiography videos. The method includes: obtaining an angiography video; extracting video frame images to generate a first image sequence; performing target detection and semantic segmentation processing on each first image in the first image sequence for the stenotic segment vessels, so as to obtain one or more target detection frames for marking the stenotic segment vessels and a stenotic segment vessel mask image within each target detection frame on each first image; performing image optimization processing on the first image sequence to obtain a specified number of optimized images; performing quantitative analysis of coronary angiography on the stenotic segment vessel mask images within each target detection frame on each optimized image to generate corresponding vessel stenosis rates. Through the present invention, the excessive dependence on manual experience in traditional practices can be eliminated, and the accuracy of image extraction and the calculation accuracy of stenosis rates can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a method and device for quantitatively analyzing coronary angiography based on angiography videos. Background Art

[0002] Coronary artery stenosis can lead to insufficient blood supply to the heart, thereby causing myocardial dysfunction and / or lesions. Angiography technology is based on the principle that X-rays cannot penetrate the contrast agent. The contrast agent is injected into the blood vessels of the detection object, and the process of the contrast agent passing through the blood vessels under X-rays is imaged to output an angiography video. When detecting the stenosis of the coronary artery, usually, first, an angiography video of the coronary blood vessels is obtained based on angiography technology, and then a doctor selects video images with stenotic segment blood vessels from the angiography video according to personal experience for quantitative analysis of the vascular stenosis rate, also known as qualitative comparative analysis (QCA), to calculate the corresponding vascular stenosis rate. This operation mode is overly dependent on human factors, such as personnel's professional experience and the recognition ability of the human eye, and is extremely prone to problems such as inaccurate extraction of video images and insufficient accuracy of stenosis rate calculation. Summary of the Invention

[0003] The purpose of the present invention is to provide a method, device, electronic device, and computer-readable storage medium for quantitatively analyzing coronary angiography based on angiography videos, aiming at the defects of the prior art. The method includes performing video interception and video frame image extraction on the angiography video of the coronary blood vessels, performing target detection and semantic segmentation processing on the extracted image sequence based on an image target detection and semantic segmentation model, selecting the extracted images based on the confidence of target recognition, and performing quantitative analysis of coronary angiography on each stenotic segment blood vessel on the selected images to generate the corresponding vascular stenosis rate. Through the present invention, the excessive dependence on artificial experience in the traditional method can be eliminated, and the accuracy of image extraction and the accuracy of stenosis rate calculation can be improved.

[0004] To achieve the above purpose, in the first aspect of the embodiments of the present invention, a method for quantitatively analyzing coronary angiography based on angiography videos is provided, and the method includes:

[0005] Obtaining an angiography video of coronary angiography;

[0006] Performing video frame image extraction on the angiography video to generate a corresponding first image sequence;

[0007] Based on a preset image object detection and semantic segmentation model, perform object detection and semantic segmentation processing on each first image in the first image sequence to obtain one or more object detection frames for marking narrow-segment blood vessels and a segment of narrow-segment blood vessel mask image in each object detection frame on each first image; each object detection frame corresponds to a detection frame confidence level;

[0008] According to the detection frame confidence level, perform image optimization processing on the first image sequence to obtain a specified number of optimized images;

[0009] Perform coronary angiography quantitative analysis on the narrow-segment blood vessel mask image within each object detection frame on each optimized image to generate a corresponding blood vessel stenosis rate.

[0010] Preferably, the extraction of video frame images from the angiography video to generate a corresponding first image sequence specifically includes:

[0011] Perform video interception on the angiography video, and retain the video content of the stage when the contrast agent fills the coronary artery to generate a corresponding intercepted angiography video;

[0012] In chronological order, perform video frame image extraction processing on the intercepted angiography video to generate a corresponding video frame image sequence, and count the number of video frame images in the video frame image sequence to generate a corresponding first total number; the video frame image sequence includes multiple video frame images;

[0013] When the first total number does not exceed a preset image total number threshold, use each video frame image as the corresponding first image, and sort all the obtained first images in chronological order to generate the first image sequence;

[0014] When the first total number exceeds the image total number threshold, extract the video frame images with all odd-index sorting indices or all even-index sorting indices from the video frame image sequence as the corresponding first images; and sort all the obtained first images in chronological order to generate the first image sequence.

[0015] Preferably, the image object detection and semantic segmentation model includes a Mask R-CNN model; when the image object detection and semantic segmentation model is specifically a Mask R-CNN model, a Residual Network ResNet50 is used as its feature extraction backbone network.

[0016] Preferably, the performing image optimization processing on the first image sequence according to the detection frame confidence level to obtain a specified number of optimized images specifically includes:

[0017] In the first image sequence, the detection box confidence of all the target detection boxes on each of the first images is averaged to generate a corresponding average confidence of the first image;

[0018] All the first images are sorted in descending order according to the corresponding average confidence of the first image, and a specified number of the first images with higher rankings are used as the preferred images.

[0019] Preferably, the coronary angiography quantitative analysis of the narrow segment vascular mask image in each of the target detection boxes on each of the preferred images to generate a corresponding vascular stenosis rate specifically includes:

[0020] Traverse all the target detection boxes on the current preferred image, and denote the currently traversed target detection box as the current target detection box;

[0021] Identify the blood vessel edge and the blood vessel centerline of the narrow segment vascular mask image in the current target detection box to generate a corresponding first blood vessel edge and a first centerline; the first centerline includes a plurality of centerline pixel points P i , where the first centerline pixel point P 1 is the point closest to the coronary artery entrance in the blood flow direction, and the last centerline pixel point P N is the point farthest from the coronary artery entrance in the blood flow direction, 1 ≤ i ≤ N, and N is the total number of centerline pixel points of the first centerline;

[0022] According to the first blood vessel edge, analyze the blood vessel diameter length corresponding to each centerline pixel point P i on the first centerline to generate a corresponding first blood vessel diameter d i ;

[0023] According to the linear change relationship of the blood vessel between the centerline pixel point P 1 and the centerline pixel point P N , and the first blood vessel diameter d i of each centerline pixel point P i , analyze the stenosis rate corresponding to each centerline pixel point P i to generate a corresponding first stenosis rate r i ;

[0024] From all the obtained first stenosis rates r i , select the maximum value as the vascular stenosis rate corresponding to the narrow segment vascular mask image in the current target detection box.

[0025] Further, according to the first blood vessel edge, for each center line pixel point P on the first center line i Analyze the corresponding blood vessel diameter length to generate a corresponding first blood vessel diameter d i , specifically including:

[0026] According to the direction relationship between the center line pixel point P i and its adjacent eight-region pixel points, draw four straight lines through the center line pixel point P i and denote them as the first, second, third, and fourth straight lines respectively; the first straight line passes through the upper left adjacent pixel point of the center line pixel point P i , the center line pixel point P i and the lower right adjacent pixel point of the center line pixel point P i ; the second straight line passes through the upper adjacent pixel point of the center line pixel point P i , the center line pixel point P i and the lower adjacent pixel point of the center line pixel point P i ; the third straight line passes through the upper right adjacent pixel point of the center line pixel point P i , the center line pixel point P i and the lower left adjacent pixel point of the center line pixel point P i ; the third straight line passes through the right adjacent pixel point of the center line pixel point P i , the center line pixel point P i and the left adjacent pixel point of the center line pixel point P i ;

[0027] Denote the line segments where the first, second, third, and fourth straight lines intersect with the first blood vessel edge as the corresponding first, second, third, and fourth line segments respectively; calculate the line segment lengths of the first, second, third, and fourth line segments to generate corresponding first, second, third, and fourth line segment lengths; and select the minimum value from the first, second, third, and fourth line segment lengths as the corresponding first blood vessel diameter d of the center line pixel point P i i .

[0028] Further, according to the blood vessel linear change relationship between the center line pixel point P 1 and the center line pixel point P N , and the first blood vessel diameter d of each center line pixel point P i , analyze the stenosis rate corresponding to each center line pixel point P i to generate a corresponding first stenosis rate r i , specifically including: i ​​

[0029] According to the first blood vessel diameter d 1 and the first blood vessel diameter d N , construct a linear function f(i) that reflects the linear change relationship of the centerline pixel point P 1 to the centerline pixel point P N of the blood vessel, f(i)=d 1 +k*(i - 1), where k=(d N -d 1 ) / (N - 1);

[0030] According to the linear change relationship function f(i), calculate the linear change diameter lengths corresponding to each of the centerline pixel points P i to generate corresponding first reference diameters d ’ i ;

[0031] According to the first blood vessel diameter d i and the first reference diameter d ’ i , calculate the corresponding first stenosis rate r i for each of the centerline pixel points P i , r i =1 - d i / d ’ i .

[0032] A second aspect of the embodiments of the present invention provides an apparatus for implementing the method described in the first aspect above, including: an acquisition module, an image preprocessing module, a narrow segment blood vessel processing module, an image optimization module, and a quantitative analysis module;

[0033] The acquisition module is used to acquire an angiography video of coronary angiography;

[0034] The image preprocessing module is used to extract video frame images from the angiography video to generate a corresponding first image sequence;

[0035] The narrow segment blood vessel processing module is used to perform target detection and semantic segmentation processing on the narrow segment blood vessels in each of the first images in the first image sequence based on a preset image target detection and semantic segmentation model, so as to obtain one or more target detection frames for marking the narrow segment blood vessels and a narrow segment blood vessel mask image in each of the target detection frames on each of the first images; each of the target detection frames corresponds to a detection frame confidence level;

[0036] The image optimization module is used to perform image optimization processing on the first image sequence according to the detection frame confidence level to obtain a specified number of optimized images;

[0037] The quantitative analysis module is used to perform coronary angiography quantitative analysis on the stenotic segment vascular mask images within each of the target detection frames on each of the preferred images to generate corresponding vascular stenosis rates.

[0038] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0039] The processor is used to be coupled with the memory, read and execute instructions in the memory to implement the method steps described in the first aspect above;

[0040] The transceiver is coupled with the processor, and the processor controls the transceiver to perform message sending and receiving.

[0041] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium storing computer instructions, and when the computer instructions are executed by a computer, the computer is caused to execute the instructions of the method described in the first aspect above.

[0042] An embodiment of the present invention provides a method, device, electronic device, and computer-readable storage medium for performing coronary angiography quantitative analysis based on an angiography video. Video interception and video frame image extraction processing are performed on the angiography video of coronary blood vessels. Target detection and semantic segmentation processing of stenotic segment blood vessels are performed on the extracted image sequence based on an image target detection and semantic segmentation model. The extracted images are optimized based on the confidence of target recognition, and coronary angiography quantitative analysis is performed on each stenotic segment blood vessel on the optimized images to generate corresponding vascular stenosis rates. By means of the present invention, the excessive dependence on manual experience in traditional practices is eliminated, and the accuracy of image extraction and the calculation accuracy of stenosis rates are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of a method for performing coronary angiography quantitative analysis based on an angiography video provided in Embodiment 1 of the present invention;

[0044] Figure 2 It is a module structure diagram of a device for performing coronary angiography quantitative analysis based on an angiography video provided in Embodiment 2 of the present invention;

[0045] Figure 3 It is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] A method for coronary angiography quantitative analysis based on angiography video provided in the first embodiment of the present invention is as Figure 1 shown in the schematic diagram of the method for coronary angiography quantitative analysis based on angiography video provided in the first embodiment of the present invention. This method mainly includes the following steps:

[0048] Step 1: Obtain the angiography video of coronary angiography.

[0049] Here, coronary angiography is to inject a contrast agent into the blood vessels of the detection object and take images of the process of the contrast agent passing through the coronary blood vessels under X-ray. The angiography video is the video data obtained from the image capture.

[0050] Step 2: Extract video frame images from the angiography video to generate a corresponding first image sequence;

[0051] Specifically, it includes: Step 21: Perform video interception on the angiography video, and retain the video content of the stage when the contrast agent fills the coronary artery to generate a corresponding intercepted angiography video;

[0052] Here, since the angiography video contains the entire process video content from the injection of the contrast agent into the blood vessels to gradually filling the entire coronary artery and then gradually dissipating, and the embodiments of the present invention focus on the video content after the contrast agent reaches the coronary artery. To improve the data analysis efficiency, it is necessary to perform video interception on the angiography video in advance. There are various interception methods; one of them is to set a relative time threshold according to the implementation experience of coronary angiography and cut off the video data in the angiography video before the relative time threshold, and retain the video data after the relative time threshold as the video content of the stage when the contrast agent fills the coronary artery to generate a corresponding intercepted angiography video;

[0053] Step 22: Extract and process the video frame images of the intercepted angiography video in chronological order to generate a corresponding video frame image sequence, and count the number of video frame images in the video frame image sequence to generate a corresponding first total number; among them, the video frame image sequence includes multiple video frame images;

[0054] Here, each type of video data is defaultly associated with a corresponding video sampling frame rate parameter. The angiography video is intercepted according to the video sampling frame rate parameter corresponding to the angiography video, and video frame images are extracted. Each extracted image is a video frame image. Sorting the video frame images in chronological order can obtain a video frame image sequence;

[0055] Step 23, when the first total number does not exceed the preset image total number threshold, each video frame image is used as the corresponding first image, and all the obtained first images are sorted in chronological order to generate a first image sequence; when the first total number exceeds the image total number threshold, the video frame images with all odd-index sorting indexes are extracted from the video frame image sequence as the corresponding first images, or the video frame images with all even-index sorting indexes are extracted as the corresponding first images; and all the obtained first images are sorted in chronological order to generate a first image sequence.

[0056] Here, if the number of images in the video frame image sequence is too large, it will affect the model operation efficiency in the subsequent steps. To improve the model operation efficiency, it is necessary to control the number of images in the video frame image sequence in advance; the control method is to preset an image total number threshold and identify whether the total number of images in the video frame image sequence, that is, the first total number, exceeds this threshold; if it does not exceed this threshold, it means that there is no need to reduce the number of images in the video frame image sequence. Each video frame image is directly used as the first image, and the first image sequence composed of the first images is sent to the subsequent steps for processing; if it exceeds this threshold, it means that the number of images in the video frame image sequence needs to be reduced. To ensure that no valid data is lost due to image reduction, the method of frame extraction and reduction of adjacent images is used during reduction to achieve the purpose of not losing valid data. Extracting all odd frames or all even frames from the video frame image sequence to form the first image sequence in the current step is to perform frame extraction and reduction processing on adjacent images.

[0057] Step 3, based on the preset image object detection and semantic segmentation model, perform object detection and semantic segmentation processing on each first image in the first image sequence to obtain one or more object detection frames for marking narrow-segment blood vessels and a narrow-segment blood vessel mask image in each object detection frame on each first image;

[0058] Among them, each object detection frame corresponds to a detection frame confidence; the image object detection and semantic segmentation model includes the Mask R-CNN model; when the image object detection and semantic segmentation model is specifically the Mask R-CNN model, the Residual Network ResNet50 is used as its feature extraction backbone network.

[0059] Here, an image object detection and semantic segmentation model is used to perform narrow-section blood vessel object detection on the input first image, so as to obtain one or more object detection frames for marking narrow-section blood vessels. The detection frame confidence of each object detection frame is used to identify the credibility of the image within the frame as a narrow-section blood vessel image; the image object detection and semantic segmentation model is also used to perform image semantic segmentation on the recognized object, that is, the recognized narrow-section blood vessel, in each object detection frame, and use the segmented mask image as the narrow-section blood vessel mask image;

[0060] There are various implementation methods for the image object detection and semantic segmentation model. One of them is to implement it based on the neural network architecture of the Mask R-CNN model; when the image object detection and semantic segmentation model is specifically the Mask R-CNN model, its neural network structure can refer to the article "Mask R-CNN" published by the authors Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick, including: a feature extraction network layer, a Region Proposal Network (RPN) layer, a Region Of Interest Align (ROI Align) network layer, and a Region Head (ROI HEAD) network layer; the feature extraction network layer is connected to the region candidate network layer, and the region candidate network layer is connected to the region alignment network layer; the region alignment network layer is connected to the region head network layer; the region head network layer includes; two sub-networks are respectively an object detection branch network and an object segmentation branch network; the object detection branch network is used to output the object detection frame and detection frame confidence of the narrow-section blood vessel, and the object segmentation branch network is used to output the narrow-section blood vessel mask image;

[0061] The feature extraction network layer of the embodiment of the present invention is specifically composed of five-level Residual Network (ResNet) and the corresponding five-level Feature Pyramid Networks (FPN); the region candidate network layer includes five-level region candidate networks, corresponding to the five-level feature pyramid network; when implementing the five-level residual network, the embodiment of the present invention uses the ResNet-50 network structure for implementation and uses it as the backbone network for feature extraction.

[0062] Step 4, according to the detection frame confidence, perform image optimization processing on the first image sequence to obtain a specified number of optimized images;

[0063] Specifically, it includes: in the first image sequence, calculating the average confidence of the detection box for all target detection boxes on each first image to generate the corresponding average confidence of the first image; sorting all the first images in descending order according to the corresponding average confidence of the first image, and taking the specified number of first images with the top ranking as the preferred images.

[0064] Here, the higher the average confidence of the first image indicates that the narrow segment vascular characteristics on the corresponding first image are more obvious; the specified number is set according to specific requirements. For example, if the specified number is 3, then in the current step, the 3 first images with the most obvious narrow segment vascular characteristics will be extracted from the first image sequence as the preferred images.

[0065] Step 5, performing coronary angiography quantitative analysis on the narrow segment vascular mask image within each target detection box on each preferred image to generate the corresponding vascular stenosis rate;

[0066] Specifically, it includes: Step 51, traversing each target detection box on the current preferred image, and marking the currently traversed target detection box as the current target detection box;

[0067] Step 52, identifying the blood vessel edge and the blood vessel centerline of the narrow segment vascular mask image within the current target detection box to generate the corresponding first blood vessel edge and the first centerline;

[0068] Among them, the first centerline includes multiple centerline pixel points P i , where the first centerline pixel point P 1 is the point closest to the coronary artery entrance in the blood flow direction, and the last centerline pixel point P N is the point farthest from the coronary artery entrance in the blood flow direction, 1 ≤ i ≤ N, and N is the total number of centerline pixel points of the first centerline;

[0069] Here, there are multiple methods for implementing blood vessel edge recognition on the stenotic segment blood vessel mask image; one of them is to perform binarization processing on the image content of the current target detection frame to obtain a first binary image, and the pixel values ​​of all pixel points of the original stenotic segment blood vessel mask image on the first binary image will be converted into a preset foreground pixel value A, and the pixel values ​​of all pixel points outside the stenotic segment blood vessel mask image will be converted into a preset background pixel value B; then, each pixel point on the first binary image whose pixel value is the foreground pixel value A is traversed point by point, and during the traversal, if one of the pixel values ​​of the neighboring eight-domain pixel points of the currently traversed pixel point is the background pixel value B, then the currently traversed pixel point is regarded as an edge point; after the traversal is completed, the closed curve obtained by connecting each edge point in sequence in a clockwise or counterclockwise manner is the blood vessel edge; here, the so-called neighboring eight-domain pixel points are actually eight pixel points of the upper left neighboring pixel point, the upper neighboring pixel point, the upper right neighboring pixel point, the right neighboring pixel point, the lower right neighboring pixel point, the lower neighboring pixel point, the lower left neighboring pixel point and the left neighboring pixel point of the pixel point;

[0070] Here, there are multiple methods for realizing the vascular centerline recognition of the stenotic segment vascular mask image; one of them is to perform binarization processing on the image content of the current target detection frame to obtain a second binary image, and the pixel values ​​of all pixels of the original stenotic segment vascular mask image on the second binary image will be converted into a preset foreground pixel value A, and the pixel values ​​of all pixels outside the stenotic segment vascular mask image will be converted into a preset background pixel value B; without changing the topological properties of the vascular image, the centerline extraction processing of the stenotic segment vascular mask image of the first binary image is performed based on the topological refinement method to generate a first centerline; here, the topological properties of the vascular image mainly refer to the connectivity of the blood vessels;

[0071] When the first center line is obtained, in order to mark the direction of the center line, the point closest to the entrance of the coronary artery, that is, the entrance point of the stenosis segment, is deliberately used as the first center line pixel point P of the first center line. 1 , the point farthest from the entrance of the coronary artery, that is, the exit point of the stenosis segment, is taken as the last centerline pixel point P of the first centerline N ;

[0072] Step 53: According to the first blood vessel edge, each center line pixel point P on the first center line is i The corresponding blood vessel diameter length is analyzed to generate the corresponding first blood vessel diameter d i ;

[0073] Specifically include: Step 531, according to the center line pixel point P i The direction relationship between the pixel points in the eight neighboring regions, through the center line pixel point P i Make four straight lines and record them as the first, second, third and fourth straight lines;

[0074] Among them, the first straight line passes through the center line pixel point P i 's upper left adjacent pixel point, the center line pixel point P i and the center line pixel point P i 's lower right adjacent pixel point; the second straight line passes through the center line pixel point P i 's upper adjacent pixel point, the center line pixel point P i and the center line pixel point P i 's lower adjacent pixel point; the third straight line passes through the center line pixel point P i 's upper right adjacent pixel point, the center line pixel point P i and the center line pixel point P i 's lower left adjacent pixel point; the third straight line passes through the center line pixel point P i 's right adjacent pixel point, the center line pixel point P i and the center line pixel point P i 's left adjacent pixel point;

[0075] Step 532, respectively record the line segments where the first, second, third, and fourth straight lines intersect with the first blood vessel edge as the corresponding first, second, third, and fourth line segments; calculate the line segment lengths of the first, second, third, and fourth line segments to generate the corresponding first, second, third, and fourth line segment lengths; and select the minimum value from the first, second, third, and fourth line segment lengths as the first blood vessel diameter d i corresponding to the center line pixel point P i ;

[0076] Here, first, it is known that the blood vessel diameter from the center line pixel point P i to the first blood vessel edge should be within the range of all straight lines passing through the center line pixel point P i ; and in the image, there are actually only 4 straight lines that can be drawn through any pixel point, that is, the above-mentioned first, second, third, and fourth straight lines, which means that the blood vessel diameter from the center line pixel point P i to the first blood vessel edge can only be one of the first, second, third, and fourth line segments; after determining the selection range of the blood vessel diameter, take the length of the shortest line segment as the first blood vessel diameter d i corresponding to the center line pixel point P i ;

[0077] Step 54, according to the blood vessel linear change relationship from the center line pixel point P 1 to the center line pixel point P N , and the first blood vessel diameter d i of each center line pixel point P i , analyze the stenosis rate corresponding to each center line pixel point P i to generate the corresponding first stenosis rate r i;

[0078] Specifically, it includes: Step 541. According to the first blood vessel diameter d 1 and the first blood vessel diameter d N , construct a linear function f(i) that reflects the linear change relationship of the centerline pixel point P 1 to the centerline pixel point P N of the blood vessel, f(i) = d 1 + k * (i - 1), where k = (d N - d 1 ) / (N - 1);

[0079] Step 542. According to the linear change relationship function f(i), calculate the linear change diameter length corresponding to each centerline pixel point P i to generate the corresponding first reference diameter d ’ i ;

[0080] Here, in the case of no blood vessel stenosis mutation, there should be a certain linear relationship between the blood vessel diameter and its distance from the coronary artery entrance. For a branched blood vessel, there is also a certain linear relationship between the diameters at the entrance and exit positions; by confirming the linear relationship at the entrance and exit of a blood vessel segment, the normal diameter of any point on this blood vessel segment, that is, the first reference diameter d ’ i ;

[0081] Step 543. According to the first blood vessel diameter d i and the first reference diameter d ’ i , calculate the first stenosis rate r i corresponding to each centerline pixel point P i , r i = 1 - d i / d ’ i ;

[0082] Here, if there is a stenosis mutation at a certain position in a blood vessel segment, then after obtaining the stenotic diameter at this position, that is, the first blood vessel diameter d i , based on its corresponding first reference diameter d ’ i , the blood vessel stenosis rate at this position, that is, the first stenosis rate r i can be calculated;

[0083] Step 55. From all the obtained first stenosis rates r i , select the maximum value as the blood vessel stenosis rate corresponding to the stenosis segment blood vessel mask image within the current target detection frame;

[0084] Here, in the embodiments of the present invention, the maximum stenosis rate in a section of blood vessel is used as the blood vessel stenosis rate of the stenosis section blood vessel mask image in this section of blood vessel, that is, within the current target detection frame.

[0085] Step 56: Take the next unprocessed target detection frame as the current target detection frame, and go back to step 52 to continue the process until the blood vessel stenosis rates of the stenosis section blood vessel mask images within all target detection frames on the current preferred image are confirmed.

[0086] Through the above steps 1-5, multiple preferred images can be extracted from a section of angiography video, and the blood vessel stenosis rates of one or more stenosis section blood vessels on each preferred image can be obtained through coronary angiography quantitative analysis. After obtaining the analysis results of multiple preferred images, the multiple preferred images with target detection frames of stenosis section blood vessels and blood vessel stenosis rates can be provided to the doctor as parameter data at the same time; alternatively, the multiple preferred images can be image-fused, and the blood vessel stenosis rates of the stenosis section blood vessels at the same position can be averaged. Finally, a fused image with a target detection frame of stenosis section blood vessel and the average value of blood vessel stenosis rate is provided to the doctor as parameter data.

[0087] Figure 2 FIG. is a module structure diagram of a device for coronary angiography quantitative analysis based on angiography video provided by the second embodiment of the present invention. This device can be a terminal device or a server for implementing the method of the embodiments of the present invention, or a device for implementing the method of the embodiments of the present invention connected to the above terminal device or server. For example, this device can be a device or a chip system of the above terminal device or server. As Figure 2 shown, the device includes: an acquisition module 201, an image preprocessing module 202, a stenosis section blood vessel processing module 203, an image optimization module 204, and a quantitative analysis module 205.

[0088] The acquisition module 201 is used to acquire the angiography video of coronary angiography.

[0089] The image preprocessing module 202 is used to extract video frame images from the angiography video to generate a corresponding first image sequence.

[0090] The stenosis section blood vessel processing module 203 is used to perform target detection and semantic segmentation processing on the stenosis section blood vessels in each first image in the first image sequence based on a preset image target detection and semantic segmentation model, so as to obtain one or more target detection frames for marking the stenosis section blood vessels on each first image and a section of stenosis section blood vessel mask image in each target detection frame; each target detection frame corresponds to a detection frame confidence level.

[0091] The image optimization module 204 is configured to perform image optimization processing on the first image sequence according to the detection box confidence level to obtain a specified number of optimized images.

[0092] The quantitative analysis module 205 is configured to perform coronary angiography quantitative analysis on the narrow segment vascular mask images within each target detection box in each optimized image to generate corresponding vascular stenosis rates.

[0093] An apparatus for coronary angiography quantitative analysis based on an angiography video provided by an embodiment of the present invention can execute the method steps in the above method embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0094] It should be noted that it should be understood that the division of each module of the above apparatus is only a logical function division. In actual implementation, it can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the acquisition module can be a separately established processing element, or can be integrated in a certain chip of the above apparatus. In addition, it can also be stored in the memory of the above apparatus in the form of program code, and called and executed by a certain processing element of the above apparatus to perform the functions of the above determination module. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or independently implemented. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the integrated logic circuit in the processor element or the instruction in the form of software.

[0095] For example, the above modules can be one or more integrated circuits configured to implement the above method, such as: one or more Application Specific Integrated Circuits (ASICs), or, one or more Digital Signal Processors (DSPs), or, one or more Field Programmable Gate Arrays (FPGAs), etc. Again, when a certain module above is implemented in the form of a program code scheduled by a processing element, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processors that can call program code. Again, these modules can be integrated together and implemented in the form of a System-on-a-chip (SOC).

[0096] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The above computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.). The above computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The above available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0097] Figure 3 The following is a schematic structural diagram of an electronic device provided in Embodiment 3 of the present invention. The electronic device may be the aforementioned terminal device or server, or may be a terminal device or server connected to the aforementioned terminal device or server to implement the method of the embodiments of the present invention. As Figure 3 shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver operations of the transceiver 303. Various instructions may be stored in the memory 302 for completing various processing functions and implementing the methods and processes provided in the above embodiments of the present invention. Preferably, the electronic device related to the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to implement communication connections between components. The above communication port 306 is used for the electronic device to connect and communicate with other peripherals.

[0098] In Figure 3The system bus mentioned above can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The system bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure to represent it, but it does not mean that there is only one bus or one type of bus. The communication interface is used to implement the communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include Random Access Memory (RAM), and may also include non-volatile memory, such as at least one disk memory.

[0099] The above-mentioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0100] It should be noted that the embodiment of the present invention also provides a computer-readable storage medium. Instructions are stored in the storage medium. When it runs on a computer, it causes the computer to execute the methods and processing procedures provided in the above embodiments.

[0101] The embodiment of the present invention also provides a chip for running instructions. The chip is used to execute the methods and processing procedures provided in the above embodiments.

[0102] The embodiment of the present invention provides a method, device, electronic device, and computer-readable storage medium for quantitative coronary angiography analysis based on angiography videos. Video interception and video frame image extraction processing are performed on the angiography videos of coronary blood vessels. Target detection and semantic segmentation processing of the stenotic segment blood vessels are performed on the extracted image sequence based on an image target detection and semantic segmentation model. The extracted images are preferably selected based on the confidence of target recognition. Quantitative coronary angiography analysis is performed on each stenotic segment blood vessel on the preferably selected images to generate corresponding blood vessel stenosis rates. By means of the present invention, the excessive dependence on manual experience in traditional practices is eliminated, and the accuracy of image extraction and the calculation accuracy of stenosis rates are improved.

[0103] Those skilled in the art should further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0104] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the technical field.

[0105] The specific embodiments described above have further elaborated on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is only the specific embodiments of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for quantitative analysis of coronary angiography based on angiography video, characterized in that, the method includes: Obtain the angiography video of coronary angiography; Extract video frame images from the angiography video to generate a corresponding first image sequence; Based on a preset image object detection and semantic segmentation model, perform object detection and semantic segmentation processing on each first image in the first image sequence to obtain one or more object detection frames for marking the stenotic segment vessels and a segment of stenotic segment vessel mask image in each object detection frame on each first image; each object detection frame corresponds to a detection frame confidence level; According to the detection frame confidence level, perform image optimization processing on the first image sequence to obtain a specified number of optimized images; Perform quantitative analysis of coronary angiography on the stenotic segment vessel mask image in each object detection frame on each optimized image to generate a corresponding vessel stenosis rate; Among them, the performing quantitative analysis of coronary angiography on the stenotic segment vessel mask image in each object detection frame on each optimized image to generate a corresponding vessel stenosis rate specifically includes: Traverse each object detection frame on the current optimized image, and mark the currently traversed object detection frame as the current object detection frame; Identify the blood vessel edges and the blood vessel centerline of the stenotic segment blood vessel mask image within the current target detection box to generate corresponding first blood vessel edges and a first centerline; the first centerline includes a plurality of centerline pixel points P i , where the first centerline pixel point P 1 is the point closest to the coronary artery entrance in the blood flow direction, and the last centerline pixel point P N is the point farthest from the coronary artery entrance in the blood flow direction, 1 ≤ i ≤ N, and N is the total number of centerline pixel points of the first centerline; Analyze the vascular diameter lengths corresponding to each centerline pixel point P on the first centerline according to the first vascular edge to generate a corresponding first vascular diameter d i i ;​ According to the linear change relationship of blood vessels from the central line pixel point P 1 to the central line pixel point P N , and the first blood vessel diameter d i of each central line pixel point P i , analyze the stenosis rate corresponding to each central line pixel point P i to generate the corresponding first stenosis rate r i ; From all the obtained first stenosis rates r i select the maximum value as the vascular stenosis rate corresponding to the stenotic segment vascular mask image within the current target detection box; Analyzing the blood vessel diameter lengths corresponding to each center line pixel point P on the first center line according to the first blood vessel edge to generate a corresponding first blood vessel diameter d i Specifically, it includes: i Specifically, it includes: According to the center line pixel point P i and the directional relationship with its adjacent eight-region pixel points, draw four straight lines through the center line pixel point P i which are respectively denoted as the first, second, third, and fourth straight lines; the first straight line passes through the upper left adjacent pixel point of the center line pixel point P i , the center line pixel point P i and the lower right adjacent pixel point of the center line pixel point P i ; the second straight line passes through the upper adjacent pixel point of the center line pixel point P i , the center line pixel point P i and the lower adjacent pixel point of the center line pixel point P i ; the third straight line passes through the upper right adjacent pixel point of the center line pixel point P i , the center line pixel point P i and the lower left adjacent pixel point of the center line pixel point P i ; the third straight line passes through the right adjacent pixel point of the center line pixel point P i , the center line pixel point P i and the left adjacent pixel point of the center line pixel point P i ; Denote the line segments where the first, second, third, and fourth straight lines intersect the first blood vessel edge as the corresponding first, second, third, and fourth line segments respectively; calculate the lengths of the first, second, third, and fourth line segments to generate the corresponding first, second, third, and fourth line segment lengths; and select the minimum value from the first, second, third, and fourth line segment lengths as the first blood vessel diameter d i corresponding to the center line pixel point P i ; The vascular linear change relationship from the centerline pixel point P 1 to the centerline pixel point P N , and for each centerline pixel point P i of the first vascular diameter d i , analyze the stenosis rate corresponding to each centerline pixel point P i to generate a corresponding first stenosis rate r i , specifically including: According to the first blood vessel diameter d 1 and the first blood vessel diameter d N , construct a linear function f(i) that reflects the linear change relationship of the centerline pixel point P 1 to the centerline pixel point P N of the blood vessel, f(i) = d 1 + k * (i - 1), k = (d N - d 1 ) / (N - 1); According to the linear variation relationship function f(i), for each of the center line pixel points P i calculate the corresponding linearly varying diameter length to generate a corresponding first reference diameter d ’ i ; According to the first blood vessel diameter d i and the first reference diameter d ’ i , calculate the corresponding first stenosis rate r of each center line pixel point P i , where r i is i r = 1 - d i / d ’ i .

2. The method for quantitative analysis of coronary angiography based on angiography video according to claim 1, characterized in that, the extracting video frame images from the angiography video to generate a corresponding first image sequence specifically includes: Perform video interception on the angiography video, and retain the video content of the stage when the contrast agent fills the coronary artery to generate a corresponding intercepted angiography video; Extract video frame images from the intercepted angiography video in chronological order to generate a corresponding video frame image sequence, and count the number of video frame images in the video frame image sequence to generate a corresponding first total number; the video frame image sequence includes multiple video frame images; When the first total number does not exceed a preset image total number threshold, use each video frame image as the corresponding first image, and sort all the obtained first images in chronological order to generate the first image sequence; When the first total number exceeds the image total number threshold, extract the video frame images with all odd-index sorting indexes or all even-index sorting indexes from the video frame image sequence as the corresponding first images; and sort all the obtained first images in chronological order to generate the first image sequence.

3. The method for quantitative analysis of coronary angiography based on angiography video according to claim 1, characterized in that, the image object detection and semantic segmentation model includes a Mask R-CNN model; when the image object detection and semantic segmentation model is specifically a Mask R-CNN model, a Residual Network ResNet50 is used as its feature extraction backbone network.

4. The method for quantitative coronary angiography analysis based on angiography video according to claim 1, wherein, the image optimization process of the first image sequence according to the detection box confidence to obtain a specified number of optimized images specifically includes: in the first image sequence, calculating the mean value of the detection box confidence of all the target detection boxes on each of the first images to generate a corresponding first image average confidence; sorting all the first images in descending order according to the corresponding first image average confidence, and taking the specified number of the first images with the top ranking as the optimized images.

5. An apparatus for implementing the method steps of the quantitative coronary angiography analysis based on angiography video according to any one of claims 1-4, wherein, the apparatus includes: an acquisition module, an image preprocessing module, a narrow segment blood vessel processing module, an image optimization module, and a quantitative analysis module; the acquisition module is used to acquire the angiography video of coronary angiography; the image preprocessing module is used to extract video frame images from the angiography video to generate a corresponding first image sequence; the narrow segment blood vessel processing module is used to perform target detection and semantic segmentation processing on the narrow segment blood vessels of each of the first images in the first image sequence based on a preset image target detection and semantic segmentation model, so as to obtain one or more target detection boxes for marking the narrow segment blood vessels and a narrow segment blood vessel mask image in each of the target detection boxes on each of the first images; each of the target detection boxes corresponds to a detection box confidence; the image optimization module is used to perform image optimization processing on the first image sequence according to the detection box confidence to obtain a specified number of optimized images; the quantitative analysis module is used to perform quantitative coronary angiography analysis on the narrow segment blood vessel mask image in each of the target detection boxes on each of the optimized images to generate a corresponding blood vessel stenosis rate.

6. An electronic device, wherein, it includes: a memory, a processor, and a transceiver; the processor is used to be coupled with the memory, read and execute the instructions in the memory to implement the method according to any one of claims 1-4; the transceiver is coupled with the processor, and the processor controls the transceiver to perform message sending and receiving.

7. A computer-readable storage medium, wherein, the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a computer, the computer is made to execute the method according to any one of claims 1-4.

Images