Methods, devices, equipment and media for identifying blood vessel branches in angiography images

Abstract

This invention discloses a method, apparatus, device, and medium for recognizing blood vessel branches in angiography images, relating to the field of computer image technology. The method improves the system's recognition accuracy and robustness by inputting the original image and extracting high-dimensional branch-level features, establishing geometric and semantic dependencies between branches. This method can improve the accuracy of pixel-level segmentation and branch-level classification in angiography images such as coronary X-rays, enhancing the reliability of the segmentation results for diagnosis. It reduces the need for intermediate manual intervention to ensure system robustness and reduces reliance on manually designed features to ensure the system's generalization performance.

Description

Technical Field

[0001] This invention relates to the field of computer image technology, and in particular to a method, apparatus, device, and medium for identifying vascular branches in angiographic images based solely on images and without human intervention. Background Technology

[0002] Identification of vascular branches in coronary angiography images plays a crucial role in the diagnosis of coronary vascular diseases. For example, the SYNTAX score quantifies lesions based on coronary angiography images and serves as a standard for selecting revascularization strategies. Therefore, rapid and automated multi-class structure identification is of great importance for the auxiliary diagnosis of vascular diseases.

[0003] In clinical practice, the structural identification of vascular branches is often done manually. This operation is time-consuming and easily affected by individual subjective factors. However, the automatic identification and marking of coronary vascular branches can improve the efficiency and robustness of quantitative analysis scoring, enabling rapid and accurate selection of revascularization strategies.

[0004] Recently, deep learning has made significant progress in medical image processing and analysis, opening up new possibilities for automated techniques in coronary artery branch identification. These methods treat vascular branch structure identification as a multi-stage process, including segmentation, key point extraction, graphical model construction, and finally, classification of vascular branches based on hand-designed features.

[0005] These methods decouple the vascular branch identification task sequentially, but still have the following shortcomings:

[0006] 1. The cumulative error caused by previous tasks such as topological completeness of segmentation results and skeleton extraction greatly affects the accuracy of subsequent tasks;

[0007] 2. The sensitivity of hand-designed features affects the accuracy of classification results;

[0008] 3. The above two points led to manual intervention to modify intermediate results. Summary of the Invention

[0009] In view of the above problems, the present invention provides a method, apparatus, device and medium for recognizing vascular branches in angiography images to overcome the above problems or at least partially solve the above problems.

[0010] This invention provides the following solution:

[0011] A method for identifying blood vessel branches in angiography images, comprising:

[0012] Multi-level features are obtained by extracting multi-level features from the acquired single-frame X-ray imaging image using an image encoder;

[0013] The multi-level features are optimized and improved using a small side branch feature supplementation module to obtain pixel-level multi-level features;

[0014] Determine whether to enter a multi-stage decoding operation; if a non-terminating iteration is determined, use the initialized features or the previously optimized branch query features as input for decoding;

[0015] The branch-level query features of each instance object are decoded from the pixel-level multi-level features using the branch feature decoder.

[0016] The branch dependency feature encoder extracts geometric and semantic features at the branch object level from the pixel-level multi-level features and the branch-level query features; and uses the geometric and semantic features at the branch object level to calculate the dependencies between branches to obtain a branch dependency graph.

[0017] When the iteration ends, the branch classifier is used to classify the branch dependency graph to obtain pixel-level segmentation results and branch-level classification results.

[0018] Preferably, the fine collateral feature supplementation module takes features of different sizes as input, upsamples the low-resolution feature map, and subtracts the intersection of the low-resolution feature map and the high-resolution feature map to obtain the difference feature map between the low-resolution feature map and the high-resolution feature map; the difference feature map is then passed through the spatiotemporal attention module and summed with the low-resolution feature map to obtain the pixel-level multi-level features, which are used to supplement the missing information of fine collaterals on the low resolution.

[0019] Preferably, the branch feature decoder uses a fully connected layer to obtain initial branch-level object query features after global feature interaction of the pixel-level multi-level features, and performs cross-attention with the pixel-level multi-level features to obtain the interacted branch-level object query features.

[0020] Preferably, the branch dependency feature encoder obtains branch pixel distribution information by multiplying the pixel-level multi-level features and the branch-level query features, and then obtains the branch-level geometric and semantic features through branch-level geometric and semantic feature extraction.

[0021] A device for recognizing blood vessel branches in angiography images, characterized in that it is used to perform the above-described method for recognizing blood vessel branches in angiography images, the device comprising:

[0022] The multi-level feature acquisition unit is used to extract multi-level features from the acquired single-frame X-ray imaging image using an image encoder.

[0023] A pixel-level multi-level feature optimization unit is used to optimize and improve the multi-level features using a fine side branch feature supplementation module to obtain pixel-level multi-level features.

[0024] The decision unit is used to determine whether to enter a multi-stage decoding operation; it determines non-terminating iterations and uses the initialized features or the previously optimized branch query features as input for decoding.

[0025] A branch feature decoding unit is used to decode the branch-level query features of each instance object from the pixel-level multi-level features using a branch feature decoder.

[0026] The branch dependency graph acquisition unit is used to extract geometric and semantic features at the branch object level from the pixel-level multi-level features and the branch-level query features using the branch dependency feature encoder; and to calculate the dependencies between branches using the geometric and semantic features at the branch object level to obtain the branch dependency graph.

[0027] The detection result acquisition unit is used to determine when the iteration ends, and then use a branch classifier to classify the branch dependency graph to obtain pixel-level segmentation results and branch-level classification results.

[0028] A device for recognizing blood vessel branches in angiographic images, the device comprising a processor and a memory:

[0029] The memory is used to store program code and transmit the program code to the processor;

[0030] The processor is used to execute the above-described angiography image vascular branch recognition method according to the instructions in the program code.

[0031] A computer-readable storage medium for storing program code for executing the above-described method for identifying blood vessel branches in angiographic images.

[0032] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0033] This application provides a method, apparatus, device, and medium for identifying blood vessel branches in angiography images. The method improves the system's recognition accuracy and robustness by inputting the original image and extracting high-dimensional branch-level features, establishing geometric and semantic dependencies between branches. This method improves the accuracy of pixel-level segmentation and branch-level classification in coronary X-ray and other angiography images, enhancing the reliability of the segmentation results for diagnosis. It reduces the need for intermediate manual intervention to ensure system robustness and reduces reliance on manually designed features to ensure the system's generalization performance.

[0034] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0036] Figure 1 This is a flowchart of the method for recognizing blood vessel branches in angiography images provided in an embodiment of the present invention;

[0037] Figure 2 This is a system framework diagram for identifying branch structures in coronary angiography images provided in an embodiment of the present invention;

[0038] Figure 3 This is a structural diagram of the small feature supplementation module provided in an embodiment of the present invention;

[0039] Figure 4 This is a structural diagram of the branch feature decoder provided in an embodiment of the present invention;

[0040] Figure 5 This is a structural diagram of the branch-dependent feature encoder provided in an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of the angiography image vascular branch recognition device provided in an embodiment of the present invention;

[0042] Figure 7 This is a schematic diagram of the angiography image vascular branch recognition device provided in an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0044] See Figure 1 This invention provides a method for identifying blood vessel branches in angiography images, as shown in the embodiments of the present invention. Figure 1 As shown, the method may include:

[0045] S101: Multi-level features are obtained by extracting multi-level features from the acquired single-frame X-ray imaging image using an image encoder.

[0046] S102: The fine side branch feature supplementation module is used to optimize and improve the multi-level features to obtain pixel-level multi-level features. In specific implementation, the embodiment of this application can provide that the fine side branch feature supplementation module takes features of different sizes as input, upsamples the low-resolution feature map, and subtracts the intersection with the high-resolution feature map to obtain the difference feature map between the low-resolution feature map and the high-resolution feature map; the difference feature map is passed through the spatiotemporal attention module and then summed with the low-resolution feature map to obtain the pixel-level multi-level features, which are used to supplement the missing information of fine side branches on the low resolution.

[0047] S103: Determine whether to enter a multi-stage decoding operation; if it is a non-terminating iteration, use the initialized features or the previously optimized branch query features as input for decoding;

[0048] S104: The branch feature decoder decodes the branch-level query features of each instance object from the pixel-level multi-level features. In specific implementation, the embodiments of this application can provide that the branch feature decoder uses a fully connected layer to obtain the initial branch-level object query features after the pixel-level multi-level features are interacted with global features, and performs cross-attention with the pixel-level multi-level features to obtain the interacted branch-level object query features.

[0049] S105: The branch dependency feature encoder extracts the geometric and semantic features at the branch object level from the pixel-level multi-level features and the branch-level query features; and calculates the dependencies between branches using the geometric and semantic features at the branch object level to obtain a branch dependency graph; in specific implementation, the embodiments of this application can provide that the branch dependency feature encoder obtains the branch pixel distribution information by multiplying the pixel-level multi-level features and the branch-level query features, and obtains the geometric and semantic features at the branch level through branch-level geometric and semantic feature extraction.

[0050] S106: When the iteration ends, the branch classifier is used to classify the branch dependency graph to obtain pixel-level segmentation results and branch-level classification results.

[0051] The structure of the method provided in this application mainly includes an image feature encoder, a fine side branch feature supplementation module, a branch feature decoder, a branch dependency encoder, and a branch classifier.

[0052] The image feature encoder is responsible for extracting multi-scale and multi-level features of the image; the fine collateral feature supplementation module prevents collateral features from being submerged in the low-resolution feature map; the branch feature decoder is responsible for decoding pixel-level features and initializing branch-level query features for each blood vessel object; the branch dependency feature encoder extracts the pixel-level multi-level features and branch-level query features from the image feature decoder into geometric and semantic features at the branch object level, then calculates the dependencies between branches to enhance the branch-level query features, and finally sends them to the classifier for category output.

[0053] The specific workflow is as follows:

[0054] In the image encoder section, a single frame of X-ray angiography image is used as input to the feature extractor to obtain multi-level features of the image. These features are then input to the fine collateral feature supplementation module to obtain optimized multi-level features. The branch feature decoder and branch dependency encoder module undergo a multi-stage tuning process. First, within an iteration, the branch feature decoder decodes the input features to obtain branch-level query features. Then, the branch dependency encoder, in the same iteration, extracts geometric and semantic dependency features and calculates inter-branch dependencies. This process is repeated to enhance the branch-level query features to obtain optimized features, which are finally fed into the classifier for category determination and pixel feature decoding.

[0055] like Figure 2 As shown, the overall processing flow mainly includes the following steps:

[0056] Step 101: The input image is processed by an image encoder to extract multi-level features. In the image encoder section, a single-frame X-ray imaging image is used as input to the feature extractor to obtain multi-level features of the image.

[0057] Step 102: The multi-level features are supplemented by the fine side branch feature supplementation module to obtain optimized and improved pixel-level multi-level features.

[0058] Step 103: Determine whether to proceed with the multi-stage decoding operation. If it is not a termination iteration, use the initialized features or the previously optimized branch query features as input for decoding.

[0059] Step 104, Image Feature Decoding. Pixel-level multi-layer features are used to decode the branch-level query features for each instance object.

[0060] Step 105, Branch Dependency Calculation. Extract the semantic and geometric features of each branch, calculate the dependencies between branches, and thereby enhance the branch-level query features of each object.

[0061] Step 106: When the iteration ends, the input classifier obtains the final pixel-level segmentation result and branch-level classification result.

[0062] like Figure 3 As shown, Figure 3 The result of the module supplementing the small collateral features is shown in the figure. It takes features of different sizes as input, upsamples the low-resolution feature map, then intersects it with the high-resolution feature map and subtracts it to obtain the difference feature map. After passing through the spatiotemporal attention module, it is summed with the low-resolution feature map to supplement the missing information of small collaterals on the low resolution.

[0063] like Figure 4 As shown, Figure 4 This is the structure diagram of the branch feature decoder. The input features, after global feature interaction, are processed by a fully connected layer to obtain initial branch-level object query features. These features are then cross-attentioned with the pixel-level multi-level features of the encoder to obtain the interacted branch-level object query features.

[0064] like Figure 5 As shown, Figure 5 This is the structure diagram of the branch dependency encoder. The module input consists of branch-level object query features and pixel-level encoder features. The input obtains branch pixel distribution information through a dot product operation, and then extracts branch-level geometric and semantic features. The extracted features construct geometric and semantic dependency graph structures between branches, forming a branch dependency graph. This graph structure is used to create a graph network to propagate and aggregate dependencies, and optimize the branch-level object query features.

[0065] In summary, the angiography image vascular branch recognition method provided in this application improves the system's recognition accuracy and robustness by inputting the original image and extracting high-dimensional branch-level features, establishing geometric and semantic dependencies between branches. This method improves the accuracy of pixel-level segmentation and branch-level classification in angiography images such as coronary X-rays, enhancing the reliability of the segmentation results for diagnosis. It reduces the need for intermediate manual intervention to ensure system robustness and reduces reliance on manually designed features to ensure the system's generalization performance.

[0066] See Figure 6 This application embodiment can also provide a device for recognizing blood vessel branches in angiographic images, such as... Figure 6 As shown, the device may include:

[0067] The multi-level feature acquisition unit 601 is used to extract multi-level features from the acquired single-frame X-ray imaging image using an image encoder to obtain multi-level features.

[0068] The pixel-level multi-level feature optimization unit 602 is used to optimize and improve the multi-level features using the fine side branch feature supplementation module to obtain pixel-level multi-level features.

[0069] The judgment unit 603 is used to determine whether to enter a multi-stage decoding operation; if a non-terminating iteration is determined, the initialized features or the previously optimized branch query features are used as input for decoding;

[0070] Branch feature decoding unit 604 is used to decode the branch-level query features of each instance object from the pixel-level multi-level features using the branch feature decoder;

[0071] The branch dependency graph acquisition unit 605 is used to extract geometric and semantic features at the branch object level from the pixel-level multi-level features and the branch-level query features using the branch dependency feature encoder; and to calculate the dependencies between branches using the geometric and semantic features at the branch object level to obtain the branch dependency graph.

[0072] The detection result acquisition unit 606 is used to determine when the iteration ends, and then use a branch classifier to classify the branch dependency graph to obtain pixel-level segmentation results and branch-level classification results.

[0073] This application embodiment can also provide a device for recognizing blood vessel branches in angiographic images, the device including a processor and a memory:

[0074] The memory is used to store program code and transmit the program code to the processor;

[0075] The processor is used to execute the steps of the angiography image vascular branch recognition method described above according to the instructions in the program code.

[0076] like Figure 7 As shown in the embodiment of this application, a vascular branch recognition device for angiography images may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0077] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0078] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiment of the angiography image vascular branch recognition method.

[0079] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0080] Multi-level features are obtained by extracting multi-level features from the acquired single-frame X-ray imaging image using an image encoder;

[0081] The multi-level features are optimized and improved using a small side branch feature supplementation module to obtain pixel-level multi-level features;

[0082] Determine whether to enter a multi-stage decoding operation; if a non-terminating iteration is determined, use the initialized features or the previously optimized branch query features as input for decoding;

[0083] The branch-level query features of each instance object are decoded from the pixel-level multi-level features using the branch feature decoder.

[0084] The branch dependency feature encoder extracts geometric and semantic features at the branch object level from the pixel-level multi-level features and the branch-level query features; and uses the geometric and semantic features at the branch object level to calculate the dependencies between branches to obtain a branch dependency graph.

[0085] When the iteration ends, the branch classifier is used to classify the branch dependency graph to obtain pixel-level segmentation results and branch-level classification results.

[0086] In one possible implementation, the memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function (such as file creation or data read / write). The data storage area may store data created during use, such as initialization data.

[0087] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0088] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0089] Of course, it should be noted that, Figure 7 The structure shown does not constitute a limitation on the angiography image vascular branch recognition device in the embodiments of this application. In practical applications, the angiography image vascular branch recognition device may include more than Figure 7 More or fewer components as shown, or combinations of certain components.

[0090] This application embodiment may also provide a computer-readable storage medium for storing program code for executing the steps of the above-described angiography image vascular branch recognition method.

[0091] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0092] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0093] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0094] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for identifying blood vessel branches in angiographic images, characterized in that, include: Multi-level features are obtained by extracting multi-level features from the acquired single-frame X-ray imaging image using an image encoder; The multi-level features are optimized and improved using a small side branch feature supplementation module to obtain pixel-level multi-level features; Determine whether to proceed with a multi-stage decoding operation; For non-terminating iterations, use the initialized features or previously optimized branch query features as input for decoding. The branch-level query features of each instance object are decoded from the pixel-level multi-level features using the branch feature decoder. The branch dependency feature encoder extracts geometric and semantic features at the branch object level from the pixel-level multi-level features and the branch-level query features; and uses the geometric and semantic features at the branch object level to calculate the dependencies between branches to obtain a branch dependency graph. When the iteration ends, the branch classifier is used to classify the branch dependency graph to obtain pixel-level segmentation results and branch-level classification results.

2. The method for identifying blood vessel branches in angiography images according to claim 1, characterized in that, The fine side branch feature supplementation module takes features of different sizes as input, upsamples the low-resolution feature map, and subtracts the intersection of the low-resolution feature map and the high-resolution feature map to obtain the difference feature map between the low-resolution feature map and the high-resolution feature map. The differential feature map is passed through a spatiotemporal attention module and then summed with the low-resolution feature map to obtain the pixel-level multi-level features, which are used to supplement the missing information of small side branches on the low resolution.

3. The method for identifying blood vessel branches in angiography images according to claim 1, characterized in that, The branch feature decoder uses a fully connected layer to obtain initial branch-level object query features after global feature interaction with the pixel-level multi-level features, and performs cross-attention with the pixel-level multi-level features to obtain the interacted branch-level object query features.

4. The method for identifying blood vessel branches in angiography images according to claim 1, characterized in that, The branch dependency feature encoder obtains branch pixel distribution information by multiplying the pixel-level multi-level features and the branch-level query features, and then obtains the branch-level geometric and semantic features through branch-level geometric and semantic feature extraction.

5. A device for recognizing blood vessel branches in angiographic images, characterized in that, The apparatus for performing the angiography image vascular branch recognition method according to any one of claims 1-4, the apparatus comprising: The multi-level feature acquisition unit is used to extract multi-level features from the acquired single-frame X-ray imaging image using an image encoder. A pixel-level multi-level feature optimization unit is used to optimize and improve the multi-level features using a fine side branch feature supplementation module to obtain pixel-level multi-level features. The decision unit is used to determine whether to enter a multi-stage decoding operation; it determines non-terminating iterations and uses the initialized features or the previously optimized branch query features as input for decoding. A branch feature decoding unit is used to decode the branch-level query features of each instance object from the pixel-level multi-level features using a branch feature decoder. The branch dependency graph acquisition unit is used to extract geometric and semantic features at the branch object level from the pixel-level multi-level features and the branch-level query features using the branch dependency feature encoder; and to calculate the dependencies between branches using the geometric and semantic features at the branch object level to obtain the branch dependency graph. The detection result acquisition unit is used to determine when the iteration ends, and then use a branch classifier to classify the branch dependency graph to obtain pixel-level segmentation results and branch-level classification results.

6. A device for recognizing blood vessel branches in angiographic images, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the angiography image vascular branch recognition method according to any one of claims 1-4 according to the instructions in the program code.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the angiography image vascular branch recognition method according to any one of claims 1-4.

Images