Contrast image blood vessel branch identification method, device, equipment and medium

Abstract

The invention discloses a angiography image blood vessel branch identification method, device and equipment and a medium, and relates to the technical field of computer images, and according to the method, the identification accuracy and robustness of a system are improved by inputting an original image and extracting high-dimensional branch-level features and establishing geometric structure dependence and semantic dependence information between branches. The method can improve the accuracy of the branch pixel-level segmentation result and the branch-level classification in the coronary X-ray contrast image and the like, and improves the reliability of the segmentation result on diagnosis. The intervention of intermediate manual operation is reduced to ensure the robustness of the system; and meanwhile, the dependence on manual design characteristics is reduced, so that the generalization performance of the system is ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer image technology, and in particular to a method, device, equipment and medium for identifying blood vessel branches in angiography images based only on images and without manual intervention. Background Art

[0002] The recognition of vascular branches in coronary X-ray angiography images plays an important role in the diagnosis of coronary artery-related diseases. For example, the SYNTAX score quantitatively analyzes lesions based on coronary X-ray angiography images and serves as a criterion for selecting revascularization strategies. Therefore, fast and automatic multi-category structure recognition plays an important role in the auxiliary diagnosis of vascular-related diseases.

[0003] In clinical practice, structural identification of vascular branches is often performed manually, which is time-consuming and easily affected by individual subjective factors. The automatic identification and labeling of coronary vascular branches can improve the efficiency of quantitative analysis and scoring and the robustness of the results, and achieve rapid and accurate selection of revascularization strategies.

[0004] Recently, deep learning has made considerable progress in medical image processing and analysis, opening up new possibilities for automated techniques for coronary artery branch recognition. These methods consider vascular branch structure recognition as a multi-stage process, including segmentation, key point extraction, graph model construction, and finally rely on manually designed features to classify vascular branches.

[0005] These methods decouple the vascular branch recognition task sequentially, but still have the following defects:

[0006] 1. The accumulated errors caused by previous tasks such as the topological integrity of the segmentation results and skeleton extraction greatly affect the accuracy of subsequent tasks;

[0007] 2. The sensitivity of manually designed features affects the accuracy of classification results;

[0008] 3. The above two points lead to manual intervention to modify the intermediate results. Summary of the invention

[0009] In view of the above problems, the present invention provides a method, device, equipment and medium for identifying blood vessel branches in angiography images for overcoming the above problems or at least partially solving the above problems.

[0010] The present invention provides the following scheme:

[0011] A method for identifying blood vessel branches in angiography images, comprising:

[0012] The image encoder is used to extract multi-level features from the acquired single-frame X-ray angiography image to obtain multi-level features;

[0013] The multi-level features are optimized and improved by using a small side branch feature supplementation module to obtain pixel-level multi-level features;

[0014] Determine whether to enter a multi-stage decoding operation; determine non-termination iterations, and use the initialized features or the previously optimized branch query features as input for decoding;

[0015] Decoding the branch-level query features of each instance object from the pixel-level multi-level features using a branch feature decoder;

[0016] A branch dependency feature encoder is used to extract geometric features and semantic features at the branch object level from the pixel level features and the branch level query features; and the geometric features and semantic features at the branch object level are used to calculate the dependencies between branches to obtain a branch dependency relationship graph;

[0017] When it is determined that the iteration is finished, the branch dependency graph is classified using a branch classifier to obtain a pixel-level segmentation result and a branch-level classification result.

[0018] Preferably: the small side branch feature supplementation module takes features of different sizes as input, upsamples the low-resolution feature map, and subtracts the intersection with the high-resolution feature map to obtain a differential feature map between the low-resolution feature map and the high-resolution feature map; the differential feature map is added to the low-resolution feature map after passing through the spatiotemporal attention module to obtain the pixel-level multi-level feature, which is used to supplement the missing information of small side branches at low resolution.

[0019] Preferably: the branch feature decoder uses a fully connected layer to obtain an initial branch-level object query feature after the pixel-level multi-level feature interacts with the global feature, and performs cross-attention with the pixel-level feature to obtain the branch-level object query feature after interaction.

[0020] Preferably: the branch-dependent feature encoder obtains branch pixel distribution information by performing a dot product operation on the pixel-level feature and the branch-level query feature, and obtains the branch-level geometric feature and semantic feature by extracting the branch-level geometric and semantic features.

[0021] A medical image key point detection device, characterized in that it is used to perform the above-mentioned medical image key point detection method, and the device comprises:

[0022] A multi-level feature acquisition unit, used for extracting multi-level features from the acquired single-frame X-ray angiography image using an image encoder to obtain multi-level features;

[0023] A pixel-level multi-level feature optimization unit, used to optimize and improve the multi-level features using a small side branch feature supplementation module to obtain pixel-level multi-level features;

[0024] A judgment unit is used to judge whether to enter a multi-stage decoding operation; determine a non-termination iteration, and use the initialized features or the previously optimized branch query features as input for decoding;

[0025] A branch feature decoding unit, configured to decode a branch-level query feature of each instance object from the pixel-level multi-level feature using a branch feature decoder;

[0026] A branch dependency graph acquisition unit is used to extract geometric features and semantic features at the branch object level from the pixel level features and the branch level query features using a branch dependency feature encoder; and to calculate dependencies between branches using the geometric features and semantic features at the branch object level to obtain a branch dependency graph;

[0027] The detection result acquisition unit is used to determine when the iteration is ended and to classify the branch dependency graph using a branch classifier to obtain a pixel-level segmentation result and a branch-level classification result.

[0028] A medical image key point detection device, the device comprising a processor and a memory:

[0029] The memory is used to store program codes and transmit the program codes to the processor;

[0030] The processor is used to execute the above-mentioned medical image key point detection method according to the instructions in the program code.

[0031] A computer-readable storage medium is used to store program codes, and the program codes are used to execute the above-mentioned medical image key point detection method.

[0032] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0033] The embodiment of the present application provides a method, device, equipment and medium for identifying blood vessel branches in angiography images. The method inputs the original image and extracts high-dimensional branch-level features to establish geometric structure dependency and semantic dependency information between branches to improve the recognition accuracy and robustness of the system. The method can improve the accuracy of branch pixel-level segmentation results and branch-level classification in angiography images such as coronary artery X-rays, and improve the reliability of segmentation results for diagnosis. The intervention of intermediate manual operations is reduced to ensure the robustness of the system; at the same time, the dependence on manually designed features is reduced to ensure the generalization performance of the system.

[0034] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 is a flow chart of a method for identifying blood vessel branches in angiography images provided by an embodiment of the present invention;

[0037] Figure 2 This is a system framework diagram for identifying branch structures of coronary X-ray angiography images provided by an embodiment of the present invention;

[0038] Figure 3 is a structural diagram of a small feature supplement module provided by an embodiment of the present invention;

[0039] Figure 4 is a structural diagram of a branch feature decoder provided by an embodiment of the present invention;

[0040] Figure 5 is a structural diagram of a branch dependency feature encoder provided by an embodiment of the present invention;

[0041] Figure 6 is a schematic diagram of a device for identifying blood vessel branches in angiography images provided by an embodiment of the present invention;

[0042] Figure 7 It is a schematic diagram of a device for identifying blood vessel branches in angiography images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0044] See also Figure 1 , is a method for identifying blood vessel branches in angiography images provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0045] S101: extracting multi-level features from the acquired single-frame X-ray angiography image using an image encoder to obtain multi-level features;

[0046] S102: Use a small side branch feature supplementation module to optimize and improve the multi-level features to obtain pixel-level multi-level features; in specific implementation, the embodiment of the present application can provide the small side branch feature supplementation module to take features of different sizes as input, upsample the low-resolution feature map, and intersect with the high-resolution feature map and then subtract them to obtain a differential feature map between the low-resolution feature map and the high-resolution feature map; add the differential feature map to the low-resolution feature map after passing through the spatiotemporal attention module to obtain the pixel-level multi-level features, which are used to supplement the missing information of small side branches at low resolution.

[0047] S103: Determine whether to enter a multi-stage decoding operation; determine a non-termination iteration, and use the initialized features or the previously optimized branch query features as input for decoding;

[0048] S104: Use a branch feature decoder to decode the branch-level query features of each instance object from the pixel-level multi-level features; in specific implementation, the embodiment of the present application can provide that the branch feature decoder uses a fully connected layer to obtain the initial branch-level object query features after the pixel-level multi-level features are interacted with the global features, and cross-attention is performed with the pixel-level features to obtain the branch-level object query features after interaction.

[0049] S105: Utilize a branch dependency feature encoder to extract geometric features and semantic features at the branch object level from the pixel level features and the branch level query features; and utilize the geometric features and semantic features at the branch object level to calculate dependencies between branches to obtain a branch dependency relationship graph; in specific implementation, the embodiment of the present application may provide that the branch dependency feature encoder obtains branch pixel distribution information by performing a dot product operation on the pixel level features and the branch level query features, and obtains the geometric features and semantic features at the branch level through branch level geometric and semantic feature extraction.

[0050] S106: When it is determined that the iteration is finished, the branch dependency graph is classified using a branch classifier to obtain a pixel-level segmentation result and a branch-level classification result.

[0051] The structure of the method provided in the embodiment of the present application mainly includes an image feature encoder, a small side branch feature supplementation module, a branch feature decoder, a branch dependency encoder and a branch classifier.

[0052] Among them, the image feature encoder is responsible for extracting multi-scale and multi-level features of the image; the small side branch feature supplementation module prevents the side branch features from being submerged in the low-resolution feature map; the branch feature decoder is responsible for decoding pixel-level features and initializing the branch-level query features of each vascular object; the branch dependency feature encoder extracts the pixel-level features and branch-level query features of the image feature decoder into geometric and semantic features at the branch object level, and then calculates the dependencies between branches to enhance the branch-level query features, which are finally sent to the classifier for category output.

[0053] The specific workflow is as follows:

[0054] In the image encoder part, a single frame of X-ray angiography is used as input into the feature extractor to obtain the multi-level features of the image, and then input into the small side branch feature supplementation module to obtain the optimized multi-level features. The branch feature decoder and branch dependency encoder module are a multi-stage tuning process. First, in an iterative stage, the branch feature decoder decodes the input features to obtain the query features at the branch level, and then the branch dependency encoder in the same iterative stage extracts the geometric and semantic dependency features and calculates the dependencies between branches. Repeat the above stages, enhance the query features at the branch level to obtain the optimized features, and finally send them to the classifier for category judgment and pixel feature decoding.

[0055] like Figure 2 As shown, the overall processing flow mainly includes the following steps:

[0056] Step 101: The input image passes through the image encoder to extract multi-level features. In the image encoder part, a single frame of X-ray angiography image is used as input to enter the feature extractor to obtain multi-level features of the image.

[0057] Step 102, the multi-level features are supplemented by a small side branch feature module to obtain optimized and improved pixel-level multi-level features.

[0058] Step 103, determine whether to enter a multi-stage decoding operation. If the iteration is not terminated, use the initialized features or the previously optimized branch query features as input for decoding.

[0059] Step 104: Image feature decoding: The pixel-level features are used to decode the branch-level query features of each instance object.

[0060] Step 105: Branch dependency calculation: Extract the semantic and geometric features of each branch, and calculate the dependencies between branches, so as to enhance the branch-level query features of each object.

[0061] Step 106, when it is determined that the iteration is finished, the classifier is input to obtain the final pixel-level segmentation result and branch-level classification result.

[0062] like Figure 3 As shown, Figure 3 This is the result of the fine side branch feature supplementation module. It takes features of different sizes as input, upsamples the low-resolution feature map, and then intersects and subtracts the high-resolution feature map to obtain the differential feature map of the two. After passing through the spatiotemporal attention module, it is summed with the low-resolution feature map to supplement the missing information of the fine side branches at low resolution.

[0063] like Figure 4 As shown, Figure 4 The structure diagram of the branch feature decoder. After the input feature interacts with the global feature, the fully connected layer is used to obtain the initial branch-level object query feature, and cross-attention is performed with the pixel-level feature of the encoder to obtain the branch-level object query feature after interaction.

[0064] like Figure 5 As shown, Figure 5 The diagram is a diagram of the branch dependency encoder structure. The module input is the branch-level object query feature and the pixel-level encoder feature. The input obtains the branch pixel distribution information through the dot multiplication operation, and the branch-level geometric and semantic features are extracted to obtain the branch-level geometric features and semantic features. The extracted features construct the geometric dependency and semantic dependency graph structures between the branches, respectively, to form a branch dependency graph. Through this graph structure, the graph network is created to transfer and aggregate dependencies, and the branch-level object query features are optimized.

[0065] In summary, the present application provides a method for identifying vascular branches in contrast images. This method improves the recognition accuracy and robustness of the system by inputting the original image and extracting high-dimensional branch-level features, establishing geometric structure dependency and semantic dependency information between branches. This method can improve the accuracy of branch pixel-level segmentation results and branch-level classification in contrast images such as coronary artery X-rays, and improve the reliability of segmentation results for diagnosis. It reduces the intervention of intermediate manual operations to ensure the robustness of the system; at the same time, it reduces the reliance on manually designed features to ensure the generalization performance of the system.

[0066] See also Figure 6 , the embodiment of the present application can also provide a medical image key point detection device, such as Figure 6 As shown, the device may include:

[0067] The multi-level feature acquisition unit 601 is used to extract multi-level features from the acquired single-frame X-ray angiography image using an image encoder to obtain multi-level features;

[0068] A pixel-level multi-level feature optimization unit 602 is used to optimize and improve the multi-level features using a small side branch feature supplementation module to obtain a pixel-level multi-level feature;

[0069] A judgment unit 603 is used to judge whether to enter a multi-stage decoding operation; determine a non-termination iteration, and use the initialized feature or the previously optimized branch query feature as input for decoding;

[0070] A branch feature decoding unit 604 is used to decode the branch level query feature of each instance object from the pixel level multi-level feature using a branch feature decoder;

[0071] A branch dependency graph acquisition unit 605 is used to extract geometric features and semantic features at the branch object level from the pixel level features and the branch level query features using a branch dependency feature encoder; and calculate the dependencies between branches using the geometric features and semantic features at the branch object level to obtain a branch dependency graph;

[0072] The detection result acquisition unit 606 is used to use a branch classifier to classify the branch dependency graph to obtain a pixel-level segmentation result and a branch-level classification result when determining to end the iteration.

[0073] The present application may also provide a medical image key point detection device, the device comprising a processor and a memory:

[0074] The memory is used to store program codes and transmit the program codes to the processor;

[0075] The processor is used to execute the steps of the above-mentioned medical image key point detection method according to the instructions in the program code.

[0076] like Figure 7 As shown, a medical image key point detection device provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 all communicate with each other through the communication bus 13.

[0077] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0078] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the medical image key point detection method.

[0079] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0080] The image encoder is used to extract multi-level features from the acquired single-frame X-ray angiography image to obtain multi-level features;

[0081] The multi-level features are optimized and improved by using a small side branch feature supplementation module to obtain pixel-level multi-level features;

[0082] Determine whether to enter a multi-stage decoding operation; determine non-termination iterations, and use the initialized features or the previously optimized branch query features as input for decoding;

[0083] Decoding the branch-level query features of each instance object from the pixel-level multi-level features using a branch feature decoder;

[0084] A branch dependency feature encoder is used to extract geometric features and semantic features at the branch object level from the pixel level features and the branch level query features; and the geometric features and semantic features at the branch object level are used to calculate the dependencies between branches to obtain a branch dependency relationship graph;

[0085] When it is determined that the iteration is finished, the branch dependency graph is classified using a branch classifier to obtain a pixel-level segmentation result and a branch-level classification result.

[0086] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0087] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0088] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0089] Of course, it should be noted that Figure 7 The structure shown does not constitute a limitation on the medical image key point detection device in the embodiment of the present application. In actual applications, the medical image key point detection device may include Figure 7 More or fewer components than shown, or combinations of certain components.

[0090] The embodiment of the present application may also provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the steps of the above-mentioned medical image key point detection method.

[0091] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0092] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0093] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for identifying blood vessel branches in angiography images, characterized in that: include: The image encoder is used to extract multi-level features from the acquired single-frame X-ray angiography image to obtain multi-level features; The multi-level features are optimized and improved by using a small side branch feature supplementation module to obtain pixel-level multi-level features; Determine whether to enter a multi-stage decoding operation; Determine non-terminating iterations and use the initialized features or previously optimized branch query features as input for decoding; Decoding the branch-level query features of each instance object from the pixel-level multi-level features using a branch feature decoder; A branch dependency feature encoder is used to extract geometric features and semantic features at the branch object level from the pixel level features and the branch level query features; and the geometric features and semantic features at the branch object level are used to calculate the dependencies between branches to obtain a branch dependency relationship graph; When it is determined that the iteration is finished, the branch dependency graph is classified using a branch classifier to obtain a pixel-level segmentation result and a branch-level classification result.

2. The method for identifying blood vessel branches in angiography images according to claim 1, characterized in that: The small side branch feature supplementation module takes features of different sizes as input, upsamples the low-resolution feature map, and subtracts the intersection with the high-resolution feature map to obtain a differential feature map between the low-resolution feature map and the high-resolution feature map; The differential feature map is added to the low-resolution feature map after passing through the spatiotemporal attention module to obtain the pixel-level multi-level feature to supplement the missing information of small side branches at low resolution.

3. The method for identifying blood vessel branches in angiography images according to claim 1, characterized in that: The branch feature decoder uses a fully connected layer to obtain an initial branch-level object query feature after the pixel-level multi-level feature interacts with the global feature, and performs cross-attention with the pixel-level feature to obtain the branch-level object query feature after interaction.

4. The method for identifying blood vessel branches in angiography images according to claim 1, characterized in that: The branch-dependent feature encoder obtains branch pixel distribution information by performing a dot product operation on the pixel-level feature and the branch-level query feature, and obtains the branch-level geometric feature and semantic feature by extracting the branch-level geometric and semantic features.

5. A medical image key point detection device, characterized in that: Used to execute the medical image key point detection method according to any one of claims 1 to 4, the device comprising: A multi-level feature acquisition unit, used for extracting multi-level features from the acquired single-frame X-ray angiography image using an image encoder to obtain multi-level features; A pixel-level multi-level feature optimization unit, used to optimize and improve the multi-level features using a small side branch feature supplementation module to obtain pixel-level multi-level features; A judgment unit is used to judge whether to enter a multi-stage decoding operation; determine a non-termination iteration, and use the initialized features or the previously optimized branch query features as input for decoding; A branch feature decoding unit, configured to decode a branch-level query feature of each instance object from the pixel-level multi-level feature using a branch feature decoder; A branch dependency graph acquisition unit is used to extract geometric features and semantic features at the branch object level from the pixel level features and the branch level query features using a branch dependency feature encoder; and to calculate dependencies between branches using the geometric features and semantic features at the branch object level to obtain a branch dependency graph; The detection result acquisition unit is used to determine when the iteration is ended and to classify the branch dependency graph using a branch classifier to obtain a pixel-level segmentation result and a branch-level classification result.

6. A medical image key point detection device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the medical image key point detection method according to any one of claims 1-4 according to the instructions in the program code.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the medical image key point detection method according to any one of claims 1 to 4.

Images