Vessel type recognition method, apparatus and electronic device

Abstract

Embodiments of the present application provide a blood vessel type recognition method, device and electronic equipment. The blood vessel type recognition method comprises: obtaining a DSA sequence in DICOM format, obtaining pixel data and body position information according to the DSA sequence, and the sequence frame in the pixel data is in single-channel format; obtaining a pixel feature from the sequence frame and an angle feature from the body position information; fusing the pixel feature and the angle feature to obtain a fused feature; and outputting a blood vessel type according to the fused feature. In the embodiments of the present application, the pixel data and the body position information are obtained from the DSA sequence in DICOM format, then the pixel feature is obtained from the sequence frame and the angle feature is obtained from the body position information, and after the fused feature is obtained, the blood vessel type is output according to the fused feature. Compared with the blood vessel type recognition method in the related art which relies on the subjective recognition of doctors, the blood vessel type recognition method in the embodiments of the present application can more objectively determine the blood vessel type, and has higher efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of medical image analysis technology, and in particular to methods, devices and electronic devices for blood vessel type identification. Background Technology

[0002] Digital subtraction angiography (DSA) is an important imaging technique for diagnosing cardiovascular diseases. In clinical practice, DSA allows physicians to identify vessel types based on the DSA sequence and the angular information acquired by the equipment.

[0003] However, in clinical practice, doctors' identification of blood vessel types based on DSA sequences and angle information acquired by the equipment is easily affected by fatigue factors, and the judgment based on subjective experience lacks quantitative analysis, thus resulting in low efficiency and accuracy of identification. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, and electronic device for identifying blood vessel types, which can solve the problem of low efficiency and accuracy in clinical practice when relying on doctors' subjective identification of blood vessel types.

[0005] To solve the above-mentioned technical problems, the embodiments of this application are implemented through the following aspects.

[0006] In a first aspect, embodiments of this application provide a method for identifying blood vessel types. The method includes: acquiring a digital subtraction angiography (DSA) sequence in a medical digital imaging and communication standard format; acquiring pixel data and body position information based on the DSA sequence, wherein the sequence frames in the pixel data are in single-channel format; acquiring pixel features from the sequence frames and acquiring angle features from the body position information; fusing the pixel features and the angle features to obtain fused features; and outputting the blood vessel type based on the fused features.

[0007] Secondly, embodiments of this application provide a blood vessel type identification device, the device comprising: a data loading and preprocessing module, used to acquire a digital subtraction angiography (DSA) sequence in a medical digital imaging and communication standard format, and to acquire pixel data and position information based on the DSA sequence, wherein the sequence frames in the pixel data are in single-channel format; a pixel feature extraction module, used to extract pixel features from the sequence frames; an angle information processing module, used to extract angle features from the position information; a feature fusion module, used to fuse the pixel features and the angle features to obtain fused features; and a classification module, used to output the blood vessel type based on the fused features.

[0008] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and computer-executable instructions stored in the memory and executable on the processor, wherein the computer-executable instructions, when executed by the processor, implement the steps of the method described in the first aspect. Fourthly, embodiments of this application provide a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method described in the first aspect. In this embodiment, the blood vessel type identification method includes: acquiring a DSA sequence in DICOM format; acquiring pixel data and body position information based on the DSA sequence, wherein the sequence frames in the pixel data are in single-channel format; acquiring pixel features from the sequence frames and acquiring angle features from the body position information; fusing the pixel features and the angle features to obtain a fused feature; and outputting the blood vessel type based on the fused feature. This embodiment acquires pixel data and body position information from a DSA sequence in DICOM format, then acquires pixel features from the sequence frames and angle features from the body position information, obtains a fused feature, and outputs the blood vessel type based on the fused feature. Therefore, compared to related technologies that rely on doctors' subjective identification of blood vessel types, the blood vessel type identification method of this embodiment can more objectively determine multimodal blood vessel types, with higher identification efficiency and accuracy. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This illustration shows a flowchart of a blood vessel type identification method provided in an embodiment of this application. Figure 2 This illustration shows another flowchart of the blood vessel type identification method provided in the embodiments of this application; Figure 3 This diagram illustrates an architecture of a blood vessel type recognition system provided in an embodiment of this application. Figure 4 This illustration shows a structural schematic diagram of a blood vessel type identification device provided in an embodiment of this application; Figure 5 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of this application.

[0012] Digital subtraction angiography (DSA) is an important imaging technique for diagnosing cardiovascular diseases. In DSA, the DSA sequence and angular information are complementary core elements. In clinical practice, physicians can identify vessel types based on the DSA sequence and the angular information acquired by the equipment. For example, in the coronary arteries, vessel types include the right coronary artery (RCA), the left anterior descending artery (LAD), and the left circumflex artery (LCX).

[0013] However, in clinical practice, doctors' identification of coronary artery types based on DSA sequence data and angular information collected by the equipment is easily affected by fatigue factors, and the judgment based on subjective experience lacks quantitative analysis. Therefore, the efficiency and accuracy of coronary artery type identification are low.

[0014] This application provides a method for identifying blood vessel types to solve the above-mentioned technical problems. Figure 1 This diagram illustrates a flowchart of a blood vessel type identification method provided in an embodiment of this application. The method can be executed by an electronic device, such as a terminal device or a server device. In other words, the method can be executed by software or hardware installed on the terminal device or server device. Servers include, but are not limited to, single servers, server clusters, cloud servers, or cloud server clusters. Figure 1 As shown, the method may include the following steps.

[0015] Step S102: Obtain a digital subtraction angiography (DSA) sequence in the format of Digital Imaging and Communications in Medicine (DICOM). Based on the DSA sequence, obtain pixel data and body position information. The sequence frames in the pixel data are in single-channel format.

[0016] Among them, after obtaining the DSA sequence in DICOM format, pixel value normalization, frame sampling and single-channel conversion can be performed to obtain a single-channel sequence frame with a fixed number of frames.

[0017] Among DSA sequences, DICOM is the only clinically standard format. This sequence data can contain multiple image frames, which can be combined to form an image sequence.

[0018] In this embodiment, the sequence frames in the pixel data can be in a single-channel format. Of course, in other implementations, the sequence frames in the pixel data can also be in a multi-channel format, such as a three-channel RGB format. Compared to a multi-channel format, the single-channel format in this application embodiment can reduce computational load and require fewer computing resources; in one embodiment, in the blood vessel type recognition method, converting the DSA sequence frames from a multi-channel format to a single-channel format can reduce computational load by 67%.

[0019] Step S104: Obtain pixel features from the sequence frames and angle features from the body position information.

[0020] It should be noted that in this application, obtaining pixel features from sequence frames and obtaining angle features from body position information can be done in parallel or sequentially.

[0021] Pixel features can be extracted from the sequence frames using a three-dimensional convolutional neural network (3D convolutional network). The first layer of this 3D convolutional neural network can have only one input channel, meaning it is suitable for single-channel input to accommodate the single-channel format of the sequence frames. This 3D convolutional neural network can be an improvement on a basic 3D convolutional neural network to adapt to the single-channel format of the sequence frames. Alternatively, in other embodiments, if the sequence frames in the pixel data are in a multi-channel format, the first layer of the 3D convolutional neural network can have more than one input channel, and the original fully connected layer in the 3D convolutional neural network can be removed.

[0022] The 3D convolutional neural network can be an R3D-18 3D convolutional network, where the original fully connected layers can be removed. The R3D-18 3D convolutional network can output 512-dimensional features, but it can also output 256-dimensional or 1024-dimensional features. However, the 3D convolutional neural network does not have to be R3D-18; it can be other 3D convolutional networks, such as the R3D-34 3D convolutional network.

[0023] The position information may include the positioner primary angle and / or the positioner secondary angle.

[0024] The primary and secondary angles of the locator are two angular parameters that describe the orientation of the X-ray tube / detector. They correspond to the dual-axis rotation angle of the C-arm of the DSA device and directly determine the projection shape of the blood vessel on the two-dimensional image.

[0025] In one implementation, angular features can be obtained from body position information based on a fully connected neural network layer. This fully connected neural network layer may include, in sequence, a linear layer, a batch normalization layer (BatchNorm), a rectified linear unit (ReLU), and a dropout layer, adapted to extract angular features from body position information. The angular features output by this fully connected neural network layer can be high-dimensional features, such as 32-dimensional, 16-dimensional, or 64-dimensional angular features.

[0026] In this embodiment, the body position information refers to the unstandardized angle information, namely the positioner primary angle and positioner secondary angle obtained directly from the DICOM protocol. This reduces the processing steps for angle information (or body position information) and avoids information loss.

[0027] In this embodiment of the application, since the sequence frame of the DSA sequence is in single-channel format and the body position information is the original information in the DSA sequence that has not been standardized, the efficiency of identifying blood vessel type is relatively high.

[0028] Step S106: Fuse the pixel features with the angle features to obtain fused features.

[0029] First, pixel features and angle features are concatenated to obtain concatenated features. For example, concatenating 512-dimensional pixel features with 32-dimensional angle features yields 544-dimensional concatenated features. Then, based on an attention mechanism network, the pixel features and angle features are fused to generate weighted fused features, achieving weighted enhancement of effective features. For example, based on the attention mechanism network, the weight of each concatenated feature in the above 544-dimensional concatenated features is calculated, thereby generating weighted fused features. The attention mechanism network can combine the Tanh activation function and Softmax weight allocation. The Tanh activation function can be used to enhance the non-linear expression of attention features, while Softmax weight allocation is used to normalize the weight distribution, ultimately allowing the attention mechanism network to automatically focus on the fused features of effective perspectives and suppress noise.

[0030] Step S108: Output the vessel type based on the fusion features. The vessel can be a coronary artery; therefore, the vessel type can include, but is not limited to, at least one of the following: right coronary artery (RCA), left anterior descending artery (LAD), and left circumflex artery (LCX). The vessel type can be output via a fusion classification layer based on the fusion features.

[0031] In one implementation, step S108 can output the finally determined blood vessel type.

[0032] In this embodiment, pixel data and body position information are obtained from a DSA sequence in DICOM format. Then, pixel features are obtained from the sequence frame and angle features are obtained from the body position information. After obtaining the fused features, the blood vessel type is output based on the fused features. Compared with related technologies that rely on doctors' subjective identification of blood vessel types, the blood vessel type identification method of this embodiment can more objectively determine the blood vessel type of multimodal coronary arteries, and the identification efficiency and accuracy are higher.

[0033] Figure 2 This diagram illustrates a flowchart of a blood vessel type identification method provided in an embodiment of this application. The method can be executed by an electronic device, such as a terminal device or a server device. In other words, the method can be executed by software or hardware installed on the terminal device or server device. Servers include, but are not limited to, single servers, server clusters, cloud servers, or cloud server clusters. Figure 2 As shown, the method may include the following steps.

[0034] Step S202: Obtain a digital subtraction angiography (DSA) sequence in the format of Digital Imaging and Communications in Medicine (DICOM). Based on the DSA sequence, obtain pixel data and body position information. The sequence frames in the pixel data are in single-channel format.

[0035] Step S204: Obtain pixel features from the sequence frames and angle features from the body position information.

[0036] Step S206: Fuse the pixel features with the angle features to obtain fused features.

[0037] Step S208: Output the vessel type and corresponding confidence level based on the fusion features. The vessel can be a coronary artery; therefore, the vessel type can include, but is not limited to, at least one of the following: right coronary artery (RCA), left anterior descending artery (LAD), and left circumflex artery (LCX). The vessel type can be output via a fusion classification layer based on the fusion features.

[0038] For example, the output vessel type is RCA, with a corresponding confidence level of A% (A can be greater than 90). Another example is that after outputting the vessel type as RCA with a corresponding confidence level of A%, it is also possible to output the vessel type as LAD with a corresponding confidence level of B%, and the vessel type as LCX with a corresponding confidence level of (1-AB)%. Therefore, based on the output vessel type and its corresponding confidence level, the vessel type of the coronary artery can be identified (or used to assist in determining) its type, improving the efficiency and accuracy of identification.

[0039] Steps S202-206 above can be referred to steps S102-106 above, and will not be repeated here.

[0040] In this embodiment, pixel data and body position information are obtained from a DSA sequence in DICOM format. Then, pixel features are obtained from the sequence frame and angle features are obtained from the body position information. After obtaining the fused features, the blood vessel type is output based on the fused features. Compared with related technologies that rely on doctors' subjective identification of blood vessel types, the blood vessel type identification method of this embodiment can more objectively determine the blood vessel type of multimodal coronary arteries, and the identification efficiency and accuracy are higher.

[0041] Figure 3This paper illustrates an architecture diagram of a blood vessel type recognition system according to an embodiment of this application. The recognition process includes first inputting a DICOM sequence image into the blood vessel recognition system, then acquiring sequence pixels and body position angles. The acquisition of sequence pixels and body position angles can be performed in parallel or synchronously to extract pixel features and angle features in parallel, thereby improving recognition efficiency. Then, the pixel features and angle adjustments are fused together, and the blood vessel type is classified and output. Finally, the recognition process ends.

[0042] Figure 4 The diagram shows a structural schematic of a blood vessel type recognition device provided in an embodiment of this application. The device 100 may include: a data loading and preprocessing module 110, a pixel feature extraction module 120, an angle information processing module 130, a feature fusion module 140, and a classification module 150.

[0043] The data loading and preprocessing module 110 is used to acquire digital subtraction angiography (DSA) sequences in a standard format for medical digital imaging and communication, and to acquire pixel data and position information based on the DSA sequences. The sequence frames in the pixel data are in single-channel format.

[0044] The pixel feature extraction module 120 is used to obtain pixel features from the sequence frames.

[0045] Angle information processing module 130 is used to obtain angle features from the body position information.

[0046] The feature fusion module 140 is used to fuse the pixel features with the angle features to obtain fused features.

[0047] The classification module 150 is used to output the blood vessel type based on the fusion features.

[0048] In one embodiment, the pixel feature extraction module 120 is specifically used to: extract pixel features from the sequence frames based on an R3D-18 3D convolutional network, wherein the first layer of the R3D-18 3D convolutional network has 1 input channel.

[0049] In one embodiment, the angle information processing module 130 is specifically used to: obtain angle features from the body position information based on a fully connected neural network layer, wherein the body position information includes the locator primary angle and the locator secondary angle, and the fully connected neural network layer includes a linear layer, batch normalization, a corrected linear unit, and a dropout layer connected in series.

[0050] In one embodiment, the feature fusion module 140 is specifically used to: splice the pixel features and angle features to obtain spliced ​​features; and calculate the weights of the spliced ​​features based on an attention mechanism network to generate weighted fusion features.

[0051] In one embodiment, the classification module 150 is specifically used to output a vessel type through a fusion classification layer based on the fusion features, wherein the vessel type includes the right coronary artery, the left anterior descending artery, or the left circumflex artery.

[0052] The device 100 provided in this application embodiment can execute the methods described in the preceding method embodiments and achieve the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0053] Figure 5 The diagram illustrates the hardware structure of an electronic device implementing the embodiments of this application. Referring to the diagram, at the hardware level, the electronic device includes a processor and optionally, an internal bus, a network interface, and a memory. The memory may include RAM, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include other hardware required for other services.

[0054] The processor, network interface, and memory can be interconnected via an internal bus, which can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be categorized as an address bus, data bus, control bus, etc. For ease of illustration, only a single bidirectional arrow is used in this diagram, but this does not imply that there is only one bus or one type of bus.

[0055] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0056] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a device at the logical level that locates the target user. The processor executes the program stored in memory and specifically performs the following: Figure 1-2 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods described in the preceding method embodiments, and will not be repeated here.

[0057] The above is as stated in this application. Figure 1-2The methods disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0058] The electronic device can also execute the methods described in the preceding method embodiments and achieve the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0059] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0060] This application also proposes a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform... Figure 1-2 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods described in the preceding method embodiments, and will not be repeated here.

[0061] The computer-readable storage medium mentioned above includes read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc.

[0062] Furthermore, embodiments of this application also provide a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, implement the following process: Figure 1-2 The methods disclosed in the embodiments shown achieve the functions and beneficial effects of the methods described in the preceding method embodiments, and will not be repeated here.

[0063] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0064] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0065] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0066] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0067] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for identifying blood vessel types, characterized in that, The method includes: Obtain a digital subtraction angiography (DSA) sequence in a standard format for medical digital imaging and communication; based on the DSA sequence, obtain pixel data and body position information; wherein the sequence frames in the pixel data are in single-channel format. Pixel features are obtained from the sequence frames, and angle features are obtained from the body position information; The pixel features and the angle features are fused to obtain the fused features; The blood vessel type is output based on the fusion features.

2. The method according to claim 1, characterized in that, Extracting pixel features from the sequence frames includes: Pixel features are extracted from the sequence frames based on a 3D convolutional network, wherein the first layer of the 3D convolutional network has 1 input channel.

3. The method according to claim 1, characterized in that, Extracting angle features from the body position information includes: Angle features are obtained from the body position information based on a fully connected neural network layer. The body position information includes the locator's primary angle and secondary angle. The fully connected neural network layer includes a linear layer, batch normalization, modified linear unit, and dropout layer connected in series.

4. The method according to claim 1, characterized in that, The step of fusing the pixel features and the angle features to obtain the fused features includes: By concatenating the pixel features and the angle features, a concatenated feature is obtained; The weights of the spliced ​​features are calculated based on the attention mechanism network to generate weighted fusion features.

5. The method according to claim 1, characterized in that, The step of outputting the blood vessel type based on the fusion features includes: Based on the fusion characteristics, the fusion classification layer outputs the vessel type, which includes the right coronary artery, the left anterior descending artery, or the left circumflex artery.

6. A blood vessel type identification device, characterized in that, The device includes: The data loading and preprocessing module is used to acquire digital subtraction angiography (DSA) sequences in a standard format for medical digital imaging and communication, and to acquire pixel data and body position information based on the DSA sequences. The sequence frames in the pixel data are in single-channel format. A pixel feature extraction module is used to obtain pixel features from the sequence frames; An angle information processing module is used to obtain angle features from the body position information; The feature fusion module is used to fuse the pixel features with the angle features to obtain fused features; The classification module is used to output the blood vessel type based on the fusion features.

7. The apparatus according to claim 6, characterized in that, The pixel feature extraction module extracts pixel features from the sequence frames, including: Pixel features are extracted from the sequence frames based on a 3D convolutional network, wherein the first layer of the 3D convolutional network has 1 input channel.

8. The apparatus according to claim 6, characterized in that, The angle information processing module is used to obtain angle features from the body position information, including: Angle features are obtained from the body position information based on a fully connected neural network layer. The body position information includes the locator's primary angle and secondary angle. The fully connected neural network layer includes a linear layer, batch normalization, modified linear unit, and dropout layer connected in series.

9. An electronic device, characterized in that, include: processor; as well as A memory configured to store computer-executable instructions, which, when executed, use the processor to perform the steps of the blood vessel type identification method according to any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer-readable medium stores one or more programs that, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the steps of the method according to any one of claims 1-4.

Images