Coronary heart disease auxiliary diagnosis system, system operation method and diagnosis device
Through the coronary heart disease assisted diagnosis system based on artificial intelligence, the coronary angiography image features are extracted using fine-tuning encoder and multi-view feature fusion module, which solves the problem of doctor experience dependence and deep learning model overfitting in the existing technology, and achieves rapid , accurate early screening and diagnosis of coronary heart disease.
Patent Information
- Application Number
- CN202411010583.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-12
AI Technical Summary
The existing technology has doctor experience dependence, high risk of misdiagnosis, "black box" problems of deep learning models and insufficient data in early screening of coronary heart disease, making it difficult to build a fast and accurate coronary heart disease auxiliary diagnosis system.
A coronary heart disease-assisted diagnosis system based on artificial intelligence was designed, including image acquisition module, fine-tuning encoder, foreground prototype extraction module, multi-view feature fusion module and classifier, freezing encoding blocks, global feature fine-tuning module and local features The fine-tuning module extracts coronary angiography image features, combines multi-view feature fusion and classifier output diagnostic results to improve the robustness and generalization ability of the model.
It realizes rapid and accurate diagnosis of vascular structure images of coronary heart disease, reduces the risk of doctors' fatigue and misdiagnosis, and can build an effective early screening assistance system based on a small amount of data, improving the accuracy and efficiency of diagnosis.
Smart Images

Figure CN118942680A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a coronary heart disease auxiliary diagnosis system, a system operation method and a diagnosis device, and belongs to the field of medical auxiliary diagnosis devices. Background Art
[0002] Coronary heart disease is a common cardiovascular disease worldwide, and its harm to the human body cannot be underestimated. With the growth of population and changes in lifestyle, the number of patients with coronary heart disease continues to increase, especially in developing countries, this trend is more significant. Therefore, how to improve the early diagnosis and treatment of coronary heart disease is particularly urgent.
[0003] In recent years, medical imaging technology has made great progress, providing a more flexible and accurate method for early screening of coronary heart disease. At present, common coronary heart disease imaging technologies include coronary angiography and cardiac magnetic resonance imaging. However, the current coronary angiography technology has high requirements for clinicians in terms of diagnosis, requiring doctors to have rich experience and skills. In addition, a large number of coronary angiography images require doctors to make judgments, which can easily lead to doctor fatigue, thereby increasing the risk of misdiagnosis.
[0004] In recent years, deep learning technology has achieved great success in the diagnosis of coronary angiography images. However, the "black box" problem of deep learning models makes its training process difficult to explain, and relying solely on indicators such as accuracy cannot fully reflect the reliability of its practical application. In addition, deep learning algorithms require a large amount of data for training to ensure that the model learns sufficient experience. However, since medical images involve patient privacy, it is not easy to collect annotated data. When the training data is insufficient, deep learning algorithms are prone to overfitting. Therefore, using a small amount of coronary angiography data to build a fast and accurate early screening auxiliary system for coronary heart disease is a challenging task, which requires comprehensive consideration of issues such as data acquisition, the interpretability of deep learning models, and practical support for doctors. Summary of the invention
[0005] In order to solve the above problems, the present invention provides a coronary heart disease auxiliary diagnosis system, an operation method and a diagnosis device, and the technical solution is as follows:
[0006] The first object of the present invention is to provide an artificial intelligence-based auxiliary diagnosis system for coronary heart disease, the system comprising:
[0007] An image acquisition module is configured to acquire a coronary angiography image of coronary heart disease;
[0008] A coronary heart disease classification module is configured to output a coronary heart disease classification result based on the coronary heart disease coronary angiography image;
[0009] The coronary heart disease classification module includes: a fine-tuning encoder, a foreground prototype extraction module, a multi-view feature fusion module and a classifier;
[0010] The fine-tuning encoder comprises: a VIT frozen encoding block, a global feature fine-tuning module and a local feature fine-tuning module, wherein the VIT frozen encoding block is configured to extract features of the coronary angiography image of coronary heart disease; the global feature fine-tuning module is configured to extract global features of the coronary angiography image of coronary heart disease; the local feature fine-tuning module is configured to capture local detail information of the blood vessel image and extract local features of the coronary angiography image of coronary heart disease;
[0011] The foreground prototype extraction module is used to extract key feature information of blood vessel information and extract representative feature vectors of blood vessels;
[0012] The multi-view feature fusion module is configured to: use the representative feature vector of the blood vessel obtained by the foreground prototype extraction module to screen multiple local features, retain the useful foreground blood vessel information, and discard the background information with less influence; fuse the global features of the coronary angiography image of coronary heart disease with the screened local features;
[0013] The classifier is configured to extract key information from the fused features and output a coronary heart disease classification result.
[0014] Optionally, the global feature fine-tuning module includes: a multi-head attention layer, a forward propagation layer, an adaptation layer, and a normalization layer connected in sequence;
[0015] The adaptation layer includes: a downsampling layer, a nonlinearity module and an upsampling layer connected in sequence. In the multi-head attention, each attention head h i The calculation formula is expressed as:
[0016]
[0017] Where q is the query, k is the key, and v is the value. is the parameter obtained after q passes through the fully connected layer, is the parameter obtained after k passes through the fully connected layer, is the parameter obtained after v passes through the fully connected layer.
[0018] Optionally, the local feature fine-tuning module includes: a downsampling layer, a nonlinear activation function and an upsampling layer connected in sequence, and the local feature fine-tuning module adaptively extracts local feature information of the coronary angiography image through gradient descent; given an input x, the feedforward process of the local feature fine-tuning module is expressed as follows:
[0019] x down =Wdown x+b down
[0020] x up =Relu(x down )W up +b up
[0021] Among them, W down represents the downsampling matrix, b down represents the downsampling bias, W up represents the upsampling matrix, b up Represents the upsampling bias.
[0022] Optionally, the working process of the foreground prototype extraction module includes:
[0023] First, several examples of vascular annotation images of coronary angiography are annotated, and then mask average pooling is used to obtain multiple prototypes, and then the multiple prototypes are averaged to finally obtain a representative feature vector of the blood vessels in the angiography part of the coronary angiography image, which is expressed as:
[0024]
[0025] Where K represents the index of each labeled image, represents the kth prototype.
[0026] Optionally, the working process of the multi-view feature fusion module includes:
[0027] Calculating the cosine similarity between the representative feature vector of the blood vessel and each local feature vector, and giving a threshold; determining the local feature vector greater than the threshold as a valid foreground feature vector; determining the local feature vector less than the threshold as background information, and then discarding it;
[0028] The effective foreground feature vector is fused with the global information to obtain a more global and comprehensive feature representation.
[0029] Optionally, the calculation of the cosine similarity is expressed as:
[0030]
[0031] in Represents the similarity graph of each local feature map, P l represents the representative feature vector of the blood vessel, represents a local feature vector.
[0032] Optionally, the classifier includes: a 3x3 convolution layer, a 1x1 convolution layer, a 3x3 DW convolution layer, a 1x1 convolution layer and a Sigmoid operation layer, and the expression of the classifier is:
[0033] P=σ(W f4 (W f3 (W f2 (W f1 (x)))))
[0034] Among them, W f1 , W f2 , W f3 and W f4 Respectively represent convolution, σ is the Sigmoid operation, where W f3 It is DW convolution.
[0035] A second object of the present invention is to provide an operating method of a coronary heart disease auxiliary diagnosis system, comprising:
[0036] Step 1: Obtain coronary angiography images of coronary heart disease;
[0037] Step 2: Classify the coronary heart disease angiography image using the coronary heart disease classification module as described in any of the above items to obtain a classification result.
[0038] The third object of the present invention is to provide a coronary heart disease auxiliary diagnosis device, comprising a memory and a processor;
[0039] The memory is used to store computer programs;
[0040] The processor is used to implement the operating method of the coronary heart disease auxiliary diagnosis system as described above when executing the computer program.
[0041] The fourth object of the present invention is to provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the operating method of the coronary heart disease auxiliary diagnosis system as described above is implemented.
[0042] The beneficial effects of the present invention are:
[0043] The coronary heart disease auxiliary diagnosis system of the present invention can batch process the coronary heart disease vascular structure images to obtain the final disease diagnosis classification results; the present invention uses a fine-tuning encoder, a foreground prototype extractor, a multi-view feature fusion module and a classifier to construct a labeling model, which effectively mines the local and global prior information in the coronary heart disease vascular structure images and solves the problem of difficulty in early screening of coronary heart disease; the present invention uses a multi-view feature fusion module to fully learn the collaborative interaction relationship of effective information between multiple perspectives to further improve the robustness and generalization ability of the coronary heart disease classification model based on artificial intelligence. Compared with the existing technology, the present invention can construct a fast and accurate coronary heart disease early screening auxiliary system based on a small amount of coronary angiography data, thereby realizing fast, accurate and high-throughput diagnosis of coronary heart disease. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0045] Figure 1 A schematic diagram of the basic operation flow of an artificial intelligence-based auxiliary diagnosis system for coronary heart disease provided in Example 2 of the present invention.
[0046] Figure 2 A schematic diagram of the structure of an artificial intelligence-based auxiliary diagnosis system for coronary heart disease provided in Example 2 of the present invention.
[0047] Figure 3 This is a schematic diagram of the structure of the global feature fine-tuning module provided in the second embodiment of the present invention.
[0048] Figure 4 This is a schematic diagram of the structure of a local feature fine-tuning module provided in Embodiment 2 of the present invention.
[0049] Figure 5 This is a schematic diagram of the structure of a foreground prototype extractor provided in Embodiment 2 of the present invention.
[0050] Figure 6 This is a schematic diagram of the classifier structure provided in Example 2 of the present invention.
[0051] Figure 7 This is the original image of coronary angiography of coronary heart disease in Example 3 of the present invention.
[0052] Figure 8 This is a diagram showing the classification results of coronary heart disease according to the third embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0054] Embodiment 1:
[0055] This embodiment provides an artificial intelligence-based coronary heart disease auxiliary diagnosis system, including:
[0056] An image acquisition module is configured to acquire a coronary angiography image of coronary heart disease;
[0057] A coronary heart disease classification module is configured to output a coronary heart disease classification result based on the coronary heart disease coronary angiography image;
[0058] The coronary heart disease classification module includes: fine-tuning encoder, foreground prototype extraction module, multi-view feature fusion module and classifier;
[0059] The fine-tuning encoder includes: a VIT frozen encoding block, a global feature fine-tuning module and a local feature fine-tuning module, wherein the VIT frozen encoding block is configured to extract features of a coronary angiography image of a coronary heart disease; the global feature fine-tuning module is configured to extract global features of a coronary angiography image of a coronary heart disease; and the local feature fine-tuning module is configured to capture local detail information of a vascular image and extract local features of a coronary angiography image of a coronary heart disease;
[0060] The foreground prototype extraction module is used to extract the key feature information of blood vessel information and extract the representative feature vector of blood vessels;
[0061] The multi-view feature fusion module is configured as follows: using the representative feature vector of blood vessels obtained by the foreground prototype extraction module, multiple local features are screened, useful foreground blood vessel information is retained, and background information with less influence is discarded; the global features of the coronary angiography image of coronary heart disease are fused with the screened local features;
[0062] The classifier is configured to extract key information from the fused features and output the coronary heart disease classification results.
[0063] Embodiment 2:
[0064] This embodiment provides an artificial intelligence-based coronary heart disease auxiliary diagnosis system, including: an image acquisition module and a coronary heart disease classification module.
[0065] The image acquisition module is used to acquire coronary angiography images of coronary heart disease. The coronary heart disease classification module includes: a fine-tuning encoder, a foreground prototype extraction module, a multi-view feature fusion module and a classifier; for details, refer to Figure 2, input the coronary angiography image of coronary heart disease, after fine-tuning the global feature fine-tuning module and the local feature fine-tuning module in the encoder, the network learns the global features and local features at the same time, and then enters the multi-view feature fusion module, using the foreground prototype extracted by the foreground prototype extractor to obtain a more global and comprehensive feature representation, and finally these two features are passed through the classifier to output the classification result.
[0066] Among them, the fine-tuning encoder includes a VIT frozen encoding block, a global feature fine-tuning module and a local feature fine-tuning module.
[0067] The VIT frozen coding block is used to extract the features of the vascular structure images of coronary heart disease, and the large model pre-trained encoder is used to extract features.
[0068] The global feature fine-tuning module is used to fine-tune the encoding using a small amount of coronary angiography images, making the entire feature encoder more suitable for cardiovascular structure image data and better focusing on global information. The structure of the global feature fine-tuning module is as follows: Figure 3 As shown in Figure 1, it includes: multi-head attention, forward propagation layer, adaptation layer and normalization layer; the adaptation layer includes: downsampling layer, nonlinearity and upsampling layer. In the multi-head attention, each attention head h i The calculation formula can be expressed as:
[0069]
[0070] Where q is the query, k is the key, and v is the value. is the parameter obtained after q passes through the fully connected layer, is the parameter obtained after k passes through the fully connected layer, is the parameter obtained after v passes through the fully connected layer.
[0071] The local feature fine-tuning module is used to capture the local details of the vascular image, allowing the network coding module to better extract important local information. The structure of the local feature fine-tuning module is as follows: Figure 4 As shown in Figure 1, it consists of a downsampling layer, a nonlinear activation function similar to RELU, and an upsampling layer. The fine-tuning module adaptively extracts local feature information of coronary angiography images through gradient descent. Given the input x, the feedforward process of the fine-tuning module can be expressed as follows:
[0072] x down =W down x+b down
[0073] x up =Relu(x down )W up +b up
[0074] In the above formula, W down represents the downsampling matrix, b down represents the downsampling bias; accordingly, W up and b up represents the matrix and bias in the upsampling operation. In general, this process can be expressed as:
[0075] Adapter(x)=Relu(W down x+b down )W up +b up
[0076] The foreground prototype extractor is to extract the key feature information of blood vessel information, extract the representative feature vector of blood vessels, and then screen multiple local feature information, retain the useful foreground blood vessel information, and discard the background information with less influence. The structure of the foreground extractor is as follows: Figure 5 As shown in the figure, firstly, the blood vessel annotation images of several coronary angiography images are annotated, and then mask average pooling is used to obtain multiple prototypes, and then the multiple prototypes are averaged to finally obtain a representative feature vector of the blood vessels in the angiography part of the coronary angiography image. Specifically, it can be expressed as:
[0077]
[0078] Where K represents the index of each labeled image.
[0079] The multi-view feature fusion module is used for feature fusion, which fuses the global feature information and effective local feature information of the vascular structure image of coronary heart disease, interacts and integrates the features at different positions, and thus extracts a more global and comprehensive feature representation. The structure of the multi-view feature fusion module is as follows: Figure 1 As shown in the figure, this module uses the foreground prototype vector obtained above to screen multiple local feature vectors, where cosine similarity is used to calculate the similarity. After the calculation, a learnable threshold is given. If it is greater than the threshold, the local feature vector is considered to be a foreground feature vector, which is beneficial to the diagnosis of the network. For local feature vectors below the threshold, they are judged as background information and then discarded. Finally, the effective local feature information is fused with the global information to obtain a more global and comprehensive feature representation. Specifically, the similarity calculation can be expressed as:
[0080]
[0081] in Represents the similarity graph of each local feature map, P l represents the foreground prototype, represents a local feature vector.
[0082] The classifier is used to extract key information from the fused features and finally give the diagnosis result. After 3x3 convolution, 1x1 convolution, 3x3 DW convolution, 1x1 convolution and Sigmoid operation, it can be specifically expressed as:
[0083] P=σ(W f4 (W f3 (W f2 (W f1 (x)))))
[0084] Among them, W f1 , W f2 , W f3 and W f4 is the convolutional layer, and σ is the Sigmoid operation. In particular, W f3 It is DW convolution.
[0085] Furthermore, the training loss function is used to train the model. The loss function is used to guide network annotation learning and can be expressed in the formula as follows:
[0086]
[0087] is the task loss, x (i) is the task output, y (i) It is the mission objective.
[0088] After the model is trained, it can be used for auxiliary diagnosis of coronary heart disease. The operation process of the auxiliary diagnosis system is as follows:
[0089] The doctor obtains the vascular structure diagram of coronary heart disease from coronary angiography and inputs it into the trained coronary heart disease classification module. The auxiliary diagnosis system outputs the final classification result, and the doctor informs the patient of the result.
[0090] Embodiment three:
[0091] In order to verify and illustrate the technical effect of the present invention, this embodiment selects a traditional technical solution of a coronary heart disease diagnosis method and compares it with the diagnosis system of the present invention to verify the real effect of the present invention.
[0092] This embodiment will perform classification experiments on coronary angiography images. Currently, available coronary angiography image data is collected. This embodiment establishes a database derived from real patient cases, collects CAG images of 510 patients, uses 350 (about 70%) as training samples and the remaining 160 (about 30%) as test samples.
[0093] The coronary heart disease auxiliary diagnosis system of the present invention is compared with the traditional solution, and the comparison results are shown in Table 1 below:
[0094] Table 1: Comparison results between the present invention and the traditional solution
[0095] Comparison Object Traditional methods Diagnostic system of the present invention Expert diagnosis Accuracy / % 80 94 90 Time / s 0.43 0.25 300-600 Scope of application Specific images wide wide
[0096] It can be seen from the above comparison data that the diagnostic system of the present invention is higher in accuracy and efficiency than traditional solutions and expert diagnosis, and is faster in time. Traditional methods cannot be promoted on a large scale, but the auxiliary diagnosis system of the present invention can use deep learning networks to efficiently promote to larger data, and finally obtain better diagnostic results.
[0097] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.
[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A coronary heart disease auxiliary diagnosis system, characterized in that: The system comprises: An image acquisition module is configured to acquire a coronary angiography image of coronary heart disease; A coronary heart disease classification module is configured to output a coronary heart disease classification result based on the coronary heart disease coronary angiography image; The coronary heart disease classification module includes: a fine-tuning encoder, a foreground prototype extraction module, a multi-view feature fusion module and a classifier; The fine-tuning encoder comprises: a VIT frozen encoding block, a global feature fine-tuning module and a local feature fine-tuning module, wherein the VIT frozen encoding block is configured to extract features of the coronary angiography image of coronary heart disease; the global feature fine-tuning module is configured to extract global features of the coronary angiography image of coronary heart disease; the local feature fine-tuning module is configured to capture local detail information of the blood vessel image and extract local features of the coronary angiography image of coronary heart disease; The foreground prototype extraction module is used to extract key feature information of blood vessel information and extract representative feature vectors of blood vessels; The multi-view feature fusion module is configured to: use the representative feature vector of the blood vessel obtained by the foreground prototype extraction module to screen multiple local features, retain the useful foreground blood vessel information, and discard the background information with less influence; fuse the global features of the coronary angiography image of coronary heart disease with the screened local features; The classifier is configured to extract key information from the fused features and output a coronary heart disease classification result.
2. The coronary heart disease auxiliary diagnosis system according to claim 1, characterized in that: The global feature fine-tuning module includes: a multi-head attention layer, a forward propagation layer, an adaptation layer, and a normalization layer connected in sequence; The adaptation layer includes: a downsampling layer, a nonlinearity module and an upsampling layer connected in sequence. In the multi-head attention, each attention head h i The calculation formula is expressed as: Where q is the query, k is the key, and v is the value. is the parameter obtained after q passes through the fully connected layer, is the parameter obtained after k passes through the fully connected layer, is the parameter obtained after v passes through the fully connected layer.
3. The coronary heart disease auxiliary diagnosis system according to claim 1, characterized in that: The local feature fine-tuning module includes: a downsampling layer, a nonlinear activation function and an upsampling layer connected in sequence, and the local feature fine-tuning module adaptively extracts local feature information of the coronary angiography image through gradient descent; given an input x, the feedforward process of the local feature fine-tuning module is expressed as follows: x down =W down x+b down x up =Relu(x down )W up +b up Among them, W down represents the downsampling matrix, b down represents the downsampling bias, W up represents the upsampling matrix, b up Represents the upsampling bias.
4. The coronary heart disease auxiliary diagnosis system according to claim 1, characterized in that: The working process of the foreground prototype extraction module includes: First, several examples of vascular annotation images of coronary angiography are annotated, and then mask average pooling is used to obtain multiple prototypes, and then the multiple prototypes are averaged to finally obtain a representative feature vector of the blood vessels in the angiography part of the coronary angiography image, which is expressed as: Where K represents the index of each labeled image, represents the kth prototype.
5. The coronary heart disease auxiliary diagnosis system according to claim 1, characterized in that: The working process of the multi-view feature fusion module includes: Calculating the cosine similarity between the representative feature vector of the blood vessel and each local feature vector, and giving a threshold; determining the local feature vector greater than the threshold as a valid foreground feature vector; determining the local feature vector less than the threshold as background information, and then discarding it; The effective foreground feature vector is fused with the global information to obtain a more global and comprehensive feature representation.
6. The coronary heart disease auxiliary diagnosis system according to claim 5, characterized in that: The calculation of the cosine similarity is expressed as: in Represents the similarity graph of each local feature map, P l represents the representative feature vector of the blood vessel, represents a local feature vector.
7. The coronary heart disease auxiliary diagnosis system according to claim 1, characterized in that: The classifier includes: a 3x3 convolution layer, a 1x1 convolution layer, a 3x3 DW convolution layer, a 1x1 convolution layer and a Sigmoid operation layer. The expression of the classifier is: Among them, W f1 , W f2 , W f3 and W f4 Respectively represent convolution, σ is the Sigmoid operation, where W f3 It is DW convolution.
8. The method for operating the coronary heart disease auxiliary diagnosis system according to any one of claims 1 to 7, characterized in that: The method comprises: Step 1: Obtain coronary angiography images of coronary heart disease; Step 2: Classify the coronary heart disease angiography image using the coronary heart disease classification module described in any one of claims 1 to 7 to obtain a classification result.
9. A device for assisting diagnosis of coronary heart disease, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the operating method of the coronary heart disease auxiliary diagnosis system as claimed in claim 8 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the operating method of the coronary heart disease auxiliary diagnosis system as claimed in claim 8 is implemented.
Citation Information
Cited By
Pituitary adenoma invasion image data analysis method and system
CN119495407A
A method and system for analyzing pituitary adenoma invasion images
CN119495407B
