Evidence reasoning method and visual evidence labeling method for medical image classification
Through feature aggregation voting and dynamic graph network reasoning structure, an evidence reasoning model is constructed, which solves the problem of lack of evidence support and reasoning ability in the existing traditional Chinese medicine image classification, and achieves higher credibility and accuracy.
Patent Information
- Application Number
- CN202210244504.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-14
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2042-03-14
AI Technical Summary
The existing computer-assisted classification methods lack credible evidence support and reasoning capabilities in medical image classification, resulting in low reliability and accuracy of classification results, and prone to pseudo-related problems.
Using feature aggregation voting and dynamic graph network inference structure, the feature map of medical images is transformed into symbolic nodes by constructing an evidence inference model, and a graph network is established to conduct evidence inference and quantification, and the interpretability and accuracy of classification results are optimized.
It improves the credibility and transparency of medical image classification, provides reliable evidence support, and improves the accuracy and interpretability of classification results.
Smart Images
Figure CN114595637B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing, and in particular relates to an evidence reasoning method and a visual evidence marking method for medical image classification. Background Art
[0002] With the development of economy and technology and the improvement of people's living standards, people are paying more and more attention to health. In recent years, with the innovation and continuous application of deep learning in the field of medical technology, the field of smart medical care has also developed rapidly. The application of computer-aided classification methods in medical image classification tasks has greatly facilitated medical staff and saved a lot of manpower and material resources.
[0003] However, existing computer-assisted classification methods still have defects: First, the classification process of the model is like a black box, and does not provide users with reliable classification basis; for example, when a medical image (such as a fundus image) is input, the existing model can only give a normal or abnormal classification result, but does not provide evidence to support the judgment of the classification; and the auxiliary classification process without evidence support is unacceptable in the medical field. Second, the existing computer-assisted classification methods do not have reasoning capabilities; the existing methods establish associations with the classification probability of medical images through low-semantic information, which is easy to cause pseudo-correlation, resulting in low reliability and poor accuracy of the classification results. Summary of the invention
[0004] One of the purposes of the present invention is to provide an evidentiary reasoning method for medical image classification, which includes feature aggregation voting and dynamic graph network reasoning structure to make the classification process more credible and transparent, and has good interpretability, high reliability and good accuracy.
[0005] A second object of the present invention is to provide a visual evidence marking method including an evidence reasoning method for medical image classification.
[0006] The evidential reasoning method for medical image classification provided by the present invention comprises the following steps:
[0007] S1. Obtain existing medical images, and mark them one-to-one with the real labels of each medical image to obtain a training data set, and then extract feature maps of the medical images;
[0008] S2. Constructing a primitive model for evidential reasoning for medical image classification:
[0009] Constructing an Evidence Reasoning Model:
[0010] S2.1. The feature map corresponding to the medical image is converted into a symbolic node by using feature aggregation voting; a symbolic node represents a class of medical image attributes;
[0011] S2.2. Use the symbolic nodes obtained in step S2.1 as nodes of the graph network, and the correlations between nodes as edges of the graph network to construct a graph network;
[0012] S2.3. Model the correlation of global attributes, and use the graph network obtained in step S2.2 to perform evidence reasoning to obtain evidence information supporting medical image classification, and quantify the evidence information into numerical parameters;
[0013] Using the constructed evidence reasoning model, medical images are classified:
[0014] S2.4. Set the original probability distribution of the medical image, and use the quantified evidence information obtained in step S2.3 to optimize the original probability distribution of the medical image;
[0015] S2.5. Perform numerical evaluation on the classification results;
[0016] S3. Using the training data set obtained in step S1, the original evidence reasoning model for medical image classification constructed in step S2 is trained to obtain an evidence reasoning model for medical image classification;
[0017] S4. Use the evidence reasoning model for medical image classification obtained in step S3 to classify the actual medical image to be classified and provide corresponding classification evidence.
[0018] Step S1 obtains existing medical images and marks them one-to-one with the real labels of each medical image to obtain a training data set, and then extracts the feature maps of the medical images. Specifically, the existing medical images are divided into normal and abnormal, and the classification results are used as real labels to correspond one-to-one with the medical images, thereby constructing a training data set, and then the Resnet50 skeleton network is used to extract the feature maps of the medical images.
[0019] The feature aggregation voting method described in step S2.1 converts the feature map corresponding to the medical image into a symbolic node; a symbolic node represents a type of medical image attribute, which specifically includes the following steps:
[0020] Aggregate voting uses each pixel value of the feature map F to vote for each node with different voting weights, and then each node aggregates the voting information of all pixel values, and the result represents each medical image attribute; specifically, the following formula is used for calculation:
[0021]
[0022] Where P n is the attribute of each type of medical image; H is the length of the feature map F; W is the height of the feature map F; F i is a local feature; Wg is the dimension conversion matrix; is the local feature F i For each node N n The weight of the vote, and v n is the trainable voting weight, and N is the number of medical image attributes.
[0023] Step S2.2 uses the symbolic nodes obtained in step S2.1 as nodes of the graph network, and uses the correlations between nodes as edges of the graph network to construct the graph network, which specifically includes the following steps:
[0024] Construct a graph network G = {N, ε}, where the medical image attributes are nodes s m , and s m ∈N, N is the number of medical image attributes; the association edge between two medical image attributes is (s m ,s n ), and (s m ,s n )∈ε, ε is the set of associated edges; A is the adjacency matrix between M attributes, and is a matrix of length M and height M.
[0025] The correlation of global attributes described in step S2.3 is modeled, and the graph network obtained in step S2.2 is used for evidence reasoning to obtain evidence information supporting medical image classification, and the evidence information is quantified into numerical parameters, which specifically includes the following steps:
[0026] A. Use the following rules to make the graph network have all symbolic nodes s m Belief propagation is performed on ∈N:
[0027]
[0028] Where H t is the feature node information at time t; σ(.) is the RELU activation function; is the normalized degree matrix; H t-1 is the characteristic node information at time t-1; W t-1 is a trainable weight matrix; is the normalized adjacency matrix at time t-1, and A t-1 is the adjacency matrix at time t-1, I M is the identity matrix;
[0029] B. Learn the relationship between medical image attributes, infer the relationship between local attributes and structure, and obtain evidence to support medical image classification; constrain the intrinsic relationship of the adjacency matrix through the following loss function:
[0030]
[0031]
[0032] Where A t is the adjacency matrix at time t, is the i-th row of the normalized adjacency matrix at time t; lr is the learning rate; l is the loss function; is the i-th row of the adjacency matrix at time t; is the minimum value of the i-th row of the adjacency matrix at time t; is the maximum value of the i-th row of the adjacency matrix at time t;
[0033] C. Use the following formula to calculate the evidence vector:
[0034] e - ,e + =MLP(σ(Conv(concat(F,F'))))
[0035] Where e + To support the evidence for classifying a medical image as abnormal, e - Evidence to support the classification of medical images as normal; MLP(.) is a multi-layer perceptron; Conv(.) is an operation using a 1*1 convolution kernel for processing; is the original feature map; F' is the feature map after inference, and is composed of H t Stretch to get H is the length of feature map F', W is the height of feature map F', C1 is the number of channels of F'; concat(.) is the operation function of adding F and F' on the channel, and
[0036] The step S2.4 of setting the original probability distribution of the medical image and optimizing the original probability distribution of the medical image using the quantized evidence information obtained in step S2.3 specifically includes the following steps:
[0037] Assume that the original probability density of the medical image is Beta distribution B(p i |α i ,β i ), where p i is the classification prediction probability, α i is the evidential parameter, β i is the evidential parameter; the following formula is used to optimize the probability density distribution:
[0038]
[0039]
[0040] Where l D is the loss function; N is the total number of medical images; y i is the true label of the medical image, and y i ∈{0,1};p i is the classification prediction probability; || || is the mean square error; Γ() is the Gamma function; α i is the evidential parameter, and α i =e + +1; β i is the evidential parameter, and β i =e - +1; l is the loss function; λ is a hyperparameter, and λ∈[0,1]; KL(X||Y) is the Kullback-Leibler divergence; is the Beta distribution, is the evidential parameter for optimization, is the optimized evidential parameter; B(p i |<1,1>) is a Beta distribution with parameter 1.
[0041] The numerical evaluation of the classification results described in step S2.5 includes calculating the predicted probability, confidence and uncertainty of the classification results; the specific steps are as follows:
[0042] The following formula is used to calculate the predicted probability of medical image classification:
[0043]
[0044]
[0045] Where p + is the probability of abnormality; p - is the normal probability; α, β are evidential parameters; E is the total amount of evidence, and E = α + β;
[0046] The prediction result is calculated using the following formula: + and p - Confidence of:
[0047]
[0048]
[0049] Where c + is the confidence level of the probability that the predicted result is abnormal; c - is the confidence level of the probability that the predicted result is normal;
[0050] Finally, the confidence u of the probability that the prediction result is uncertain is calculated as u = 1-c + -c- .
[0051] The present invention also provides a visual evidence marking method including the above-mentioned evidence reasoning method for medical image classification, which also includes the following steps:
[0052] S5. According to the classification result obtained in step S4, the evidence corresponding to the classification result is visualized, and the evidence is visually marked and output, so as to obtain a medical image with visual evidence marks.
[0053] The evidence reasoning method and visual evidence marking method for medical image classification provided by the present invention integrate evidence reasoning into a deep learning framework for the first time, and can be used to improve the interpretability and generalization of medical image classification. At the same time, a new explicit evidence reasoning paradigm is proposed to calculate the classification results of medical images. Therefore, the evidence reasoning method of the present invention has good interpretability, high reliability and good accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 Schematic diagram of the method flow of the evidence reasoning method of the present invention.
[0055] Figure 2 The figure is a flow chart of the visual evidence marking method of the present invention.
[0056] Figure 3 It is a schematic diagram of the marking results of the visual evidence marking method of the present invention. DETAILED DESCRIPTION
[0057] like Figure 1 The figure shows a schematic diagram of the method flow of the evidence reasoning method of the present invention: the evidence reasoning method for medical image classification provided by the present invention comprises the following steps:
[0058] S1. Obtain existing medical images, and mark them one-to-one with the real labels of each medical image to obtain a training data set, and then extract the feature map of the medical image; specifically, the existing medical images are divided into normal and abnormal, and the classification results are used as the real labels, which are one-to-one correspondence with the medical images to construct a training data set, and then the Resnet50 skeleton network is used to extract the feature map of the medical image;
[0059] S2. Constructing a primitive model for evidential reasoning for medical image classification:
[0060] Constructing an Evidence Reasoning Model:
[0061] S2.1. The feature map corresponding to the medical image is converted into a symbolic node by using feature aggregation voting; a symbolic node represents a class of medical image attributes; specifically, the following steps are included:
[0062] Aggregate voting uses each pixel value of the feature map F to vote for each node with different voting weights, and then each node aggregates the voting information of all pixel values, and the result represents each medical image attribute; specifically, the following formula is used for calculation:
[0063]
[0064] Where P n is the attribute of each type of medical image; H is the length of the feature map F; W is the height of the feature map F; F i is a local feature; W g is the dimension conversion matrix; is the local feature F i For each node N n The weight of the vote, and v n is the trainable voting weight, N is the number of medical image attributes;
[0065] The difference from the prior art is that an aggregate voting part is designed; the prior art directly uses the scattered pixel-level low semantic information of the feature map as the basis for diagnosis, which lacks interpretability and cannot be used as evidence for diagnosis; the aggregate voting proposed by the present invention uses each pixel value of the feature map F to vote for each node with different voting weights, and then each node aggregates the voting information of all pixel values, and the result represents each fundus image attribute; the input of feature extraction is a medical image, and the label is at the image level; the Resnet50 network is used for feature extraction to obtain the feature map F; the attribute semantic information of the medical image extracted in this step of the present invention is distributed in the entire feature map, and the accuracy and reliability are higher;
[0066] S2.2. Use the symbolic nodes obtained in step S2.1 as nodes of the graph network, and the correlations between nodes as edges of the graph network to construct a graph network; specifically, the following steps are included:
[0067] Construct a graph network G = {N, ε}, where the medical image attributes are nodes s m , and s m ∈N, N is the number of medical image attributes; the association edge between two medical image attributes is (s m ,s n ), and (s m ,s n )∈ε, ε is the set of associated edges; A is the adjacency matrix between M attributes, and is a matrix of length M and height M; in the initialization phase, the relationship between them is constructed using a fully connected method;
[0068] S2.3. The prior art uses the obtained feature map F for stretching, and then obtains the predicted classification probability of the medical image through the fully connected layer and softmax; however, the prior art models the correlation between scattered pixel-level semantic information and the medical image, which is a data fitting method and is prone to pseudo-correlation, thereby causing errors or mistakes in classification, and the classification process is not very interpretable; therefore, the present invention proposes a dynamic graph reasoning method, which uses the global correlation of the obtained medical image attributes for reasoning, thereby obtaining local medical image attributes and the relationship between the attributes as evidence to support the results of medical image classification; different medical image attributes can be inferred through dynamic graph networks to obtain the relationship between local medical image attributes, and the local medical image attributes and the medical image classification probability are modeled and quantified into evidence to support the results of medical image classification;
[0069] Therefore, the present invention models the correlation of global attributes, and uses the graph network obtained in step S2.2 to perform evidence reasoning to obtain evidence information supporting medical image classification, and quantifies the evidence information into numerical parameters; specifically, the present invention includes the following steps:
[0070] A. Use the following rules to make the graph network have all symbolic nodes s m Belief propagation is performed on ∈N:
[0071]
[0072] Where H t is the feature node information at time t; σ(.) is the RELU activation function; is the normalized degree matrix; H t-1 is the characteristic node information at time t-1; W t-1 is a trainable weight matrix; is the normalized adjacency matrix at time t-1, and A t-1 is the adjacency matrix at time t-1, I M is the identity matrix;
[0073] B. Learn the relationship between medical image attributes, infer the relationship between local attributes and structure, and obtain evidence to support medical image classification; constrain the intrinsic relationship of the adjacency matrix through the following loss function:
[0074]
[0075]
[0076] Where A t is the adjacency matrix at time t, is the i-th row of the normalized adjacency matrix at time t; lr is the learning rate; l is the loss function; is the i-th row of the adjacency matrix at time t; is the minimum value of the i-th row of the adjacency matrix at time t; is the maximum value of the i-th row of the adjacency matrix at time t;
[0077] C. Use the following formula to calculate the evidence vector:
[0078] e - ,e + =MLP(σ(Conv(concat(F,F'))))
[0079] Where e + To support the evidence for classifying a medical image as abnormal, e - Evidence to support the classification of medical images as normal; MLP(.) is a multi-layer perceptron; Conv(.) is an operation using a 1*1 convolution kernel for processing; is the original feature map; F' is the feature map after inference, and is composed of H t Stretch to get H is the length of feature map F', W is the height of feature map F', C1 is the number of channels of F'; concat(.) is the operation function of adding F and F' on the channel, and
[0080] Using the constructed evidence reasoning model, medical images are classified:
[0081] S2.4. Set the original probability distribution of the medical image, and use the quantified evidence information obtained in step S2.3 to optimize the original probability distribution of the medical image; specifically, the steps include:
[0082] Assume that the original probability density of the medical image is Beta distribution B(p i |α i ,β i ), where p i is the classification prediction probability, α i is the evidential parameter, β i is the evidential parameter; the following formula is used to optimize the probability density distribution:
[0083]
[0084]
[0085] Where l D is the loss function; N is the total number of medical images; y i is the true label of the medical image, and y i ∈{0,1};pi is the classification prediction probability; || || is the mean square error; Γ() is the Gamma function; α i is the evidential parameter, and α i =e + +1; β i is the evidential parameter, and β i =e - +1; l is the loss function; λ is a hyperparameter, and λ∈[0,1]; KL(X||Y) is the Kullback-Leibler divergence; is the Beta distribution, is the evidential parameter for optimization, is the optimized evidential parameter; B(p i |<1,1>) is a Beta distribution with parameter 1;
[0086] S2.5. Perform numerical evaluation on the classification results; the specific steps are as follows:
[0087] The following formula is used to calculate the predicted probability of medical image classification:
[0088]
[0089]
[0090] Where p + is the probability of abnormality; p - is the normal probability; α, β are evidential parameters; E is the total amount of evidence, and E = α + β;
[0091] The prediction result is calculated using the following formula: + and p - Confidence of:
[0092]
[0093]
[0094] Where c + is the confidence level of the probability that the predicted result is abnormal; c - is the confidence level of the probability that the predicted result is normal;
[0095] Finally, the confidence u of the probability that the prediction result is uncertain is calculated as u = 1-c + -c - ;
[0096] S3. Using the training data set obtained in step S1, the original evidence reasoning model for medical image classification constructed in step S2 is trained to obtain an evidence reasoning model for medical image classification;
[0097] S4. Use the evidence reasoning model for medical image classification obtained in step S3 to classify the actual medical image to be classified and provide corresponding classification evidence.
[0098] The following further describes the evidence reasoning method of the present invention by taking fundus images as the implementation object:
[0099] The evidence reasoning method for fundus image classification provided by the present invention comprises the following steps:
[0100] S1. Obtain existing fundus images, and mark them one-to-one with the real labels of each fundus image to obtain a training data set, and then extract the feature map of the fundus image; specifically, the existing fundus images are divided into normal and abnormal, and the classification results are used as the real labels, which are one-to-one correspondence with the fundus images to construct a training data set, and then the Resnet50 skeleton network is used to extract the feature map of the fundus image;
[0101] S2. Constructing the original model of evidence reasoning for fundus image classification:
[0102] Constructing an Evidence Reasoning Model:
[0103] S2.1. The feature map corresponding to the fundus image is converted into a symbol node by using feature aggregation voting; a symbol node represents a type of fundus image attribute; specifically, the steps include:
[0104] Aggregate voting uses each pixel value of the feature map F to vote for each node with different voting weights, and then each node aggregates the voting information of all pixel values. The result represents each fundus image attribute (including optic cup, optic disc, vein, artery, etc.); specifically, the following formula is used for calculation:
[0105]
[0106] Where P n is the fundus image attribute of each category; H is the length of the feature map F; W is the height of the feature map F; F i is a local feature; W g is the dimension conversion matrix; is the local feature F i For each node N n The weight of the vote, and v n is the trainable voting weight, N is the number of fundus image attributes;
[0107] S2.2. Use the symbolic nodes obtained in step S2.1 as nodes of the graph network, and the correlations between nodes as edges of the graph network to construct a graph network; specifically, the following steps are included:
[0108] Construct a graph network G = {N, ε}, where fundus image attributes are used as node s m , and s m ∈N, N is the number of fundus image attributes; the associated edge between two fundus image attributes is (s m ,s n ), and (s m ,s n )∈ε, ε is the set of associated edges; A is the adjacency matrix between M attributes, and is a matrix of length M and height M; in the initialization phase, the relationship between them is constructed using a fully connected method;
[0109] S2.3. Model the correlation of global attributes, and use the graph network obtained in step S2.2 to perform evidence reasoning to obtain evidence information supporting fundus image classification, and quantify the evidence information into numerical parameters; different fundus attributes (veins, arteries, optic cups, optic discs, etc.) can be reasoned through dynamic graph networks to obtain the relationship between local fundus attributes (optic cups, optic discs, etc.), and model these local fundus attributes and fundus image classification probabilities, and quantify them into evidence to support the results of fundus image classification; specifically, the following steps are included:
[0110] A. Use the following rules to make the graph network have all symbolic nodes s m Belief propagation is performed on ∈N:
[0111]
[0112] Where H t is the feature node information at time t; σ(.) is the RELU activation function; is the normalized degree matrix; H t-1 is the characteristic node information at time t-1; W t-1 is a trainable weight matrix; is the normalized adjacency matrix at time t-1, and A t-1 is the adjacency matrix at time t-1, I M is the identity matrix;
[0113] B. Learn the relationship between fundus image attributes, infer the relationship between local attributes and structure, and thus obtain evidence to support fundus image classification; constrain the intrinsic relationship of the adjacency matrix through the following loss function:
[0114]
[0115]
[0116] Where A tis the adjacency matrix at time t, is the i-th row of the normalized adjacency matrix at time t; lr is the learning rate; l is the loss function; is the i-th row of the adjacency matrix at time t; is the minimum value of the i-th row of the adjacency matrix at time t; is the maximum value of the i-th row of the adjacency matrix at time t;
[0117] C. Use the following formula to calculate the evidence vector:
[0118] e - ,e + =MLP(σ(Conv(concat(F,F'))))
[0119] Where e + To support the classification of fundus images as abnormal, e - Evidence to support the classification of fundus images as normal; MLP(.) is a multi-layer perceptron; Conv(.) is an operation using a 1*1 convolution kernel for processing; is the original feature map; F' is the feature map after inference, and is composed of H t Stretch to get H is the length of feature map F', W is the height of feature map F', C1 is the number of channels of F'; concat(.) is the operation function of adding F and F' on the channel, and
[0120] The constructed evidence reasoning model is used to classify fundus images:
[0121] S2.4. Set the original probability distribution of the fundus image, and use the quantified evidence information obtained in step S2.3 to optimize the original probability distribution of the fundus image; specifically, the steps include:
[0122] Assume that the original probability density of the fundus image is Beta distribution B(p i |α i ,β i ), where p i is the classification prediction probability, α i is the evidential parameter, β i is the evidential parameter; the following formula is used to optimize the probability density distribution:
[0123]
[0124]
[0125] Where l D is the loss function; N is the total number of fundus images; y iis the true label of the fundus image, and y i ∈{0,1};p i is the classification prediction probability; || || is the mean square error; Γ() is the Gamma function; α i is the evidential parameter, and α i =e + +1; β i is the evidential parameter, and β i =e - +1; l is the loss function; λ is a hyperparameter, and λ∈[0,1]; KL(X||Y) is the Kullback-Leibler divergence; is the Beta distribution, is the evidential parameter for optimization, is the optimized evidential parameter; B(p i |<1,1>) is a Beta distribution with parameter 1;
[0126] S2.5. Perform numerical evaluation on the classification results; the specific steps are as follows:
[0127] The predicted probability of fundus image classification is calculated using the following formula:
[0128]
[0129]
[0130] Where p + is the probability of abnormality; p - is the normal probability; α, β are evidential parameters; E is the total amount of evidence, and E = α + β;
[0131] The prediction result is calculated using the following formula: + and p - Confidence of:
[0132]
[0133]
[0134] Where c + is the confidence level of the probability that the predicted result is abnormal; c - is the confidence level of the probability that the predicted result is normal;
[0135] Finally, the confidence u of the probability that the prediction result is uncertain is calculated as u = 1-c + -c - ;
[0136] S3. Using the training data set obtained in step S1, the original evidence reasoning model for fundus image classification constructed in step S2 is trained to obtain an evidence reasoning model for fundus image classification;
[0137] S4. Using the evidence reasoning model for fundus image classification obtained in step S3, classify the actual fundus image to be classified and provide corresponding classification evidence.
[0138] like Figure 2 The method flow chart of the visual evidence marking method of the present invention is shown as follows: The visual evidence marking method provided by the present invention, which includes the above-mentioned evidence reasoning method for medical image classification, comprises the following steps:
[0139] S1. Obtain existing medical images, and mark them one-to-one with the real labels of each medical image to obtain a training data set, and then extract the feature map of the medical image; specifically, the existing medical images are divided into normal and abnormal, and the classification results are used as the real labels, which are one-to-one correspondence with the medical images to construct a training data set, and then the Resnet50 skeleton network is used to extract the feature map of the medical image;
[0140] S2. Constructing a primitive model for evidential reasoning for medical image classification:
[0141] Constructing an Evidence Reasoning Model:
[0142] S2.1. The feature map corresponding to the medical image is converted into a symbolic node by using feature aggregation voting; a symbolic node represents a class of medical image attributes; specifically, the following steps are included:
[0143] Aggregate voting uses each pixel value of the feature map F to vote for each node with different voting weights, and then each node aggregates the voting information of all pixel values, and the result represents each medical image attribute; specifically, the following formula is used for calculation:
[0144]
[0145] Where P n is the attribute of each type of medical image; H is the length of the feature map F; W is the height of the feature map F; F i is a local feature; W g is the dimension conversion matrix; is the local feature F i For each node N n The weight of the vote, and v n is the trainable voting weight, N is the number of medical image attributes;
[0146] S2.2. Use the symbolic nodes obtained in step S2.1 as nodes of the graph network, and the correlations between nodes as edges of the graph network to construct a graph network; specifically, the following steps are included:
[0147] Construct a graph network G = {N, ε}, where the medical image attributes are nodes s m , and s m ∈N, N is the number of medical image attributes; the association edge between two medical image attributes is (s m ,s n ), and (s m ,s n )∈ε, ε is the set of associated edges; A is the adjacency matrix between M attributes, and is a matrix of length M and height M; in the initialization phase, the relationship between them is constructed using a fully connected method;
[0148] S2.3. Model the correlation of global attributes, and use the graph network obtained in step S2.2 to perform evidence reasoning to obtain evidence information supporting medical image classification, and quantify the evidence information into numerical parameters; specifically, the steps include:
[0149] A. Use the following rules to make the graph network have all symbolic nodes s m Belief propagation is performed on ∈N:
[0150]
[0151] Where H t is the feature node information at time t; σ(.) is the RELU activation function; is the normalized degree matrix; H t-1 is the characteristic node information at time t-1; W t-1 is a trainable weight matrix; is the normalized adjacency matrix at time t-1, and A t-1 is the adjacency matrix at time t-1, I M is the identity matrix;
[0152] B. Learn the relationship between medical image attributes, infer the relationship between local attributes and structure, and obtain evidence to support medical image classification; constrain the intrinsic relationship of the adjacency matrix through the following loss function:
[0153]
[0154]
[0155] Where A t is the adjacency matrix at time t, is the i-th row of the normalized adjacency matrix at time t; lr is the learning rate; l is the loss function; is the i-th row of the adjacency matrix at time t; is the minimum value of the i-th row of the adjacency matrix at time t; is the maximum value of the i-th row of the adjacency matrix at time t;
[0156] C. Use the following formula to calculate the evidence vector:
[0157] e - ,e + =MLP(σ(Conv(concat(F,F'))))
[0158] Where e + To support the evidence for classifying a medical image as abnormal, e - Evidence to support the classification of medical images as normal; MLP(.) is a multi-layer perceptron; Conv(.) is an operation using a 1*1 convolution kernel for processing; is the original feature map; F' is the feature map after inference, and is composed of H t Stretch to get H is the length of feature map F', W is the height of feature map F', C1 is the number of channels of F'; concat(.) is the operation function of adding F and F' on the channel, and
[0159] Using the constructed evidence reasoning model, medical images are classified:
[0160] S2.4. Set the original probability distribution of the medical image, and use the quantified evidence information obtained in step S2.3 to optimize the original probability distribution of the medical image; specifically, the steps include:
[0161] Assume that the original probability density of the medical image is Beta distribution B(p i |α i ,β i ), where p i is the classification prediction probability, α i is the evidential parameter, β i is the evidential parameter; the following formula is used to optimize the probability density distribution:
[0162]
[0163]
[0164] Where l D is the loss function; N is the total number of medical images; y i is the true label of the medical image, and y i ∈{0,1};pi is the classification prediction probability; || || is the mean square error; Γ() is the Gamma function; α i is the evidential parameter, and α i =e + +1; β i is the evidential parameter, and β i =e - +1; l is the loss function; λ is a hyperparameter, and λ∈[0,1]; KL(X||Y) is the Kullback-Leibler divergence; is the Beta distribution, is the evidential parameter for optimization, is the optimized evidential parameter; B(p i |<1,1>) is a Beta distribution with parameter 1;
[0165] S2.5. Perform numerical evaluation on the classification results; the specific steps are as follows:
[0166] The following formula is used to calculate the predicted probability of medical image classification:
[0167]
[0168]
[0169] Where p + is the probability of abnormality; p - is the normal probability; α, β are evidential parameters; E is the total amount of evidence, and E = α + β;
[0170] The prediction result is calculated using the following formula: + and p - Confidence of:
[0171]
[0172]
[0173] Where c + is the confidence level of the probability that the predicted result is abnormal; c - is the confidence level of the probability that the predicted result is normal;
[0174] Finally, the confidence u of the probability that the prediction result is uncertain is calculated as u = 1-c + -c - ;
[0175] S3. Using the training data set obtained in step S1, the original evidence reasoning model for medical image classification constructed in step S2 is trained to obtain an evidence reasoning model for medical image classification;
[0176] S4. Using the evidence reasoning model for medical image classification obtained in step S3, classify the actual medical image to be classified and provide corresponding classification evidence;
[0177] S5. According to the classification result obtained in step S4, the evidence corresponding to the classification result is visualized, and the evidence is visually marked and output, so as to obtain a medical image with visual evidence marks.
[0178] The visual evidence marking method provided by the present invention is used to mark visual evidence on the fundus image. The marking result is as follows: Figure 3 As shown: Figure 3 (a) is the original fundus image obtained. Figure 3 The highlighted part marked in (b) is the optic disc; Figure 3 The highlighted part marked in (c) is the optic cup part; Figure 3 The highlighted part marked in (d) is the arterial part; Figure 3 The highlighted part in (e) is the vein. Figure 3 It can be seen that the visual evidence marking method of the present invention can perform better evidence marking on medical images (taking fundus images as an example).
[0179] The visual evidence marking method proposed in the present invention can be used in medical image (taking fundus image as an example) imaging devices in existing medical equipment; in specific implementation, the medical image (taking fundus image as an example) imaging device normally obtains a medical image (taking fundus image as an example) of a person; then the visual evidence marking method provided in the present invention is used to classify the obtained medical image (taking fundus image as an example), and visually mark the evidence (such as in the form of highlighting, rectangular frame marking, etc.); then the obtained classification results and marking results are secondary imaged on the medical image (taking fundus image as an example), and a medical image (taking fundus image as an example) containing the classification results and evidence marking results is obtained; finally, the medical image (taking fundus image as an example) with the classification results and marking results is output; at this time, the medical image (taking fundus image as an example) imaging device becomes a medical image (taking fundus image as an example) device with more functions.
Claims
1. An evidential reasoning method for medical image classification, comprising the following steps: S1. Obtain existing medical images, and mark them one-to-one with the real labels of each medical image to obtain a training data set, and then extract feature maps of the medical images; S2. Constructing a primitive model for evidential reasoning for medical image classification: Constructing an Evidence Reasoning Model: S2.
1. The feature graph corresponding to the medical image is converted into a symbolic node by feature aggregation voting; a symbolic node represents a class of medical image attributes; S2.
2. Use the symbolic nodes obtained in step S2.1 as nodes of the graph network, and the correlations between nodes as edges of the graph network to construct a graph network; S2.
3. Model the correlation of global attributes, and use the graph network obtained in step S2.2 to perform evidence reasoning to obtain evidence information supporting medical image classification, and quantify the evidence information into numerical parameters; Using the constructed evidence reasoning model, medical images are classified: S2.
4. Set the original probability distribution of the medical image, and use the quantified evidence information obtained in step S2.3 to optimize the original probability distribution of the medical image; S2.
5. Perform numerical evaluation on the classification results; S3. Using the training data set obtained in step S1, the original evidence reasoning model for medical image classification constructed in step S2 is trained to obtain an evidence reasoning model for medical image classification; S4. Use the evidence reasoning model for medical image classification obtained in step S3 to classify the actual medical image to be classified and provide corresponding classification evidence.
2. The evidential reasoning method for medical image classification according to claim 1, characterized in that Step S1 obtains existing medical images and marks them one-to-one with the real labels of each medical image to obtain a training data set, and then extracts the feature maps of the medical images. Specifically, the existing medical images are divided into normal and abnormal, and the classification results are used as real labels to correspond one-to-one with the medical images, thereby constructing a training data set, and then the Resnet50 skeleton network is used to extract the feature maps of the medical images.
3. The evidential reasoning method for medical image classification according to claim 2 is characterized in that The feature aggregation voting method described in step S2.1 converts the feature map corresponding to the medical image into a symbol node; a symbol node represents a type of medical image attribute, specifically including the following steps: Aggregate voting uses each pixel value of the feature map F to vote for each node with different voting weights, and then each node aggregates the voting information of all pixel values, and the result represents each medical image attribute; specifically, the following formula is used for calculation: Where P n is the attribute of each type of medical image; H is the length of the feature map F; W is the height of the feature map F; F i is a local feature; W g is the dimension conversion matrix; is the local feature F i For each node N n The weight of the vote, and v n is the trainable voting weight, and N is the number of medical image attributes.
4. The evidential reasoning method for medical image classification according to claim 3 is characterized in that Step S2.2 uses the symbolic nodes obtained in step S2.1 as nodes of the graph network, and uses the correlations between nodes as edges of the graph network to construct the graph network, which specifically includes the following steps: Construct a graph network G = {N, ε}, where the medical image attributes are nodes s m , and s m ∈N, N is the number of medical image attributes; the association edge between two medical image attributes is (s m ,s n ), and (s m ,s n )∈ε, ε is the set of associated edges; A is the adjacency matrix between M attributes, and is a matrix of length M and height M.
5. The evidential reasoning method for medical image classification according to claim 4 is characterized in that The correlation of global attributes described in step S2.3 is modeled, and the graph network obtained in step S2.2 is used for evidence reasoning to obtain evidence information supporting medical image classification, and the evidence information is quantified into numerical parameters, which specifically includes the following steps: A. Use the following rules to make the graph network have all symbolic nodes s m Belief propagation is performed on ∈N: Where H t is the feature node information at time t; σ(.) is the RELU activation function; is the normalized degree matrix; H t-1 is the characteristic node information at time t-1; W t-1 is a trainable weight matrix; is the normalized adjacency matrix at time t-1, and A t-1 is the adjacency matrix at time t-1, I M is the identity matrix; B. Learn the relationship between medical image attributes, infer the relationship between local attributes and structure, and obtain evidence to support medical image classification; constrain the intrinsic relationship of the adjacency matrix through the following loss function: Where A t is the adjacency matrix at time t, is the i-th row of the normalized adjacency matrix at time t; lr is the learning rate; l is the loss function; is the i-th row of the adjacency matrix at time t; is the minimum value of the i-th row of the adjacency matrix at time t; is the maximum value of the i-th row of the adjacency matrix at time t; C. Use the following formula to calculate the evidence vector: from - ,and + =MLP(σ(Conv(concat(F,F')))) Where e + To support the evidence for classifying medical images as abnormal, e - Evidence to support the classification of medical images as normal; MLP(.) is a multi-layer perceptron; Conv(.) is an operation using a 1*1 convolution kernel for processing; is the original feature map; F' is the feature map after inference, and is composed of H t Stretch H is the length of feature map F', W is the height of feature map F', C1 is the number of channels of F'; concat(.) is the operation function of adding F and F' on the channel, and 6. The evidential reasoning method for medical image classification according to claim 5, characterized in that The step S2.4 of setting the original probability distribution of the medical image and optimizing the original probability distribution of the medical image using the quantized evidence information obtained in step S2.3 specifically includes the following steps: Assume that the original probability density of the medical image is Beta distribution B(p i |α i ,β i ), where p i is the classification prediction probability, α i is the evidential parameter, β i is the evidential parameter; the following formula is used to optimize the probability density distribution: Where l D is the loss function; N is the total number of medical images; y i is the true label of the medical image, and y i ∈{0,1};p i is the classification prediction probability; || || is the mean square error; Γ() is the Gamma function; α i is the evidential parameter, and α i =e + +1; β i is the evidential parameter, and β i =e - +1; l is the loss function; λ is a hyperparameter, and λ∈[0,1]; KL(X||Y) is the Kullback-Leibler divergence; is the Beta distribution, is the evidential parameter for optimization, is the optimized evidential parameter; B(p i |<1,1>) is a Beta distribution with parameter 1.
7. The evidential reasoning method for medical image classification according to claim 6, characterized in that The numerical evaluation of the classification results described in step S2.5 includes calculating the predicted probability, confidence and uncertainty of the classification results; the specific steps are as follows: The following formula is used to calculate the predicted probability of medical image classification: Where p + is the probability of abnormality; p - is the normal probability; α, β are evidential parameters; E is the total amount of evidence, and E = α + β; The prediction result is calculated using the following formula: + and p - Confidence of: Where c + is the confidence level of the probability that the predicted result is abnormal; c - is the confidence level of the probability that the predicted result is normal; Finally, the confidence u of the probability that the prediction result is uncertain is calculated as u = 1-c + -c - .
8. A visual evidence marking method comprising the evidence reasoning method for medical image classification according to any one of claims 1 to 7, characterized in that The following steps are also included: S5. According to the classification result obtained in step S4, the evidence corresponding to the classification result is visualized, and the evidence is visually marked and output, so as to obtain a medical image with visual evidence marks.