A medical image recognition method based on AI

Abstract

The present invention relates to the field of medical image processing technology, and discloses an AI-based medical image recognition method, comprising the following steps: S1, obtaining multimodal medical image data, and preprocessing the image, the steps including: denoising, contrast enhancement, super-resolution reconstruction, and key area segmentation to highlight the area related to the disease; S2, using a multi-expert annotation system to annotate the image, and generating a synthetic image through a generative adversarial network to expand the data set; S3, using a multimodal fusion model architecture combining a convolutional neural network and CapsuleNetworks. Through the multimodal fusion model, different types of medical images are used in combination, and an adaptive weight allocation mechanism is used to dynamically adjust the influence weight of each modality according to the feature similarity between the modalities, which can make full use of the complementary information of multiple image data, improve the accuracy of the recognition results and the ability to identify multiple types of diseases, and enhance the robustness of the model in different environments and equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a medical image recognition method based on AI. Background Art

[0002] With the development of artificial intelligence technology, the application of AI in medical image recognition has gradually become a research hotspot. Medical image recognition technology can assist clinicians in disease diagnosis and treatment decisions by analyzing a variety of imaging data from X-rays, CT scans, MRI, ultrasound, etc. Existing medical image recognition methods usually rely on single-modality image data, such as using CT or MRI for disease detection and classification. The detection and classification methods are easily limited by the information of a single data modality, resulting in low recognition capabilities for complex disease types. In order to address this problem, a multimodal fusion solution is proposed. By integrating multiple types of image data, it attempts to use complementary information between different modalities to improve recognition accuracy. However, how to effectively fuse multimodal data, fully tap its complementary features, and ensure the generalization capabilities of the model in different devices and environments is an important challenge.

[0003] Chinese Patent Publication No.: CN109859836B discloses a medical image recognition method and device, the method comprising: obtaining a medical image; using a first machine learning model to classify the medical image to obtain a first vector, the first vector indicating a first confidence level that the type of the medical image is healthy or abnormal; using a second machine learning model to classify the medical image to obtain a second vector, the second vector indicating a second confidence level that the type of the medical image is various disease types. The method improves the disease detection capability by combining different models, but does not introduce an adaptive weight allocation mechanism when processing multimodal data fusion, and cannot dynamically adjust the influence weight of each modality according to the feature similarity between different modalities, resulting in the inability to fully utilize the complementary information of multiple image data, affecting the accuracy of the recognition results and the ability to identify multiple disease types. In addition, the method in CN109859836B does not involve explainable AI technology, which makes it difficult for clinicians to understand the prediction basis of the model, thereby limiting the clinical application of the model and the trust and adoption rate of doctors in AI decisions.

[0004] In addition, existing technologies generally lack a feedback mechanism for human-machine collaboration. The errors and deviations shown by AI models in actual clinical applications need to be corrected through manual labeling and offline adjustment of model parameters. This method is inefficient and difficult to reflect changes in clinical needs in real time, which in turn limits the application value and adaptability of AI models in actual clinical environments.

[0005] Therefore, those skilled in the art provide an AI-based medical image recognition method to solve the problems raised in the above background technology. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides an AI-based medical image recognition method to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: A medical image recognition method based on AI, comprising the following steps: S1, acquiring multimodal medical image data, and preprocessing the image, the steps including: denoising, contrast enhancement, super-resolution reconstruction, and key area segmentation to highlight the area related to the disease;

[0008] S2, use a multi-expert annotation system to annotate images, and generate synthetic images through a generative adversarial network to expand the dataset;

[0009] S3, using a multimodal fusion model architecture that combines convolutional neural networks and capsule networks, and using the following formula to achieve modal fusion:

[0010] F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 ,

[0011] Among them, α, β, and γ are adaptive weights, which are dynamically adjusted by the following formula:

[0012]

[0013] Among them, k is the adjustment parameter, b is the bias, and the feature similarity is calculated based on the feature similarity between the modalities;

[0014] S4. Through transfer learning, the model pre-trained on the dataset is migrated to a specific medical task. Combined with continuous learning strategies, fairness constraint technology is used to optimize the model so that it performs consistently across different populations.

[0015] S5. Integrate explainable AI technology to generate visual explanations to highlight key areas in model decisions, and quantify the strength of the explanation through the following formula:

[0016]

[0017] Among them, w i is the weight and contribution of the i-th feature i The contribution of this feature to the model output;

[0018] S6. Validate the model in a multi-center clinical setting, collect feedback from clinicians, and introduce adversarial training and differential privacy techniques to enhance the robustness of the model in different devices and environments.

[0019] S7. Use a third machine learning model to classify the fused feature vector, and the output includes the confidence of health, abnormality, and specific disease types, and the interpretable output is combined in the decision-making process;

[0020] S8. Develop a human-machine collaboration platform that allows clinicians to interact with the model, check and adjust the model's predictions, and the model records the learning physician's feedback;

[0021] S9. Establish an automated feedback mechanism to monitor the difference between the model's predictions and the doctor's diagnosis results in real time, and make dynamic adjustments through the following algorithm formula:

[0022]

[0023] Among them, η is the learning rate, λ is the feedback adjustment coefficient, and θ is the model parameter;

[0024] S10. Use federated learning technology and differential privacy algorithms to train and update models without exposing patient data.

[0025] Preferably, the key area segmentation is achieved by combining a convolutional neural network with a U-Net architecture;

[0026] The architecture of combining the convolutional neural network with U-Net is specifically implemented as follows:

[0027] Convolution operation: A series of convolution operations are performed on the input image I to extract features:

[0028] F (l) =σ(W (l) *F (l-1) +b (l) ),

[0029] Among them, F (l) is the feature map of the lth layer, W (l) is the convolution kernel of the lth layer, b (l w is the bias, * represents the convolution operation, and σ is the activation function;

[0030] Downsampling and upsampling of U-Net architecture:

[0031] Downsampling path: extract multi-level features through convolutional layers and maximum pooling layers:

[0032]

[0033] Upsampling path: restore image resolution through deconvolution layers and skip connections, and focus on the target area:

[0034]

[0035] in, and is the convolution kernel of the encoder and decoder parts, UpSample is the upsampling operation, and Feature maps for downsampling and upsampling paths;

[0036] Final output: Through a 1×11×1 convolutional layer, the output of the upsampling path is mapped to the target category space to achieve region segmentation:

[0037]

[0038] Among them, W out is the convolution kernel of the output layer, b out is the bias of the output layer, O is the final segmentation result output, and the result is converted into a probability map through the Softmax activation function.

[0039] Preferably, the synthetic data generated by the generative adversarial network includes multiple disease types, and the most representative synthetic image is determined by a Bayesian optimization algorithm.

[0040] Preferably, the architecture of the generative adversarial network generation mechanism and Bayesian optimization strategy is specifically implemented as follows:

[0041] Synthetic data generation by generative adversarial networks: Generative adversarial networks consist of a generator G and a discriminator D. Its goal is to generate synthetic images through the generator and distinguish between real images and generated images through the discriminator.

[0042] Generator loss function:

[0043] Among them, z is the random noise vector input by the generator, and the distribution p z (z), G(z) is the synthetic image output by the generator, and D(G(z)) is the probability that the discriminator determines that the generated image is a real image;

[0044] Discriminator loss function:

[0045]

[0046] Among them, x is the real image sample, p data (x) is the distribution of real data, and D(x) is the probability of the discriminator distinguishing the real image;

[0047] The goal of the generative adversarial network is to make the images generated by the generator realistic by alternately optimizing the generator and the discriminator;

[0048] Bayesian optimization is used to optimize synthetic images: A Bayesian optimization strategy is used to optimize representative image samples in the synthetic image space, and the synthetic images are determined by modeling the probability distribution of the objective function;

[0049] Objective function modeling: y = f(x) + ∈,

[0050] Among them, f(x) is the potential objective function, x is the generated synthetic image feature, y is the evaluation index, and ∈ is the observation noise;

[0051] Bayesian optimization process:

[0052] Determine the new sampling point x + By maximizing expected improvement:

[0053]

[0054] Among them, f(x + ) is the optimal objective function value;

[0055] According to the new sampling points, the Gaussian process model is updated to adjust the probability distribution estimate of the objective function;

[0056] Through iterative Bayesian optimization steps, the synthetic image with the highest expected improvement is finally determined as the most representative sample;

[0057] The synthetic images determined by Bayesian optimization are used to expand the training dataset.

[0058] Preferably, the multimodal fusion of the model is achieved through an adaptive weight allocation mechanism to dynamically adjust the weight of each modality according to the feature similarity of different modal inputs;

[0059] The fusion of multimodal features and adaptive weight adjustment are specifically implemented as follows:

[0060] Multimodal feature extraction: Extract features from input data of different modes. Assume that the features of each mode are expressed as: F CNN , F Capsule , F 其他模态 ,…, each feature F i It is a high-dimensional feature vector obtained by convolutional neural network or other feature extraction models;

[0061] Feature similarity calculation: Calculate the feature similarity between different modalities, using cosine similarity, Euclidean distance or other similarity metrics. The cosine similarity algorithm formula is:

[0062]

[0063] Among them, F i ,Fj Indicates F i and F j The inner product of ∥F i ∥ and ∥F j ∥ is F i and F j The norm of ;

[0064] Adaptive weight calculation: Calculate the adaptive weight based on feature similarity. Assuming there are n modal inputs, the weight α i , calculated by the following formula:

[0065] Among them, λ is the adjustment parameter, Sim(F i ,F j ) is the feature similarity of mode i and mode j. This formula normalizes the weight of each mode through the Softmax function;

[0066] Multimodal feature fusion: weighted fusion of modal features to obtain the final fusion feature F 融合 : F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 ,

[0067] Among them, α, β, γ are adaptive weights, F 融合 represents the final fusion feature, F CNN represents the features extracted by convolutional neural network, F Capsule represents the features extracted by the capsule network, F 其他模态 Represent features from other modalities;

[0068] Fusion features are used for classification: The fused features F 融合 The input is fed into the subsequent classification model for final classification, and the output includes the confidence level of health, abnormality, and specific disease types.

[0069] Preferably, the interpretable output is visualized by generating a heat map and compared with the clinician's annotations to further verify and optimize the decision path of the model;

[0070] The generation of a heat map and quantification of its interpretability are specifically implemented as follows:

[0071] Feature map extraction: Feature map A is extracted from the middle layer of the convolutional neural network k , where k represents the channel index of the feature map. Assume that the dimension of the feature map is H×W×K, where H is the height, W is the width, and K is the number of channels;

[0072] Gradient calculation of category score and feature map: For the score yc of the target category c, calculate the gradient of the category score to the feature map: This gradient represents the sensitivity of category c to feature map channel k;

[0073] Feature map weight calculation: Use global average pooling to calculate the weight of each feature map channel Used to measure the contribution of each channel to the final decision: in, Represents the activation value of the i-th, j-th position in the feature map;

[0074] Generate class activation maps: By combining the feature map with the weights Perform weighted summation to generate the final class activation map: The mapping Show the contribution of the region in the image to the prediction of category c;

[0075] Heatmap normalization and visualization: The generated class activation maps are normalized so that their values ​​range from 0 to 1 for visualization: Normalized mapping Can be superimposed on the original image to form a heat map for visualization and explanation of model decisions;

[0076] Explainability quantification and comparison with doctor annotations: The overlap between model interpretability and doctor annotations is quantified by the following formula: Among them, IOU represents the overlap ratio between the hotspot map and the doctor's annotation area, which is used to verify the reliability of the model interpretation;

[0077] Optimize the model decision path: Adjust and optimize the model based on the comparison results with the doctor's annotations. If the overlap is lower than the preset threshold T, the interpretability can be improved by fine-tuning the model parameters or adjusting the feature map extraction level.

[0078] Preferably, the third machine learning model uses a support vector machine or a logistic regression algorithm to classify the fused feature vector.

[0079] Preferably, the classification process of the support vector machine and logistic regression is specifically implemented as follows:

[0080] Input feature vector: the fused feature vector F obtained in the multimodal fusion step 融合 As input to the classification model: F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 ,

[0081] Among them, α, β, γ are adaptive weights, F 融合 represents the final fusion feature, FCNN represents the features extracted by convolutional neural network, F Capsule represents the features extracted by the capsule network, F 其他模态 Represent features from other modalities;

[0082] Support Vector Machine Classification:

[0083] Support vector machine decision function: The support vector machine divides the data through the optimal hyperplane, which is defined by the weight vector w and the bias b. Its decision function is:

[0084] f(x)=w·F 融合 + b,

[0085] Classification: Classify the input according to the sign of the decision function:

[0086] y=sign(f(x))=sign(w·F 融合 +b),

[0087] Among them, y=1 indicates abnormality, and y=-1 indicates health;

[0088] Optimization objective: The support vector machine determines the optimal hyperplane by optimizing the following objective function:

[0089]

[0090] Where C is the regularization parameter;

[0091] Logistic Regression Classification:

[0092] Logistic regression decision function: Logistic regression uses the sigmoid function to map input features to probability values. Its decision function is: p(y=1|F 融合 )=σ(w·F 融合 +b),

[0093] in, is the sigmoid function;

[0094] Classification: Classify the input according to the probability output of the decision function:

[0095]

[0096] Among them, τ is the classification threshold, y=1 indicates abnormality, and y=0 indicates health.

[0097] Preferably, the human-computer collaboration platform includes an interactive interface for doctors to provide real-time feedback on model outputs and automatically update model parameters to reflect the doctor's adjustments.

[0098] Preferably, the feedback mechanism dynamically adjusts the model parameters, which is specifically implemented as follows:

[0099] Real-time feedback collection: When doctors provide feedback on the model’s output results through the interactive interface, the actual feedback label y is recorded 真实 and the model's predicted label y 预测 , calculate the error:

[0100] ∈=y 真实 -y 预测 , where y 真实 is the real label annotated by the doctor, y 预测 is the predicted label output by the model;

[0101] Update of loss function: Define the loss function after feedback improvement, and incorporate the doctor's feedback error into the loss calculation to guide the parameter update of the model:

[0102] Among them, L 原始 (θ) is the original loss function of the model, λ is the weight coefficient of feedback adjustment, and N is the number of feedback samples;

[0103] Gradient update rule: According to the new loss function L 反馈 (θ), update the model parameters θ by gradient descent: Where η is the learning rate, is the gradient of the loss function with respect to the parameter θ;

[0104] Dynamic learning rate adjustment: Dynamically adjust the learning rate according to the size of the feedback error to quickly adapt to the doctor's feedback: η t+1 =η t ·(1+β·|∈|), where β is the dynamic adjustment coefficient;

[0105] Application of update rules: For each feedback, follow The formula is updated with parameters and continuously iterated, and the model adjusts its output to better match the doctor’s judgment:

[0106]

[0107] Through feedback loops, the model can adjust adaptively;

[0108] Parameter convergence and verification: Through continuous feedback and parameter update, if the feedback error ∈ converges to a certain threshold δ, stop parameter update and evaluate model performance through verification:

[0109] Stop condition: |∈|<δ.

[0110] The present invention provides an AI-based medical image recognition method, which has the following beneficial effects:

[0111] 1. The present invention combines different types of medical images through a multimodal fusion model, adopts an adaptive weight allocation mechanism, and dynamically adjusts the influence weight of each modality according to the feature similarity between the modalities. It can then make full use of the complementary information of multiple image data, improve the accuracy of the recognition results and the ability to identify multiple disease types, and enhance the robustness of the model in different environments and equipment.

[0112] 2. The present invention integrates explainable AI technology to generate visual explanations for each prediction result, highlighting the key areas in the model decision-making. Its function makes the model output more transparent, allowing clinicians to better understand the model's decision-making process, thereby increasing trust and adoption rate, and facilitating the realization of human-computer collaborative diagnosis.

[0113] 3. The present invention develops a human-computer collaboration platform and an automated feedback mechanism, allowing clinicians to check and adjust the prediction results of the model in real time, and then use the feedback for continuous optimization of the model. This mechanism can improve the accuracy of the model, enable the model to continuously learn and adapt to clinical needs, and thus enhance clinical practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0115] In order to make the technical personnel in the technical field understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in combination with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a partial embodiment of the present invention, not a complete embodiment. Based on the embodiment of the present invention, other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0116] The present invention is described in detail below in conjunction with the accompanying drawings:

[0117] Example:

[0118] Please see attached Figure 1 , an embodiment of the present invention provides an AI-based medical image recognition method, comprising the following steps: S1, acquiring multimodal medical image data, and preprocessing the image, the steps comprising: denoising, contrast enhancement, super-resolution reconstruction, and key area segmentation to highlight disease-related areas;

[0119] S2, use a multi-expert annotation system to annotate images, and generate synthetic images through a generative adversarial network to expand the dataset;

[0120] S3, using a multimodal fusion model architecture that combines convolutional neural networks and capsule networks, and using the following formula to achieve modal fusion:

[0121] F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 ,

[0122] Among them, α, β, and γ are adaptive weights, which are dynamically adjusted by the following formula:

[0123]

[0124] Among them, k is the adjustment parameter, b is the bias, and the feature similarity is calculated based on the feature similarity between the modalities;

[0125] S4. Through transfer learning, the model pre-trained on the dataset is migrated to a specific medical task. Combined with continuous learning strategies, fairness constraint technology is used to optimize the model so that it performs consistently across different populations.

[0126] S5. Integrate explainable AI technology to generate visual explanations to highlight key areas in model decisions, and quantify the strength of the explanation through the following formula:

[0127]

[0128] Among them, w i is the weight and contribution of the i-th feature i The contribution of this feature to the model output;

[0129] S6. Validate the model in a multi-center clinical setting, collect feedback from clinicians, and introduce adversarial training and differential privacy techniques to enhance the robustness of the model in different devices and environments.

[0130] S7. Use a third machine learning model to classify the fused feature vector, and the output includes the confidence of health, abnormality, and specific disease types, and the interpretable output is combined in the decision-making process;

[0131] S8. Develop a human-machine collaboration platform that allows clinicians to interact with the model, check and adjust the model's predictions, and the model records the learning physician's feedback;

[0132] S9. Establish an automated feedback mechanism to monitor the difference between the model's predictions and the doctor's diagnosis results in real time, and make dynamic adjustments through the following algorithm formula:

[0133]

[0134] Among them, η is the learning rate, λ is the feedback adjustment coefficient, and θ is the model parameter;

[0135] S10. Use federated learning technology and differential privacy algorithms to train and update models without exposing patient data.

[0136] S1 The benefits of obtaining multimodal medical image data and preprocessing it can help highlight disease-related areas, reduce noise interference, and improve the model's sensitivity to subtle features in the image; S2 The benefits of using a multi-expert annotation system for annotation and expanding the data set can increase the diversity and scale of the data, help alleviate the problems of insufficient and imbalanced data, and enhance the generalization ability of the model so that it can more accurately identify different types of diseases; S3 The benefits of multimodal fusion model architecture and adaptive weight adjustment can better integrate multiple image data, enhance the model's ability to identify different diseases, improve the accuracy of recognition results, and maintain robust performance in different environments and devices; S4 The benefits of transfer learning and continuous learning strategies combined with fairness constraints can significantly shorten the training time for specific medical tasks, improve the initial performance of the model, reduce bias, and improve adaptability and fairness to patient groups; S5 The benefits of integrating explainable AI technology and quantifying the strength of explanation can further verify The rationality of model decisions can enhance doctors’ trust in and willingness to use AI, and promote human-machine collaborative diagnosis; S6 The benefits of model verification, adversarial training, and differential privacy technology can improve the robustness of the model to abnormal situations and different environments. Differential privacy technology ensures the protection of patient privacy during data sharing and model updating, and achieves safe and compliant application of the model; S7 The benefits of combining the third machine learning model classification with explainable output enable clinicians to make diagnostic decisions based on clear explanations and reduce concerns about black box models; S8 The benefits of human-machine collaborative platform development and feedback records can improve accuracy and practicality; S9 The benefits of automated feedback mechanism and dynamic adjustment enable the model to adapt to changes quickly, reduce the time cost of manual intervention, and improve the response speed and efficiency of the model in clinical practice; S10 The benefits of federated learning technology and differential privacy algorithm can improve the generalization ability and performance of the model, meet legal and regulatory and ethical requirements, and promote the widespread application of AI in the medical field.

[0137] Key area segmentation is achieved through the architecture combining convolutional neural network and U-Net;

[0138] The architecture of the convolutional neural network combined with U-Net is implemented as follows:

[0139] Convolution operation: A series of convolution operations are performed on the input image I to extract features:

[0140] F (l) =σ(W (l) *F (l-1) +b (l)),

[0141] Among them, F (l) is the feature map of the lth layer, W (l) is the convolution kernel of the lth layer, b (l) is the bias, * represents the convolution operation, and σ is the activation function;

[0142] Downsampling and upsampling of U-Net architecture:

[0143] Downsampling path: extract multi-level features through convolutional layers and maximum pooling layers:

[0144]

[0145] Upsampling path: restore image resolution through deconvolution layers and skip connections, and focus on the target area:

[0146]

[0147] in, and is the convolution kernel of the encoder and decoder parts, UpSample is the upsampling operation, and Feature maps for downsampling and upsampling paths;

[0148] Final output: Through a 1×11×1 convolutional layer, the output of the upsampling path is mapped to the target category space to achieve region segmentation:

[0149]

[0150] Among them, W out is the convolution kernel of the output layer, b out is the bias of the output layer, O is the final segmentation result output, and the result is converted into a probability map through the Softmax activation function.

[0151] The formulas are combined to form a complete convolutional neural network and U-Net architecture for key area segmentation. Its overall function is to achieve accurate segmentation of the input image through feature extraction, feature fusion of downsampling and upsampling paths, and final output mapping. Specifically:

[0152] Through convolution operations, the model can extract rich features from the input image, including low-level details and high-level abstract information; reduce the spatial resolution of the image and extract features of different scales. The features provide fine-grained information in the jump connection, which helps to preserve spatial details; restore the original resolution of the image through deconvolution operations, and integrate low-level and high-level features through jump connections, so that the model can focus on key segmentation areas; through 1×11×1 convolution and Softmax activation, the features of the upsampling path are converted into category probability maps to achieve accurate segmentation output;

[0153] Overall, the role of the algorithm formula is to form an efficient segmentation model, which can fully utilize the feature extraction capabilities of convolutional neural networks and the segmentation structure advantages of U-Net to perform high-precision segmentation of key areas in medical images. This process greatly improves the accuracy and reliability of the model in detecting different diseases, providing a solid foundation for subsequent AI diagnosis and clinical applications.

[0154] The synthetic data generated by the generative adversarial network contains a variety of disease types, and the most representative synthetic images are determined through the Bayesian optimization algorithm;

[0155] The architecture of the generative adversarial network generation mechanism and Bayesian optimization strategy is implemented as follows:

[0156] Synthetic data generation by generative adversarial networks: Generative adversarial networks consist of a generator G and a discriminator D. Its goal is to generate synthetic images through the generator and distinguish between real images and generated images through the discriminator.

[0157] Generator loss function:

[0158] Among them, z is the random noise vector input by the generator, and the distribution p z (z), G(z) is the synthetic image output by the generator, and D(G(z)) is the probability that the discriminator determines that the generated image is a real image;

[0159] Discriminator loss function:

[0160]

[0161] Among them, x is the real image sample, p data (x) is the distribution of real data, and D(x) is the probability of the discriminator distinguishing the real image;

[0162] The goal of the generative adversarial network is to make the images generated by the generator realistic by alternately optimizing the generator and the discriminator;

[0163] Bayesian optimization is used to optimize synthetic images: A Bayesian optimization strategy is used to optimize representative image samples in the synthetic image space, and the synthetic images are determined by modeling the probability distribution of the objective function;

[0164] Objective function modeling: y = f(x) + ∈,

[0165] Among them, f(x) is the potential objective function, x is the generated synthetic image feature, y is the evaluation index, and ∈ is the observation noise;

[0166] Bayesian optimization process:

[0167] Determine the new sampling point x + By maximizing expected improvement:

[0168]

[0169] Among them, f(x + ) is the optimal objective function value;

[0170] According to the new sampling points, the Gaussian process model is updated to adjust the probability distribution estimate of the objective function;

[0171] Through iterative Bayesian optimization steps, the synthetic image with the highest expected improvement is finally determined as the most representative sample;

[0172] The synthetic images determined by Bayesian optimization are used to expand the training dataset.

[0173] Through the generative adversarial network, the generator G generates synthetic images from the random noise vector z, and the discriminator D is used to distinguish the synthetic images from the real images. The generator and the discriminator compete with each other through alternating optimization. The generator improves the quality of its generated images, making the synthetic images close to the real images. The process ensures the diversity and realism of the synthetic images, which can cover a variety of disease types.

[0174] In the generated synthetic images, the Bayesian optimization algorithm is used to determine the most representative samples. By probabilistically modeling the objective function, Bayesian optimization can effectively search in the synthetic image space. Using the expected improvement formula, Bayesian optimization can intelligently determine new sampling points and converge to the optimal synthetic image sample. Its strategy ensures that the synthetic image is highly representative and high quality.

[0175] Representative synthetic images determined through Bayesian optimization are used to expand the training dataset. High-quality synthetic data increases the diversity and scale of the dataset, allowing the model to better generalize to different disease types and samples during training, thereby improving recognition accuracy and robustness.

[0176] The multimodal fusion of the model is achieved through an adaptive weight allocation mechanism to dynamically adjust the weight of each modality according to the feature similarity of different modal inputs;

[0177] The fusion of multimodal features and adaptive weight adjustment are specifically implemented as follows:

[0178] Multimodal feature extraction: Extract features from input data of different modes. Assume that the features of each mode are expressed as: F CNN , F Capsule , F 其他模态 ,…, each feature F i It is a high-dimensional feature vector obtained by convolutional neural network or other feature extraction models;

[0179] Feature similarity calculation: Calculate the feature similarity between different modalities, using cosine similarity, Euclidean distance or other similarity metrics. The cosine similarity algorithm formula is:

[0180]

[0181] Among them, F i ,F j Indicates F i and F j The inner product of ∥F i ∥ and ∥F j ∥ is F i and F j The norm of

[0182] Adaptive weight calculation: Calculate the adaptive weight based on feature similarity. Assuming there are n modal inputs, the weight α i , calculated by the following formula:

[0183] Among them, λ is the adjustment parameter, Sim(F i ,F j ) is the feature similarity of mode i and mode j. This formula normalizes the weight of each mode through the Softmax function;

[0184] Multimodal feature fusion: weighted fusion of modal features to obtain the final fusion feature F 融合 : F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 ,

[0185] Among them, α, β, γ are adaptive weights, F 融合 represents the final fusion feature, F CNN represents the features extracted by convolutional neural network, F Capsulerepresents the features extracted by the capsule network, F 其他模态 Represent features from other modalities;

[0186] Fusion features are used for classification: The fused features F 融合 The input is fed into the subsequent classification model for final classification, and the output includes the confidence level of health, abnormality, and specific disease types.

[0187] Through the adaptive weight allocation mechanism, the model can dynamically adjust the weight of each modal feature, so that the most representative modal feature occupies a more important position in the fusion process, which can reduce the influence of noise or irrelevant features and improve the effectiveness of the final feature fusion;

[0188] The fusion features contain rich information from each modality. The weighted combination enables the model to utilize the diversity and complementarity of multimodal data to improve the ability to identify complex disease types. The weight adjustment ensures the optimal combination of features from different modalities, effectively improving the performance of the classification model.

[0189] By dynamically adjusting the weights of different modal features, the model can adapt to changes in multiple input sources and devices, improving its robustness in different environments and devices. The adaptive mechanism enables the model to have better tolerance to changes in input data, ensuring its stability and reliability in actual clinical applications.

[0190] Interpretable output is visualized by generating heat maps and compared with clinicians’ annotations to further validate and optimize the model’s decision path;

[0191] Generate a heat map and quantify its interpretability. The specific implementation is as follows:

[0192] Feature map extraction: Feature map A is extracted from the middle layer of the convolutional neural network k , where k represents the channel index of the feature map. Assume that the dimension of the feature map is H×W×K, where H is the height, W is the width, and K is the number of channels;

[0193] Gradient calculation of category score and feature map: For the score yc of the target category c, calculate the gradient of the category score to the feature map: This gradient represents the sensitivity of category c to feature map channel k;

[0194] Feature map weight calculation: Use global average pooling to calculate the weight of each feature map channel Used to measure the contribution of each channel to the final decision: in, Represents the activation value of the i-th, j-th position in the feature map;

[0195] Generate class activation maps: By combining the feature map with the weights Perform weighted summation to generate the final class activation map: The mapping Show the contribution of the region in the image to the prediction of category c;

[0196] Heatmap normalization and visualization: The generated class activation maps are normalized so that their values ​​range from 0 to 1 for visualization: Normalized mapping Can be superimposed on the original image to form a heat map for visualization and explanation of model decisions;

[0197] Explainability quantification and comparison with doctor annotations: The overlap between model interpretability and doctor annotations is quantified by the following formula: Among them, IOU represents the overlap ratio between the hotspot map and the doctor's annotation area, which is used to verify the reliability of the model interpretation;

[0198] Optimize the model decision path: Adjust and optimize the model based on the comparison results with the doctor's annotations. If the overlap is lower than the preset threshold T, the interpretability can be improved by fine-tuning the model parameters or adjusting the feature map extraction level.

[0199] By generating class activation maps and normalizing them into heat maps, the model can intuitively display its decision basis. Its visual explanation helps doctors understand the model's decision-making process, increase their trust in the model, and facilitate human-machine collaboration.

[0200] By using IOU to calculate the overlap between the model's explanatory output and the doctor's annotations, the credibility and reliability of the model's interpretation can be quantified. The index provides a quantitative basis for verifying and optimizing the model, ensuring that the model's decision-making is more in line with actual clinical needs.

[0201] By comparing the overlap between the model's hotspot map and the doctor's annotation area, the model's decision bias and deficiencies can be identified. Based on the comparison results, the feature extraction level can be adjusted or the model parameters can be fine-tuned, which can gradually improve the model's interpretability and decision accuracy, and ultimately optimize the model's performance in practical applications.

[0202] The third machine learning model uses support vector machine or logistic regression algorithm to classify the fused feature vector;

[0203] The classification process of support vector machine and logistic regression is implemented as follows:

[0204] Input feature vector: the fused feature vector F obtained in the multimodal fusion step 融合 As input to the classification model: F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 ,

[0205] Among them, α, β, γ are adaptive weights, F 融合 represents the final fusion feature, F CNN represents the features extracted by convolutional neural network, F Capsule represents the features extracted by the capsule network, F 其他模态 Represent features from other modalities;

[0206] Support Vector Machine Classification:

[0207] Support vector machine decision function: The support vector machine divides the data through the optimal hyperplane, which is defined by the weight vector w and the bias b. Its decision function is:

[0208] f(x)=w·F 融合 + b,

[0209] Classification: Classify the input according to the sign of the decision function:

[0210] y=sign(f(x))=sign(w·F 融合 +b),

[0211] Among them, y=1 indicates abnormality, and y=-1 indicates health;

[0212] Optimization objective: The support vector machine determines the optimal hyperplane by optimizing the following objective function:

[0213]

[0214] Where C is the regularization parameter;

[0215] Logistic Regression Classification:

[0216] Logistic regression decision function: Logistic regression uses the sigmoid function to map input features to probability values. Its decision function is: p(y=1|F 融合 )=σ(w·F 融合 +b),

[0217] in, is the sigmoid function;

[0218] Classification: Classify the input according to the probability output of the decision function:

[0219]

[0220] Among them, τ is the classification threshold, y=1 indicates abnormality, and y=0 indicates health.

[0221] Through the multimodal fusion step, data features from different modalities are extracted and combined to form a fused feature vector. This fused feature is used as the input of the classification model, which greatly enriches the data dimension and information volume of the classification model and helps to improve the accuracy of classification.

[0222] The support vector machine divides the fused feature vector into healthy and abnormal by finding the optimal hyperplane. Its optimization process ensures the maximum interval, thereby improving the robustness and generalization ability of classification. By adjusting the regularization parameter C, the support vector machine can find the best balance between model complexity and classification accuracy.

[0223] Logistic regression uses the sigmoid function to convert linear classification results into probability outputs, so that each prediction result can be interpreted as the probability of belonging to a certain category. It is used in medical decisions that require clear explanations to help doctors understand and trust the classification results of the model;

[0224] Through the application of support vector machine and logistic regression classification models, different classification requirements can be flexibly responded to. Support vector machine clearly defines healthy and abnormal classification problems, while logistic regression provides probabilistic output and more detailed decision-making capabilities. Combining the advantages of the models, the system can perform well in a variety of environments and requirements.

[0225] The human-machine collaboration platform includes an interactive interface for doctors to provide real-time feedback on model outputs and automatically updates model parameters to reflect the doctor's adjustments;

[0226] The feedback mechanism dynamically adjusts the model parameters, which is implemented as follows:

[0227] Real-time feedback collection: When doctors provide feedback on the model’s output results through the interactive interface, the actual feedback label y is recorded 真实 and the model's predicted label y 预测 , calculate the error:

[0228] ∈=y 真实 -y 预测 , where y 真实 is the real label annotated by the doctor, y 预测 is the predicted label output by the model;

[0229] Update of loss function: Define the loss function after feedback improvement, and incorporate the doctor's feedback error into the loss calculation to guide the parameter update of the model:

[0230] Among them, L 原始 (θ) is the original loss function of the model, λ is the weight coefficient of feedback adjustment, and N is the number of feedback samples;

[0231] Gradient update rule: According to the new loss function L 反馈(θ), update the model parameters θ by gradient descent: Where η is the learning rate, is the gradient of the loss function with respect to the parameter θ;

[0232] Dynamic learning rate adjustment: Dynamically adjust the learning rate according to the size of the feedback error to quickly adapt to the doctor's feedback: η t+1 =η t ·(1+β·|∈|), where β is the dynamic adjustment coefficient;

[0233] Application of update rules: For each feedback, follow The formula is updated with parameters and continuously iterated, and the model adjusts its output to better match the doctor’s judgment:

[0234]

[0235] Through feedback loops, the model can adjust adaptively;

[0236] Parameter convergence and verification: Through continuous feedback and parameter update, if the feedback error ∈ converges to a certain threshold δ, stop parameter update and evaluate model performance through verification:

[0237] Stop condition: |∈|<δ.

[0238] By collecting and calculating feedback errors in real time, the model can immediately identify the gap between predictions and doctors’ expectations, and incorporate the errors into the calculation of the loss function, so that the model can be improved in a targeted manner in the next step of training;

[0239] Combined with gradient updates and dynamic learning rate adjustment, the model can adaptively adjust under different feedback conditions. The learning rate is dynamically adjusted according to the error size, ensuring that the model can respond quickly to significant errors and can be fine-tuned stably to prevent over-adjustment.

[0240] Through the feedback loop, the model is continuously iterated and updated, gradually reducing the difference with the doctor's judgment. If the error converges to the set threshold, the adjustment is stopped and the model performance is evaluated through the validation set to ensure that the final optimization result is consistent with the doctor's feedback and can maintain good generalization ability and robustness.

[0241] The framework supports efficient interaction between doctors and models. Doctors' feedback can directly affect the model's learning process, making the model gradually more in line with actual clinical needs. The dynamic adjustment mechanism improves the model's predictive accuracy, enhances doctors' trust and acceptance of AI models, and provides more effective decision-making support tools for medical diagnosis.

[0242] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A medical image recognition method based on AI, characterized in that: The steps include: S1. Acquire multimodal medical image data and preprocess the images, including: denoising, contrast enhancement, super-resolution reconstruction, and key area segmentation to highlight disease-related areas; S2, use a multi-expert annotation system to annotate images, and generate synthetic images through a generative adversarial network to expand the dataset; S3, using a multimodal fusion model architecture that combines convolutional neural networks and capsule networks, and using the following formula to achieve modal fusion: F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 , Among them, α, β, and γ are adaptive weights, which are dynamically adjusted by the following formula: Among them, k is the adjustment parameter, b is the bias, and the feature similarity is calculated based on the feature similarity between the modalities; S4. Through transfer learning, the model pre-trained on the dataset is migrated to a specific medical task. Combined with continuous learning strategies, fairness constraint technology is used to optimize the model so that it performs consistently across different populations. S5. Integrate explainable AI technology to generate visual explanations to highlight key areas in model decisions, and quantify the strength of the explanation through the following formula: Among them, w i is the weight and contribution of the i-th feature i The contribution of this feature to the model output; S6. Validate the model in a multi-center clinical setting, collect feedback from clinicians, and introduce adversarial training and differential privacy techniques to enhance the robustness of the model in different devices and environments. S7. Use a third machine learning model to classify the fused feature vector, and the output includes the confidence of health, abnormality, and specific disease types, and the interpretable output is combined in the decision-making process; S8. Develop a human-machine collaboration platform that allows clinicians to interact with the model, check and adjust the model's predictions, and the model records the learning physician's feedback; S9. Establish an automated feedback mechanism to monitor the difference between the model's predictions and the doctor's diagnosis results in real time, and make dynamic adjustments through the following algorithm formula: Among them, η is the learning rate, λ is the feedback adjustment coefficient, and θ is the model parameter; S10. Use federated learning technology and differential privacy algorithms to train and update models without exposing patient data.

2. The AI-based medical image recognition method according to claim 1, characterized in that: The key area segmentation is achieved through an architecture combining a convolutional neural network and U-Net; The architecture of combining the convolutional neural network with U-Net is specifically implemented as follows: Convolution operation: A series of convolution operations are performed on the input image I to extract features: F (l) =σ(W (l) *F (l-1) +b (l) ), Among them, F (l) is the feature map of the lth layer, W (l) is the convolution kernel of the lth layer, b (l) is the bias, * represents the convolution operation, and σ is the activation function; Downsampling and upsampling of U-Net architecture: Downsampling path: extract multi-level features through convolutional layers and maximum pooling layers: Upsampling path: restore image resolution through deconvolution layers and skip connections, and focus on the target area: in, and is the convolution kernel of the encoder and decoder parts, UpSample is the upsampling operation, and Feature maps for downsampling and upsampling paths; Final output: Through a 1×11×1 convolutional layer, the output of the upsampling path is mapped to the target category space to achieve region segmentation: Among them, W out is the convolution kernel of the output layer, b out is the bias of the output layer, O is the final segmentation result output, and the result is converted into a probability map through the Softmax activation function.

3. The AI-based medical image recognition method according to claim 1, characterized in that: The synthetic data generated by the generative adversarial network includes multiple disease types, and the most representative synthetic image is determined by a Bayesian optimization algorithm.

4. The AI-based medical image recognition method according to claim 3, characterized in that: The architecture of the generative adversarial network generation mechanism and Bayesian optimization strategy is specifically implemented as follows: Synthetic data generation by generative adversarial networks: Generative adversarial networks consist of a generator G and a discriminator D. Its goal is to generate synthetic images through the generator and distinguish between real images and generated images through the discriminator. Generator loss function: Among them, z is the random noise vector input by the generator, and the distribution p z (z), G(z) is the synthetic image output by the generator, and D(G(z)) is the probability that the discriminator determines that the generated image is a real image; Discriminator loss function: Among them, x is the real image sample, p data (x) is the distribution of real data, and D(x) is the probability of the discriminator distinguishing the real image; The goal of the generative adversarial network is to make the images generated by the generator realistic by alternately optimizing the generator and the discriminator; Bayesian optimization is used to optimize synthetic images: A Bayesian optimization strategy is used to optimize representative image samples in the synthetic image space, and the synthetic images are determined by modeling the probability distribution of the objective function; Objective function modeling: y = f(x) + ∈, ∈ ~ N (0, σ 2 ), Among them, f(x) is the potential objective function, x is the generated synthetic image feature, y is the evaluation index, and ∈ is the observation noise; Bayesian optimization process: Determine the new sampling point x + By maximizing expected improvement: Among them, f(x + ) is the optimal objective function value; According to the new sampling points, the Gaussian process model is updated to adjust the probability distribution estimate of the objective function; Through iterative Bayesian optimization steps, the synthetic image with the highest expected improvement is finally determined as the most representative sample; The synthetic images determined by Bayesian optimization are used to expand the training dataset.

5. The AI-based medical image recognition method according to claim 1, characterized in that: The multimodal fusion of the model is achieved through an adaptive weight allocation mechanism to dynamically adjust the weight of each modality according to the feature similarity of different modal inputs; The fusion of multimodal features and adaptive weight adjustment are specifically implemented as follows: Multimodal feature extraction: Extract features from input data of different modes. Assume that the features of each mode are expressed as: F CNN , F Capsule , F 其他模态 ,…, each feature F i It is a high-dimensional feature vector obtained by convolutional neural network or other feature extraction models; Feature similarity calculation: Calculate the feature similarity between different modalities, using cosine similarity, Euclidean distance or other similarity metrics. The cosine similarity algorithm formula is: Among them, F i ·F j Indicates F i and F j The inner product of ||F i || and ||F j || is F i and F j The norm of ; Adaptive weight calculation: Calculate the adaptive weight based on feature similarity. Assuming there are n modal inputs, the weight α i , calculated by the following formula: Among them, λ is the adjustment parameter, Sim(F i ,F j ) is the feature similarity of mode i and mode j. This formula normalizes the weight of each mode through the Softmax function; Multimodal feature fusion: weighted fusion of modal features to obtain the final fusion feature F 融合 : F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 , Among them, α, β, γ are adaptive weights, F 融合 represents the final fusion feature, F CNN represents the features extracted by convolutional neural network, F Capsule represents the features extracted by the capsule network, F 其他模态 Represent features from other modalities; Fusion features are used for classification: The fused features F 融合 The input is fed into the subsequent classification model for final classification, and the output includes the confidence level of health, abnormality, and specific disease types.

6. The AI-based medical image recognition method according to claim 1, characterized in that: The interpretable output is visualized by generating a heat map and compared with the clinician's annotations to further validate and optimize the model's decision path; The generation of a heat map and quantification of its interpretability are specifically implemented as follows: Feature map extraction: Feature map A is extracted from the middle layer of the convolutional neural network k , where k represents the channel index of the feature map. Assume that the dimension of the feature map is H×W×K, where H is the height, W is the width, and K is the number of channels; Gradient calculation of category score and feature map: score y for target category c c , calculate the gradient of the category score to the feature map: This gradient represents the sensitivity of category c to feature map channel k; Feature map weight calculation: Use global average pooling to calculate the weight of each feature map channel Used to measure the contribution of each channel to the final decision: in, Represents the activation value of the i-th, j-th position in the feature map; Generate class activation maps: By combining the feature map with the weights Perform weighted summation to generate the final class activation map: The mapping Show the contribution of the region in the image to the prediction of category c; Heatmap normalization and visualization: The generated class activation maps are normalized so that their values ​​range from 0 to 1 for visualization: Normalized mapping Can be superimposed on the original image to form a heat map for visualization and explanation of model decisions; Explainability quantification and comparison with doctor annotations: The overlap between model interpretability and doctor annotations is quantified by the following formula: Among them, IOU represents the overlap ratio between the hotspot map and the doctor's annotation area, which is used to verify the reliability of the model interpretation; Optimize the model decision path: Adjust and optimize the model based on the comparison results with the doctor's annotations. If the overlap is lower than the preset threshold T, the interpretability can be improved by fine-tuning the model parameters or adjusting the feature map extraction level.

7. The AI-based medical image recognition method according to claim 1, characterized in that: The third machine learning model uses a support vector machine or a logistic regression algorithm to classify the fused feature vector.

8. The AI-based medical image recognition method according to claim 7, characterized in that: The classification process of the support vector machine and logistic regression is specifically implemented as follows: Input feature vector: the fused feature vector F obtained in the multimodal fusion step 融合 As input to the classification model: F 融合 =α·F CNN +β·F Capsule +γ·F 其他模态 , Among them, α, β, γ are adaptive weights, F 融合 represents the final fusion feature, F CNN represents the features extracted by convolutional neural network, F Capsule represents the features extracted by the capsule network, F 其他模态 Represent features from other modalities; Support Vector Machine Classification: Support vector machine decision function: The support vector machine divides the data through the optimal hyperplane, which is defined by the weight vector w and the bias b. Its decision function is: f(x)=w·F 融合 +b, Classification: Classify the input according to the sign of the decision function: y=sign(f(x))=sign(w·F 融合 +b), Among them, y=1 indicates abnormality, and y=-1 indicates health; Optimization objective: The support vector machine determines the optimal hyperplane by optimizing the following objective function: Where C is the regularization parameter; Logistic Regression Classification: Logistic regression decision function: Logistic regression uses the sigmoid function to map input features to probability values. Its decision function is: p(y=1|F 融合 )=σ(w·F 融合 +b), in, is the sigmoid function; Classification: Classify the input according to the probability output of the decision function: Among them, τ is the classification threshold, y=1 indicates abnormality, and y=0 indicates health.

9. The AI-based medical image recognition method according to claim 1, characterized in that: The human-machine collaboration platform includes an interactive interface for doctors to provide real-time feedback on model outputs and automatically update model parameters to reflect the doctor's adjustments.

10. The AI-based medical image recognition method according to claim 9, characterized in that: The feedback mechanism dynamically adjusts the model parameters, which is specifically implemented as follows: Real-time feedback collection: When doctors provide feedback on the model’s output results through the interactive interface, the actual feedback label y is recorded 真实 and the model's predicted label y 预测 , calculate the error: ∈=y 真实 -y 预测 , where y 真实 is the true label annotated by the doctor, y 预测 is the predicted label output by the model; Update of loss function: Define the loss function after feedback improvement, and incorporate the doctor's feedback error into the loss calculation to guide the parameter update of the model: Among them, L 原始 (θ) is the original loss function of the model, λ is the weight coefficient of feedback adjustment, and N is the number of feedback samples; Gradient update rule: According to the new loss function L 反馈 (θ), update the model parameters θ by gradient descent: Where η is the learning rate, is the gradient of the loss function with respect to the parameter θ; Dynamic learning rate adjustment: Dynamically adjust the learning rate according to the size of the feedback error to quickly adapt to the doctor's feedback: η t+1 =η t ·(1+β·|∈|), where β is the dynamic adjustment coefficient; Application of update rules: For each feedback, follow The formula is updated with parameters and continuously iterated, and the model adjusts its output to better match the doctor’s judgment: Through feedback loops, the model can adjust adaptively; Parameter convergence and verification: Through continuous feedback and parameter update, if the feedback error ∈ converges to a certain threshold δ, stop parameter update and evaluate model performance through verification: Stop condition: |∈|<δ.

Images