Medical image recognition method based on AI
By combining multimodal image fusion and gated attention mechanisms with CNN and capsule networks, the problem of insufficient single-modal image recognition capability is solved, achieving high-precision identification and lesion localization of complex diseases, and improving the efficiency and accuracy of medical image analysis.
Patent Information
- Application Number
- CN202511069584.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-11-11
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing medical image recognition methods rely on single-modality image data, resulting in low recognition capabilities for complex diseases. Furthermore, they fail to adequately address the data imbalance problem, causing models to be biased towards certain disease types.
This study employs multimodal medical image fusion technology, combining convolutional neural networks and capsule networks. It aligns images of different modalities through an image registration method, dynamically adjusts image modality weights using a gated attention mechanism, and optimizes the model through a joint loss function to extract low-level and spatial features of multimodal images.
It improves the ability to identify complex diseases, reduces the risk of misdiagnosis and missed diagnosis, improves the accuracy of disease classification and lesion localization, rationally allocates medical resources, and reduces patient waiting time.
Smart Images
Figure CN120932071A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data analysis technology, and more specifically, to an AI-based medical image recognition method. Background Technology
[0002] With the rapid development of artificial intelligence (AI) technology, its application in medical image recognition has gradually become a research hotspot. Medical image recognition technology can assist clinicians in making disease diagnosis and treatment decisions by analyzing various image data such as X-rays, CT scans, MRI, and ultrasound. However, existing medical image recognition methods typically rely on single-modality image data, such as using CT or MRI alone for disease detection and classification. This method is easily limited by information from a single data modality, resulting in lower recognition capabilities for complex disease types.
[0003] Furthermore, existing technologies do not adequately address how to handle the data imbalance problem that is prevalent in medical image data. When data for certain disease types may be far more abundant than data for other types, the model may be biased towards predicting the category with the larger number of cases.
[0004] Against this backdrop, the present invention introduces multimodal medical image fusion technology, combining a multimodal fusion model that integrates convolutional neural networks and capsule networks, which can extract important features from images of different modalities and improve the ability to identify complex diseases. Summary of the Invention
[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an AI-based medical image recognition method to address the problems mentioned in the background section.
[0006] To achieve the above objectives, the present invention provides the following technical solution: an AI-based medical image recognition method, specifically comprising: Medical image data from different modalities are acquired and preprocessed. An image registration method is used to align images from different modalities and fuse the image data from different modalities. CNN is used to extract low-level features from the fused image. Capsule network is used to model the spatial features of the extracted low-level features to capture the spatial relationships and transformation information in the image. The low-level features extracted by CNN are then fused with the spatial features processed by capsule network. The model is trained using a large-scale labeled medical image dataset, which includes different types of diseases and normal samples. During the training process, a gated attention mechanism is set up to automatically adjust the weights of different image modalities based on the characteristics of different diseases. The fused features are input into the trained model, which outputs the classification label of the image, accurately identifying the location of lesions in the image.
[0007] Preferably, as a preferred embodiment of the AI-based medical image recognition method of the present invention, it includes acquiring medical image data from different modalities and preprocessing it, using an image registration method to align images from different modalities, and fusing image data from different modalities, specifically including: Image data is acquired from different modalities of medical devices, including CT, MRI, ultrasound, and X-ray. CT is used for skeletal structures, MRI for soft tissue imaging, and PET for metabolic activity information. The images from different modalities are denoised and the contrast and clarity of the images are improved through image enhancement methods. By transforming the model, images of different modalities are aligned so that they are in the same coordinate space. For two images... and The transformation parameters are calculated by minimizing the difference between the overlapping regions of the two images. ,in, It is an image and The joint probability density function, and These are the edge probabilities of the two images; Wavelet transform is used to decompose the aligned image into different frequency bands. By fusing information from these different frequency bands, a fused image is obtained. ,in, It's the low-frequency range. It's the high-frequency part. It is the enhancement coefficient.
[0008] Preferably, as a preferred embodiment of the AI-based medical image recognition method of the present invention, it includes using a CNN to extract low-level features from the fused image, using a capsule network to perform spatial feature modeling on the extracted low-level features, capturing spatial relationships and transformation information in the image, and fusing the low-level features extracted by the CNN with the spatial features processed by the capsule network, specifically including: Local features of the fused image are extracted through convolution operations. The ReLU activation function is applied to the result, and the size of the feature map is reduced by max pooling to obtain the low-level feature map extracted by the CNN. ,in, This is the first convolution kernel; the convolution operation is performed by the symbol... express, For max pooling operation, It is a low-level feature map; Low-level features extracted by CNN The capsule layers in the capsule network are input as an image. A transformation matrix is used to transform the low-level features, yielding the predicted capsule output. After transformation, the capsule network calculates the weight coefficients using a dynamic routing algorithm. and update capsule output. ,in, It is a transformation matrix. It is the bias term in the routing algorithm. These are the capsule features of the final output; By assigning weights to each feature and The features output by the CNN and capsule network are weighted and fused to obtain the fused features: ,in, These are low-level features extracted by CNN. It is the capsule feature of the final output.
[0009] Preferably, as a preferred embodiment of the AI-based medical image recognition method of the present invention, it includes training using a large-scale labeled medical image dataset, the dataset including different types of diseases and normal samples. During the model training process, a disease-aware gating attention mechanism is set up to automatically adjust the weights of different image modalities according to the characteristics of different diseases, specifically including: Using publicly available medical image datasets Training was conducted, including... This represents the image of the Nth modality of the g-th sample. This is the target label of the g-th sample, representing the disease type of the sample, including cancer and benign tumors; the weights of different image modalities are automatically adjusted based on the features of each image sample and the disease type, further including: The images of each modality are processed using a convolutional neural network to extract features, resulting in the features for each modality. And calculate the initial importance score for each mode as follows: A gating attention mechanism is introduced, and for disease labels, the learning modality-disease association weights are calculated as follows: The weights of each modality are dynamically adjusted based on disease characteristics, using the following formula: ,in, It is the feature representation of the u-th modality image. It is a convolutional neural network for the u-th modality image. These are the parameters of the network. This is the initial importance score for the u-th mode. It is the spatial dimension of the feature map. These are the pixel coordinates in the image. It is the association weight of disease y with the u-th modality. It is a learnable weight vector specific to disease y. It is a global average pooling operation that compresses the feature map into a vector. It is the bias term of disease y. It is an activation function. It is the final dynamic weight of disease y on the u-th modality; The extracted features are combined with the calculated attention weights to obtain the final comprehensive feature representation: Use the fused features As input, it is passed to the classifier for disease classification, and the model is trained using the cross-entropy loss function, the specific formula of which is: , ,in, These are the parameters of the classifier. It's a real label. These are the model's predicted labels. It is the cross-entropy loss function.
[0010] Preferably, as a preferred embodiment of the AI-based medical image recognition method of the present invention, it includes inputting the fused features into a trained model and outputting a classification label for the image to accurately identify the location of lesions in the image, specifically including: Weighted fusion of features from CNN and capsule network The input is fed into the trained model, and the output image's classification label is... ,in, These are the image classification labels predicted by the model. Represents the parameters of the classifier. These are the features obtained after weighted fusion; Using an object detection network to predict the bounding box of the lesion, the features are fused. As input, the bounding box coordinates of the lesion are output through a regression layer to accurately identify the spatial location of the lesion. The bounding box coordinates are... ,in, These are the center coordinates of the lesion's bounding box. and It refers to the width and height of the lesion; A joint loss function, combining classification and regression losses, is used to optimize image classification and lesion location prediction. The joint loss function is: ,in, These are genuine category tags. These are the classification labels predicted by the model. It is a smoothed L1 loss used to regress the bounding box location. These are hyperparameters used to balance classification loss and regression loss. It is the joint loss function.
[0011] On the other hand, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements the steps of an AI-based medical image recognition method as described above.
[0012] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements the steps of an AI-based medical image recognition method as described above.
[0013] The technical effects and advantages provided by the present invention in the above technical solution are as follows: By combining CNN and capsule network and fusing image features from different modalities, low-level and spatial features in images can be extracted more effectively, improving the accuracy of disease feature identification. The weights of each modal image are dynamically adjusted according to the features of different diseases, and the accuracy of disease classification and lesion localization is further improved through gating attention mechanism and joint loss function optimization. Through the automated processing of AI, this invention enables a more reasonable allocation of medical resources and reduces patient waiting time caused by excessive manual examination. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.
[0015] Figure 1 This is a flowchart of the method of the present invention.
[0016] Figure 2 This is a comparison chart of the classification accuracy of the multimodal fusion model.
[0017] Table 1 is a data recording table of the simulation experiment of this invention. Detailed Implementation
[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0020] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.
[0021] Example 1 This embodiment provides, for example Figure 1 The illustrated AI-based medical image recognition method specifically includes: Medical image data from different modalities are acquired and preprocessed. An image registration method is used to align images from different modalities and fuse the image data from different modalities. CNN is used to extract low-level features from the fused image. Capsule network is used to model the spatial features of the extracted low-level features to capture the spatial relationships and transformation information in the image. The low-level features extracted by CNN are then fused with the spatial features processed by capsule network. The model is trained using a large-scale labeled medical image dataset, which includes different types of diseases and normal samples. During the training process, a gated attention mechanism is set up to automatically adjust the weights of different image modalities based on the characteristics of different diseases. The fused features are input into the trained model, which outputs the classification label of the image, accurately identifying the location of lesions in the image.
[0022] Preferably, the acquisition and preprocessing of medical image data from different modalities, the use of image registration methods to align images from different modalities, and the fusion of image data from different modalities can improve the accuracy of disease feature identification and reduce the risk of misdiagnosis and missed diagnosis. Specifically, this includes: Image data is acquired from different modalities of medical devices, including CT, MRI, ultrasound, and X-ray. CT is used for skeletal structures, MRI for soft tissue imaging, and PET for metabolic activity information. The images from different modalities are denoised and the contrast and clarity of the images are improved through image enhancement methods. By transforming the model, images of different modalities are aligned so that they are in the same coordinate space. For two images... and The transformation parameters are calculated by minimizing the difference between the overlapping regions of the two images. ,in, It is an image and The joint probability density function, and These are the edge probabilities of the two images; Wavelet transform is used to decompose the aligned image into different frequency bands. By fusing information from these different frequency bands, a fused image is obtained. ,in, It's the low-frequency range. It's the high-frequency part. It is the enhancement coefficient.
[0023] Preferably, the step of using CNN to extract low-level features from the fused image, utilizing capsule networks to model the spatial features of the extracted low-level features, capturing spatial relationships and transformation information in the image, and fusing the low-level features extracted by CNN with the spatial features processed by capsule networks, improves the model's ability to understand and recognize complex image features, specifically including: Local features of the fused image are extracted through convolution operations. The ReLU activation function is applied to the result, and the size of the feature map is reduced by max pooling to obtain the low-level feature map extracted by the CNN. ,in, This is the first convolution kernel; the convolution operation is performed by the symbol... express, For max pooling operation, It is a low-level feature map; Low-level features extracted by CNN The capsule layers in the capsule network are input as an image. A transformation matrix is used to transform the low-level features, yielding the predicted capsule output. After transformation, the capsule network calculates the weight coefficients using a dynamic routing algorithm. and update capsule output. ,in, It is a transformation matrix. It is the bias term in the routing algorithm. These are the capsule features of the final output; By assigning weights to each feature and The features output by the CNN and capsule network are weighted and fused to obtain the fused features: ,in, These are low-level features extracted by CNN. It is the capsule feature of the final output.
[0024] Preferably, the model is trained using a large-scale labeled medical image dataset, which includes different types of diseases and normal samples. During model training, a disease-aware gated attention mechanism is set up to automatically adjust the weights of different image modalities based on the characteristics of different diseases, thereby improving the classification and localization accuracy of different disease types. Specifically, this includes: Using publicly available medical image datasets Training was conducted, including... This represents the image of the Nth modality of the g-th sample, where each sample... The target label is This indicates the disease type of the sample; the weights of different image modalities are automatically adjusted based on the features and disease type of each image sample, further including: The images of each modality are processed using a convolutional neural network to extract features, resulting in the features for each modality. And calculate the initial importance score for each mode as follows: A gating attention mechanism is introduced, and for disease labels, the learning modality-disease association weights are calculated as follows: The weights of each modality are dynamically adjusted based on disease characteristics, using the following formula: ,in, It is the feature representation of the u-th modality image. It is a convolutional neural network for the u-th modality image. These are the parameters of the network. This is the initial importance score for the u-th mode. It is the spatial dimension of the feature map. These are the pixel coordinates in the image. It is the association weight of disease y with the u-th modality. It is a learnable weight vector specific to disease y. It is a global average pooling operation that compresses the feature map into a vector. It is the bias term of disease y. It is an activation function. It is the final dynamic weight of disease y on the u-th modality; The extracted features are combined with the calculated attention weights to obtain the final comprehensive feature representation: Use the fused features As input, it is passed to the classifier for disease classification, and the model is trained using the cross-entropy loss function, the specific formula of which is: , ,in, These are the parameters of the classifier. It's a real label. These are the model's predicted labels. It is the cross-entropy loss function.
[0025] Preferably, the fused features are input into the trained model to output image classification labels, accurately identifying the location of lesions in the image. By combining a joint loss function of classification loss and regression loss, image classification and lesion location prediction are optimized, reducing the conflict between classification and localization. Specifically, this includes: Weighted fusion of features from CNN and capsule network The input is fed into the trained model, and the output image's classification label is... ,in, These are the image classification labels predicted by the model. Represents the parameters of the classifier. These are the features obtained after weighted fusion; Using an object detection network to predict the bounding box of the lesion, the features are fused. As input, the bounding box coordinates of the lesion are output through a regression layer to accurately identify the spatial location of the lesion. The bounding box coordinates are... ,in, These are the center coordinates of the lesion's bounding box. and It refers to the width and height of the lesion; A joint loss function, combining classification and regression losses, is used to optimize image classification and lesion location prediction. The joint loss function is: ,in, These are genuine category tags. These are the classification labels predicted by the model. It is a smoothed L1 loss used to regress the bounding box location. These are hyperparameters used to balance classification and regression losses, optimized through the joint loss function. The model can simultaneously improve image classification accuracy and lesion location accuracy.
[0026] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.
[0027] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of implementing an AI-based medical image recognition method as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0028] Example 2 The following is another embodiment of the present invention, which provides an AI-based medical image recognition method. In order to verify the beneficial effects of the present invention, a simulation experiment is conducted for scientific demonstration.
[0029] This experiment used medical image data from different modalities and verified the accuracy and robustness of the method in medical image recognition by analyzing the consistency between the disease classification output by the model and the actual images. The experiment aims to improve the efficiency of medical image analysis, accurately identify the location of lesions, and assist in clinical diagnosis by using image data acquisition, preprocessing, feature extraction, deep learning models and classification techniques. The simulation experiment steps are implemented according to the content of the AI-based medical image recognition method provided in Example 1, and the specific steps include: Medical image data from different modalities were acquired and preprocessed to ensure that the image data quality was suitable for subsequent analysis. An image registration method is used to align medical images of different modalities and then fuse the aligned images to obtain complete image features from different perspectives and modalities, ensuring the effective fusion of images of different modalities and providing favorable support for subsequent feature extraction. We utilize convolutional neural networks to extract low-level features from images, and then use capsule networks to model the spatial features of the extracted low-level features, capturing spatial relationships and transformation information in the images. Finally, we fuse the low-level features extracted by CNN with the spatial features processed by capsule networks to enhance our understanding of structural and morphological information in images. The model is trained using a large-scale labeled medical image dataset, which includes different types of diseases and normal samples. During the training process, a gated attention mechanism is set up to automatically adjust the weights of different image modalities according to the characteristics of different diseases, thereby improving the model's adaptability and diagnostic ability. The fused features are input into the classifier, which outputs the image's classification label to accurately identify the location of lesions in the image. Based on the image's feature information, the classifier identifies and locates the lesion area and classifies the disease type.
[0030] The specific data from the above simulation experiment are as follows: Time / minute Image modality Image quality CNN Feature Extraction Capsule network spatial characteristics Features after fusion Classifier output Lesion location identification Classification results 0-5 CT good Low-level feature 1 Spatial Feature 1 Fusion Feature 1 cancer yes Cancer (lung) 5-10 MRI good Low-level feature 2 Spatial Feature 2 Fusion Feature 2 Benign tumor no Benign (brain) 10-15 CT medium Low-level feature 3 Spatial Feature 3 Fusion Feature 3 High risk yes Cancer (stomach) 15-20 X-ray good Low-level feature 4 Spatial Feature 4 Fusion Feature 4 Early lesions no Lesion (chest) 20-25 CT good Low-level feature 5 Spatial feature 5 Fusion Feature 5 cancer yes Cancer (liver) 25-30 MRI medium Low-level feature 6 Spatial Features 6 Fusion Feature 6 Benign tumor no Benign (lung) Table 1 Experimental Analysis: By comparing the consistency between the model's prediction results and the actual diagnostic labels, the accuracy and robustness of the system in identifying and predicting lesion locations were verified. The experiment showed that the AI-based medical image recognition method, combined with CNN, capsule network, and gated attention mechanism, can effectively improve the accuracy and reliability of medical image analysis. This method can give full play to its advantages in the fusion of multimodal image data, achieve accurate lesion localization and classification, and provide strong support for medical image analysis.
[0031] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.
Claims
1. An AI-based medical image recognition method, characterized in that: Specifically, it includes: Medical image data from different modalities are acquired and preprocessed. An image registration method is used to align images from different modalities and fuse the image data from different modalities. CNN is used to extract low-level features from the fused image. Capsule network is used to model the spatial features of the extracted low-level features to capture the spatial relationships and transformation information in the image. The low-level features extracted by CNN are then fused with the spatial features processed by capsule network. The model is trained using a large-scale labeled medical image dataset, which includes different types of diseases and normal samples. During the training process, a gated attention mechanism is set up to automatically adjust the weights of different image modalities based on the characteristics of different diseases. The fused features are input into the trained model, which outputs the classification label of the image, accurately identifying the location of lesions in the image.
2. The AI-based medical image recognition method according to claim 1, characterized in that: The process of acquiring and preprocessing medical image data from different modalities, aligning images from different modalities using an image registration method, and fusing the image data from different modalities specifically includes: Image data is acquired from different modalities of medical equipment, including CT, MRI, ultrasound, and X-ray. The images from different modalities are denoised and the contrast and clarity of the images are improved through image enhancement methods. By transforming the model, images of different modalities are aligned so that they are in the same coordinate space. For two images... and The transformation parameters are calculated by minimizing the difference between the overlapping regions of the two images. ,in, It is an image and The joint probability density function, and These are the edge probabilities of the two images; Wavelet transform is used to decompose the aligned image into different frequency bands. By fusing information from these different frequency bands, a fused image is obtained. ,in, It's the low-frequency range. It's the high-frequency part. It is the enhancement coefficient.
3. The AI-based medical image recognition method according to claim 1, characterized in that: The process involves using a CNN to extract low-level features from the fused image, utilizing a capsule network to model the spatial features of the extracted low-level features, capturing spatial relationships and transformation information in the image, and fusing the low-level features extracted by the CNN with the spatial features processed by the capsule network. Specifically, this includes: Local features of the fused image are extracted through convolution operations. The ReLU activation function is applied to the result, and the size of the feature map is reduced by max pooling to obtain the low-level feature map extracted by the CNN. ,in, This is the first convolution kernel; the convolution operation is performed by the symbol... express, For max pooling operation, It is a low-level feature map; Low-level features extracted by CNN The capsule layers in the capsule network are input as an image. A transformation matrix is used to transform the low-level features, yielding the predicted capsule output. After transformation, the capsule network calculates the weight coefficients using a dynamic routing algorithm. and update capsule output. ,in, It is a transformation matrix. It is the bias term in the routing algorithm. These are the capsule features of the final output; By assigning weights to each feature and The features output by the CNN and capsule network are weighted and fused to obtain the fused features: ,in, These are low-level features extracted by CNN. It is the capsule feature of the final output.
4. The AI-based medical image recognition method according to claim 1, characterized in that: The model is trained using a large-scale labeled medical image dataset, which includes different types of diseases and normal samples. During training, a gated attention mechanism is implemented to automatically adjust the weights of different image modalities based on the characteristics of different diseases. Specifically, this includes: Using publicly available medical image datasets Training was conducted, including... This represents the image of the Nth modality of the g-th sample, where each sample... The target label is , indicating the disease type of the sample; The weights of different image modalities are automatically adjusted based on the features of each image sample and the disease type.
5. The AI-based medical image recognition method according to claim 4, characterized in that: The automatic adjustment of weights for different image modalities based on the features and disease type of each image sample further includes: The images of each modality are processed using a convolutional neural network to extract features, resulting in the features for each modality. And calculate the initial importance score for each mode as follows: A gating attention mechanism is introduced, and for disease labels, the learning modality-disease association weights are calculated as follows: The weights of each modality are dynamically adjusted based on disease characteristics, using the following formula: ,in, It is the feature representation of the u-th modality image. It is a convolutional neural network for the u-th modality image. These are the parameters of the network. This is the initial importance score for the u-th mode. It is the spatial dimension of the feature map. These are the pixel coordinates in the image. It is the association weight of disease y with the u-th modality. It is a learnable weight vector specific to disease y. It is a global average pooling operation that compresses the feature map into a vector. It is the bias term of disease y. It is an activation function. It is the final dynamic weight of disease y on the u-th modality; The extracted features are combined with the calculated attention weights to obtain the final comprehensive feature representation: Use the fused features As input, it is passed to the classifier for disease classification, and the model is trained using the cross-entropy loss function, the specific formula of which is: , ,in, These are the parameters of the classifier. It's a real label. These are the model's predicted labels. It is the cross-entropy loss function.
6. The AI-based medical image recognition method according to claim 1, characterized in that: The process of inputting the fused features into the trained model and outputting image classification labels to accurately identify the location of lesions in the image specifically includes: Weighted fusion of features from CNN and capsule network The input is fed into the trained model, and the output image's classification label is... ,in, These are the image classification labels predicted by the model. Represents the parameters of the classifier. These are the features obtained after weighted fusion; Using an object detection network to predict the bounding box of the lesion, the features are fused. As input, the bounding box coordinates of the lesion are output through a regression layer to accurately identify the spatial location of the lesion. The bounding box coordinates are... ,in, These are the center coordinates of the lesion's bounding box. and It refers to the width and height of the lesion; A joint loss function, combining classification and regression losses, is used to optimize image classification and lesion location prediction. The joint loss function is: ,in, These are genuine category tags. These are the classification labels predicted by the model. It is a smoothed L1 loss used to regress the bounding box location. These are hyperparameters used to balance classification loss and regression loss. It is the joint loss function.