Hepatobiliary disease prediction method and system based on eye OCT image

Abstract

The invention provides a liver and gall disease prediction method and system based on an eye OCT image. The method comprises the steps that an original fundus OCT image is acquired; performing image preprocessing on the original fundus OCT image to obtain a standardized image; inputting the standardized image into a preset image recognition model to enable the image recognition model to sequentially perform feature extraction and global average pooling on the standardized image, outputting corresponding prediction labels through a plurality of binary classifiers in a full connection layer, and further generating a liver and gall disease prediction result; wherein the image recognition model is obtained based on a pre-trained neural network model and a plurality of historical fundus OCT images by adopting a cross entropy loss function with category weights and transfer learning training, and the category weights are set in an inverse proportion according to the proportion of each category in the plurality of historical fundus OCT images; and the accuracy of predicting liver and gall diseases by the artificial intelligence model and the use experience of the user are improved.

Description

A method and system for predicting hepatobiliary diseases based on ocular OCT images Technical Field

[0001] This application relates to the fields of neural network and image processing technology, and in particular to a method and system for predicting hepatobiliary diseases based on ocular OCT images. Background Technology

[0002] According to the latest report from the World Health Organization (WHO), hepatobiliary diseases, such as chronic viral hepatitis, cirrhosis, and liver cancer, are among the leading causes of global public health burden. Due to the lack of obvious symptoms in the early stages of hepatobiliary diseases, limited clinical experience and non-invasive screening technologies, coupled with uneven distribution of medical resources, misdiagnosis and delayed treatment are common. On the other hand, the eye, as one of the most important windows for expressing information about the whole body, possesses a unique anatomical structure that allows for direct observation of neurovascular tissue in vivo. This establishes a key breakthrough in linking ocular manifestations with overall health. This link is known as "oculomics," which uses macroscopic, microscopic, and molecular-level characteristics to understand the relationship between the eye and overall health. Inspired by the theory of oculomics, the eye is the only tissue that allows for non-invasive observation of neurovascular tissue in vivo, and its changes have a potential association with systemic diseases, including hepatobiliary diseases.

[0003] In recent years, the rapid development of artificial intelligence technology, especially deep learning methods, has powerfully driven the in-depth application of medical image analysis in the field of hepatobiliary diseases, and has made significant progress in several key tasks, such as lesion detection and analysis based on X-ray, computed tomography, and magnetic resonance imaging. There are also some studies attempting to use ocular images (such as fundus photography) for risk assessment of systemic diseases (such as diabetes and hypertension). However, existing technologies have the following drawbacks: 1. Technological misalignment: Most existing AI-assisted diagnostic research directly targets liver images (such as CT / MRI) rather than using the concept of "ocularomics" for indirect assessment, failing to leverage the non-invasive and convenient advantages of ocular imaging. 2. Insufficient model generalization ability: Many studies use models trained on small-scale, single-center datasets, which are prone to overfitting, resulting in significant performance degradation when applied to images acquired from different populations and devices. 3. "Black box" problem and low credibility: Most deep learning models lack interpretability; doctors cannot understand the basis for the model's diagnostic decisions, leading to a lack of trust in AI results and difficulty in integrating them into actual clinical workflows. 4. Neglecting Class Imbalance: In disease screening scenarios, healthy individuals far outnumber patients, leading to extreme class imbalance in the training data. Many models lack specific optimization strategies to address this issue, causing them to favor outputting healthy case types and resulting in low recognition rates for positive cases.

[0004] In summary, existing technologies suffer from numerous drawbacks, including being invasive, costly, lagging, and highly subjective. Existing AI methods also exhibit poor interpretability, weak generalization ability, and neglect of class imbalance. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a method and system for predicting hepatobiliary diseases based on ocular OCT images, thereby improving the accuracy of artificial intelligence models in predicting hepatobiliary diseases and enhancing the user experience.

[0006] In a first aspect, embodiments of this application provide a method for predicting hepatobiliary diseases based on ocular OCT images, comprising: acquiring original fundus OCT images; performing image preprocessing on the original fundus OCT images to obtain standardized images; inputting the standardized images into a preset image recognition model, such that the image recognition model sequentially performs feature extraction and global average pooling on the standardized images, and then outputs corresponding prediction labels through several binary classifiers in a fully connected layer, thereby generating hepatobiliary disease prediction results; wherein, the image recognition model is based on a pre-trained neural network model and several historical fundus OCT images, and is obtained by using a cross-entropy loss function with class weights and transfer learning training, wherein the class weights are set inversely proportional to the proportion of each class in the several historical fundus OCT images.

[0007] This application provides a method for predicting hepatobiliary diseases based on ocular OCT images. By introducing a pre-trained neural network model and combining it with a transfer learning method using a cross-entropy loss function with class weights, the accuracy and robustness of hepatobiliary disease prediction are effectively improved. Traditional methods often ignore the imbalance of the number of samples in each class in the dataset when dealing with multi-class classification problems. Especially in medical images, healthy samples usually dominate, while a small number of diseased samples are easily submerged, causing the model to tend to predict the common class, thereby reducing the detection rate of rare diseases. This invention sets a weight coefficient that is inversely proportional to the proportion of samples in each class, so that the model pays more attention to the rare class during the learning process, thereby alleviating the prediction bias problem caused by class imbalance. In addition, by leveraging the advantages of transfer learning, the model can make full use of visual features pre-learned on large-scale general datasets, avoiding overfitting on small-scale medical image datasets and enhancing the model's generalization ability. Finally, regarding the selection of input data, this application embodiment uses fundus OCT images as input data for the model, combining ocular omics with artificial intelligence technology. It cleverly utilizes the potential correlation between fundus image features and hepatobiliary diseases to predict hepatobiliary diseases, achieving non-invasive prediction and greatly improving the user experience while ensuring the accuracy of hepatobiliary disease prediction.

[0008] Furthermore, the step of preprocessing the original fundus OCT image to obtain a standardized image includes: converting the original fundus OCT image into RGB format to obtain a corresponding RGB image; converting the RGB image into a tensor format image based on the pixel values ​​in the RGB image; and normalizing the tensor format image according to preset normalization parameters to obtain the standardized image, wherein the normalization parameters include a mean parameter and a standard deviation parameter.

[0009] This application provides an image preprocessing method that significantly improves the quality of raw fundus OCT images and their comparability across different devices and environments by standardizing the preprocessing. Specifically, converting the images to RGB format ensures color space consistency and helps eliminate color shifts caused by differences in imaging devices. Subsequently, converting them to tensor format lays the foundation for efficient computation of deep neural networks, enabling large-scale batch processing. More importantly, normalization using preset mean and standard deviation parameters not only accelerates model convergence but also reduces image noise interference caused by factors such as light intensity and contrast. This series of preprocessing measures greatly enhances the stability and recognizability of image features, thus providing high-quality data for subsequent feature extraction and classification tasks, and improving the accuracy of artificial intelligence models in predicting hepatobiliary diseases.

[0010] In one possible implementation, the image recognition model sequentially extracts features and performs global average pooling on the standardized image, then generates a hepatobiliary disease prediction result by having several binary classifiers in a fully connected layer each output a corresponding predicted label. This includes: sequentially performing convolution and max pooling operations on the standardized image through convolutional and pooling layers to obtain initial image features; performing several feature transformations and feature fusions on the initial image features through residual blocks with several residual connections to obtain a fused feature map; performing global average pooling on the fused feature map to obtain a feature vector; outputting a corresponding predicted label based on the feature vector through several binary classifiers in a fully connected layer; and combining the predicted labels to obtain the hepatobiliary disease prediction result.

[0011] This application provides a method for prediction based on standardized images. First, through the alternating action of convolutional and pooling layers, the model can progressively capture rich feature information from local details to global structure, forming a highly abstract feature representation. Second, the introduction of residual block structures with residual connections effectively solves the common problems of gradient vanishing and information degradation in deep networks, promoting stable training of deeper networks and enhancing the efficiency of feature transfer and fusion between network layers. Subsequently, the compact feature vectors generated through global average pooling not only reduce the computational complexity of the model but also retain crucial spatial context information. Finally, through the collaborative work of multiple independent binary classifiers in fully connected layers, accurate discrimination of multiple categories of hepatobiliary diseases is achieved, significantly improving the model's flexibility and accuracy in multi-label classification tasks.

[0012] In one possible implementation, the image recognition model is based on a pre-trained neural network model and several historical fundus OCT images, and is obtained through training using a cross-entropy loss function with class weights and transfer learning. The process includes: obtaining a pre-trained ResNet18 model; setting the classifier in the fully connected layer structure of the ResNet18 model as a binary classifier according to a preset number of predicted labels, and setting the number of binary classifiers according to the number of predicted labels to obtain an initial image recognition model; adding several disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images, thereby constructing a training dataset; and training the initial image recognition model using transfer learning based on the training dataset and the cross-entropy loss function with class weights to obtain the image recognition model.

[0013] This application provides a training method for an image recognition model. Based on a pre-trained ResNet18 model and employing a carefully designed transfer learning framework, it successfully overcomes numerous challenges in training small-scale medical image datasets. By adjusting a model pre-trained on an existing large public dataset using the transfer learning framework, training time is shortened, general visual knowledge is effectively utilized, and over-reliance on specific datasets is avoided. Furthermore, by modifying the fully connected layers into a configuration of multiple binary classifiers, the model is endowed with the ability to simultaneously handle multiple hepatobiliary diseases, meeting the requirement of this application for joint prediction of multiple diseases. Simultaneously, a cross-entropy loss function with class weights is used to specifically balance the influence of samples from each class, preventing the model from being dominated by the dominant class. This end-to-end solution ensures both the model's powerful expressive ability and high adaptability to real-world application scenarios, improving training efficiency and accuracy in predicting hepatobiliary diseases.

[0014] Furthermore, the step of adding several disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images, and then constructing a training dataset, includes: adding several disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images; converting each of the historical fundus OCT images into RGB format to obtain corresponding historical RGB images; performing random rotation and random affine transformation on each of the historical RGB images to generate several transformed images, wherein the disease labels of each transformed image are consistent with the disease labels of the corresponding historical RGB images; converting each of the historical RGB images and each of the transformed images into tensor format to obtain corresponding historical tensor images; normalizing each of the historical tensor images according to preset normalization parameters to obtain corresponding historical standardized images; and constructing a training dataset based on each of the historical standardized images and the corresponding disease labels.

[0015] This application provides a method for constructing a training dataset, fully considering the scarcity of medical imaging data. Through a series of innovative data augmentation techniques, it significantly expands the diversity of available samples. By applying geometric perturbation operations such as random rotation and affine transformations to historical fundus OCT images, the model's resistance to external interference is enhanced, while simultaneously expanding the sample size. More importantly, all augmented images maintain disease labels consistent with the original images, ensuring semantic integrity during the data expansion process. Furthermore, unified normalization further eliminates the influence of non-biological variation factors, resulting in a high-quality labeled dataset. This provides a solid foundation for robust model training, improving training efficiency and the accuracy of predicting hepatobiliary diseases.

[0016] Furthermore, the method for predicting hepatobiliary diseases also includes, after obtaining the image recognition model, calculating the model performance index of the image recognition model through a preset validation set and test set, and determining whether the model performance index meets preset conditions. The model performance index includes accuracy, area under the ROC curve, specificity index, sensitivity index, and F1 score.

[0017] This application embodiment conducts a comprehensive performance evaluation of the model before deployment, covering multiple evaluation metrics such as accuracy, area under the AUC-ROC curve, specificity, sensitivity, and F1 score, to ensure that the image recognition model's performance meets expectations. This multi-dimensional quantitative analysis method can comprehensively reflect the model's performance under various conditions, which is particularly important when facing highly sensitive disease screening tasks. Through rigorous validation and testing processes, not only can potential model defects be identified and targeted optimizations made, but users can also be provided with clear confidence level references, enhancing healthcare professionals' trust in automated diagnostic tools and ensuring the accuracy of model predictions.

[0018] In one possible implementation, the hepatobiliary disease prediction method further includes generating a corresponding prediction heatmap based on the hepatobiliary disease prediction result and the original fundus OCT image, including: obtaining a target category input by the user; determining a corresponding target prediction label in the hepatobiliary disease prediction result based on the target category; obtaining a prediction score corresponding to the target prediction label from the image recognition model based on the target prediction label; using the last residual block in the image recognition model as the target feature layer, obtaining gradient information of the prediction score relative to the target feature layer; averaging the gradient information in the spatial dimension to obtain the importance weights of each channel of the target feature layer; weighted fusing the feature maps corresponding to each channel of the target feature layer according to the importance weights of each channel of the target feature layer to obtain a grayscale heatmap; and overlaying the grayscale heatmap with the original fundus OCT image to generate the prediction heatmap.

[0019] This application provides a method for generating prediction heatmaps, which largely solves the long-standing "black box" problem of deep learning models. By calculating the gradient information of the target category prediction score and mapping it back to the corresponding area of ​​the original image, the key visual cues relied upon by the model in making classification decisions are intuitively revealed. This allows clinicians to clearly observe which eye structures are most related to the pathological changes of hepatobiliary diseases, thereby deepening the understanding of disease mechanisms, verifying the logical rationality of model inferences, improving the credibility of diagnostic results, and enhancing the user experience.

[0020] Secondly, embodiments of this application provide a hepatobiliary disease prediction system based on ocular OCT images, including an acquisition module, a preprocessing module, and an image recognition module; wherein, the acquisition module is used to acquire raw fundus OCT images; the preprocessing module is used to perform image preprocessing on the raw fundus OCT images to obtain standardized images; the image recognition module is used to input the standardized images into a preset image recognition model, so that the image recognition model sequentially performs feature extraction and global average pooling on the standardized images, and then outputs corresponding prediction labels through several binary classifiers in the fully connected layer, thereby generating hepatobiliary disease prediction results; wherein, the image recognition model is based on a pre-trained neural network model and several historical fundus OCT images, and is obtained by using a cross-entropy loss function with class weights and transfer learning training, wherein the class weights are set inversely proportional to the proportion of each class in the several historical fundus OCT images.

[0021] In one possible implementation, the hepatobiliary disease prediction system further includes a model training module. This module is used to train the image recognition model based on a pre-trained neural network model and several historical fundus OCT images, employing a cross-entropy loss function with class weights and transfer learning. The training module includes: acquiring a pre-trained ResNet18 model; setting the classifier in the fully connected layer structure of the ResNet18 model as a binary classifier according to a preset number of predicted labels, and setting the number of binary classifiers according to the number of predicted labels to obtain an initial image recognition model; adding several disease labels to each of the historical fundus OCT images based on their respective disease examination results, thereby constructing a training dataset; and training the initial image recognition model using transfer learning based on the training dataset and the cross-entropy loss function with class weights to obtain the image recognition model.

[0022] In one possible implementation, the hepatobiliary disease prediction system further includes a heatmap generation module. This module generates a corresponding prediction heatmap based on the hepatobiliary disease prediction result and the original fundus OCT image. The module includes: acquiring a target category input by the user; determining a corresponding target prediction label in the hepatobiliary disease prediction result based on the target category; obtaining a prediction score corresponding to the target prediction label from the image recognition model based on the target prediction label; using the last residual block in the image recognition model as the target feature layer, obtaining gradient information of the prediction score relative to the target feature layer; averaging the gradient information in the spatial dimension to obtain the importance weights of each channel of the target feature layer; weighted fusing the feature maps corresponding to each channel of the target feature layer according to the importance weights of each channel to obtain a grayscale heatmap; and overlaying the grayscale heatmap with the original fundus OCT image to generate the prediction heatmap. Attached Figure Description

[0023] Figure 1 is a flowchart illustrating a method for predicting hepatobiliary diseases based on ocular OCT images provided in an embodiment of this application; Figure 2 is a flowchart illustrating the training process of the image recognition model in a method for predicting hepatobiliary diseases based on ocular OCT images provided in an embodiment of this application; Figure 3 is a structural diagram illustrating a system for predicting hepatobiliary diseases based on ocular OCT images provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0025] It should be noted that the step numbers in this document are only for the convenience of explaining the specific embodiments and are not intended to limit the order in which the steps are performed. In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0026] Example 1: As shown in Figure 1, Example 1 provides a method for predicting hepatobiliary diseases based on ocular OCT images, including steps S1-S3: Step S1: Obtain the original fundus OCT image; Step S2: Perform image preprocessing on the original fundus OCT image to obtain a standardized image; Step S3: Input the standardized image into a preset image recognition model, so that the image recognition model sequentially performs feature extraction and global average pooling on the standardized image, and then outputs corresponding prediction labels through several binary classifiers in the fully connected layer, thereby generating hepatobiliary disease prediction results; wherein, the image recognition model is based on a pre-trained neural network model and several historical fundus OCT images, and is obtained by using a cross-entropy loss function with class weights and transfer learning training, and the class weights are set inversely proportional to the proportion of each class in the several historical fundus OCT images.

[0027] This application provides a method for predicting hepatobiliary diseases based on ocular OCT images. By introducing a pre-trained neural network model and combining it with a transfer learning method using a cross-entropy loss function with class weights, the accuracy and robustness of hepatobiliary disease prediction are effectively improved. Traditional methods often ignore the imbalance of the number of samples in each class in the dataset when dealing with multi-class classification problems. Especially in medical images, healthy samples usually dominate, while a small number of diseased samples are easily submerged, causing the model to tend to predict the common class, thereby reducing the detection rate of rare diseases. This invention sets a weight coefficient that is inversely proportional to the proportion of samples in each class, so that the model pays more attention to the rare class during the learning process, thereby alleviating the prediction bias problem caused by class imbalance. In addition, by leveraging the advantages of transfer learning, the model can make full use of visual features pre-learned on large-scale general datasets, avoiding overfitting on small-scale medical image datasets and enhancing the model's generalization ability. Finally, regarding the selection of input data, this application embodiment uses fundus OCT images as input data for the model, combining ocular omics with artificial intelligence technology. It cleverly utilizes the potential correlation between fundus image features and hepatobiliary diseases to predict hepatobiliary diseases, achieving non-invasive prediction and greatly improving the user experience while ensuring the accuracy of hepatobiliary disease prediction.

[0028] Furthermore, in step S2, the image preprocessing of the original fundus OCT image to obtain a standardized image includes: converting the original fundus OCT image into RGB format to obtain a corresponding RGB image; converting the RGB image into a tensor format image according to the pixel values ​​in the RGB image; and normalizing the tensor format image according to preset normalization parameters to obtain the standardized image, wherein the normalization parameters include a mean parameter and a standard deviation parameter.

[0029] This application provides an image preprocessing method that significantly improves the quality of raw fundus OCT images and their comparability across different devices and environments by standardizing the preprocessing. Specifically, converting the images to RGB format ensures color space consistency and helps eliminate color shifts caused by differences in imaging devices. Subsequently, converting them to tensor format lays the foundation for efficient computation of deep neural networks, enabling large-scale batch processing. More importantly, normalization using preset mean and standard deviation parameters not only accelerates model convergence but also reduces image noise interference caused by factors such as light intensity and contrast. This series of preprocessing measures greatly enhances the stability and recognizability of image features, thus providing high-quality data for subsequent feature extraction and classification tasks, and improving the accuracy of artificial intelligence models in predicting hepatobiliary diseases.

[0030] In a preferred embodiment, the acquired raw fundus OCT image data is preprocessed to improve the model's generalization ability. Specifically, the raw fundus OCT images are uniformly converted to RGB format to ensure channel consistency, the image pixel values ​​are converted to tensor format, and the images are standardized using normalization parameters with a mean of [0.5, 0.5, 0.5] and a standard deviation of [0.5, 0.5, 0.5] to eliminate the influence of pixel value scale differences on model recognition.

[0031] In one possible implementation, in step S3, after the image recognition model sequentially extracts features and performs global average pooling on the standardized image, it outputs corresponding predicted labels through several binary classifiers in the fully connected layer, thereby generating a hepatobiliary disease prediction result. This includes: sequentially performing convolution and max pooling operations on the standardized image through convolutional and pooling layers to obtain initial image features; performing several feature transformations and feature fusions on the initial image features through several residual blocks with residual connections to obtain a fused feature map; performing global average pooling on the fused feature map to obtain a feature vector; based on the feature vector, outputting corresponding predicted labels through several binary classifiers in the fully connected layer; and combining the various predicted labels to obtain the hepatobiliary disease prediction result.

[0032] This application provides a method for prediction based on standardized images. First, through the alternating action of convolutional and pooling layers, the model can progressively capture rich feature information from local details to global structure, forming a highly abstract feature representation. Second, the introduction of residual block structures with residual connections effectively solves the common problems of gradient vanishing and information degradation in deep networks, promoting stable training of deeper networks and enhancing the efficiency of feature transfer and fusion between network layers. Subsequently, the compact feature vectors generated through global average pooling not only reduce the computational complexity of the model but also retain crucial spatial context information. Finally, through the collaborative work of multiple independent binary classifiers in fully connected layers, accurate discrimination of multiple categories of hepatobiliary diseases is achieved, significantly improving the model's flexibility and accuracy in multi-label classification tasks.

[0033] In a preferred embodiment, the internal processing flow of the image recognition model for the input image is as follows: First stage: Front-end feature extraction. The preprocessed normalized image (RGB format, normalized tensor) is fed into the model as the starting point of the entire process. The image first passes through the front-end layer of ResNet18, which typically includes a convolutional layer and a max-pooling layer, to perform basic visual feature extraction, such as low-level features like edges, textures, and colors, to obtain initial image features.

[0034] The initial image features are then sequentially passed through four layers of ResNet18 (layer 1, layer 2, layer 3, layer 4), each layer consisting of residual blocks with multiple residual connections. As the layers deepen, the network gradually extracts and combines more complex and abstract semantic features. The residual connections ensure effective gradient propagation in deep networks, mitigating the vanishing gradient problem and making it possible to train very deep networks.

[0035] The second stage: Feature map transformation and compression. The feature map output by the last residual block (layer 4) is a three-dimensional tensor (width × height × number of channels). The GAP (Global Average Pooling) layer averages the feature map of each channel across all spatial locations (width and height), compressing a channel into a single value, and finally obtaining the feature vector.

[0036] The third stage: Multi-label classification decision. The feature vector obtained after GAP is fed into a modified fully connected layer. This fully connected layer is essentially composed of multiple parallel binary classifiers. Each classifier works independently, responsible for determining a specific disease label. Specifically, the fully connected layer performs a weighted summation of the input feature vectors and adds a bias, then passes it through an activation function (such as sigmoid, suitable for multi-label binary classification) to output a probability value between 0 and 1 for each label. The model ultimately generates a list containing four probability values, for example: [probability of hepatobiliary disease, probability of chronic hepatitis, probability of cirrhosis, probability of liver cancer]. Finally, based on a preset threshold, these probabilities are converted into corresponding predicted labels, and the various predicted labels are combined to obtain the prediction result for hepatobiliary diseases.

[0037] In one possible implementation, the image recognition model is based on a pre-trained neural network model and several historical fundus OCT images, and is obtained through training using a cross-entropy loss function with class weights and transfer learning. The process includes: obtaining a pre-trained ResNet18 model; setting the classifier in the fully connected layer structure of the ResNet18 model as a binary classifier according to a preset number of predicted labels, and setting the number of binary classifiers according to the number of predicted labels to obtain an initial image recognition model; adding several disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images, thereby constructing a training dataset; and training the initial image recognition model using transfer learning based on the training dataset and the cross-entropy loss function with class weights to obtain the image recognition model.

[0038] This application provides a training method for an image recognition model. Based on a pre-trained ResNet18 model and employing a carefully designed transfer learning framework, it successfully overcomes numerous challenges in training small-scale medical image datasets. By adjusting a model pre-trained on an existing large public dataset using the transfer learning framework, training time is shortened, general visual knowledge is effectively utilized, and over-reliance on specific datasets is avoided. Furthermore, by modifying the fully connected layers into a configuration of multiple binary classifiers, the model is endowed with the ability to simultaneously handle multiple hepatobiliary diseases, meeting the requirement of this application for joint prediction of multiple diseases. Simultaneously, a cross-entropy loss function with class weights is used to specifically balance the influence of samples from each class, preventing the model from being dominated by the dominant class. This end-to-end solution ensures both the model's powerful expressive ability and high adaptability to real-world application scenarios, improving training efficiency and accuracy in predicting hepatobiliary diseases.

[0039] Furthermore, the step of adding several disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images, and then constructing a training dataset, includes: adding several disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images; converting each of the historical fundus OCT images into RGB format to obtain corresponding historical RGB images; performing random rotation and random affine transformation on each of the historical RGB images to generate several transformed images, wherein the disease labels of each transformed image are consistent with the disease labels of the corresponding historical RGB images; converting each of the historical RGB images and each of the transformed images into tensor format to obtain corresponding historical tensor images; normalizing each of the historical tensor images according to preset normalization parameters to obtain corresponding historical standardized images; and constructing a training dataset based on each of the historical standardized images and the corresponding disease labels.

[0040] This application provides a method for constructing a training dataset, fully considering the scarcity of medical imaging data. Through a series of innovative data augmentation techniques, it significantly expands the diversity of available samples. By applying geometric perturbation operations such as random rotation and affine transformations to historical fundus OCT images, the model's resistance to external interference is enhanced, while simultaneously expanding the sample size. More importantly, all augmented images maintain disease labels consistent with the original images, ensuring semantic integrity during the data expansion process. Furthermore, unified normalization further eliminates the influence of non-biological variation factors, resulting in a high-quality labeled dataset. This provides a solid foundation for robust model training, improving training efficiency and the accuracy of predicting hepatobiliary diseases.

[0041] Furthermore, the method for predicting hepatobiliary diseases also includes, after obtaining the image recognition model, calculating the model performance index of the image recognition model through a preset validation set and test set, and determining whether the model performance index meets preset conditions. The model performance index includes accuracy, area under the ROC curve, specificity index, sensitivity index, and F1 score.

[0042] This application embodiment conducts a comprehensive performance evaluation of the model before deployment, covering multiple evaluation metrics such as accuracy, area under the AUC-ROC curve, specificity, sensitivity, and F1 score, to ensure that the image recognition model's performance meets expectations. This multi-dimensional quantitative analysis method can comprehensively reflect the model's performance under various conditions, which is particularly important when facing highly sensitive disease screening tasks. Through rigorous validation and testing processes, not only can potential model defects be identified and targeted optimizations made, but users can also be provided with clear confidence level references, enhancing healthcare professionals' trust in automated diagnostic tools and ensuring the accuracy of model predictions.

[0043] In a preferred embodiment, the training process of the image recognition model is shown in Figure 2, including the following four steps: Step 1: Acquisition and preprocessing of fundus OCT image data. This involves acquiring clinical fundus OCT images generated during the medical process. The dataset contains 1164 fundus OCT images from 582 patients. Each patient is labeled with four tags based on their examination results: presence of hepatobiliary disease, presence of chronic hepatitis, presence of cirrhosis, and presence of liver cancer. The recognition model will predict each of these four tags. The final prediction results include a multi-label task covering four subcategories, ensuring that it can not only identify whether a patient has hepatobiliary disease but also further indicate the specific disease type. Next, the image data was preprocessed to improve the model's generalization ability. Specifically, the original fundus OCT images were uniformly converted to RGB format to ensure channel consistency. The dataset was expanded by random rotation (rotation angle range from -45 to 45 degrees) and random affine transformation (translation range of 20% of image width and height) to alleviate the problem of data imbalance. Then, the image pixel values ​​were converted to tensor format and normalized using normalization parameters with a mean of [0.5, 0.5, 0.5] and a standard deviation of [0.5, 0.5, 0.5] to eliminate the impact of pixel value scale differences on model training.

[0044] Step 2: Convolutional Neural Network Model Construction Based on Transfer Learning. A hepatobiliary disease identification model is constructed based on a pre-trained convolutional neural network (CNN). Transfer learning is utilized to improve feature extraction capabilities. Specifically, a pre-trained ResNet18 is used as the basic feature extraction network. This network includes convolutional layers, residual blocks, and fully connected layers. Its pre-trained weights are derived from a large-scale natural image dataset, effectively extracting abstract features from the low to high layers of the image, accelerating model convergence, and improving model performance. The fully connected layer structure of ResNet18 is modified, changing the original 1000-class output used for the ImageNet dataset to multiple 2-class outputs. The input to the fully connected layer is the number of output features from the last feature extraction layer of ResNet18, and the output is multiple 2D classification results. The modified model is then deployed to a GPU computing device, utilizing parallel computing capabilities to accelerate the model training and inference process.

[0045] Step 3: Model Training Strategy Design and Optimization. To address the imbalanced distribution of liver and gallbladder disease samples, a targeted training strategy is designed to improve model classification accuracy. Specifically, a cross-entropy loss function with class weights is used, where the class weights are set inversely proportional to the number of samples to mitigate model bias caused by sample imbalance. For example, the class weight for a certain class C can be set as follows: Where N is the total number of samples and C is the total number of categories. Let be the number of samples in class c. The Adam optimizer is used for parameter updates, with an initial learning rate set to 1e-4. A learning rate decay strategy is employed, multiplying the learning rate by 0.8 every 10 training epochs to prevent the model from getting trapped in local optima. Five-fold cross-validation is used to divide the training and validation sets. In each fold, 80% of the samples are randomly allocated to the training subset, and 20% to the validation subset. Multiple rounds of training and validation are used to improve model robustness. During training, forward propagation is performed in each epoch to calculate the loss, backpropagation is performed to update the parameters, and the mean training loss is recorded.

[0046] Step 4: Model Performance Evaluation and Result Analysis. A comprehensive evaluation of model performance is conducted using multi-dimensional indicators, generating visualized analysis results. Specifically, model performance indicators are calculated on the validation and test sets, including accuracy, area under the ROC curve (AUC), specificity, sensitivity, and F1 score. Accuracy is the ratio of correctly classified samples to the total number of samples; AUC is calculated using an ROC curve (with false positives on the horizontal axis and true positives on the vertical axis) to reflect the model's ability to distinguish between positive and negative samples; specificity is the proportion of correctly identified healthy samples; sensitivity is the proportion of correctly identified positive samples; and the F1 score is the harmonic mean of accuracy and sensitivity to comprehensively evaluate the model's classification stability. ROC curves for the validation and test sets are plotted, and AUC values ​​are labeled to visually demonstrate model performance differences and provide direction for model optimization. Furthermore, indicators are calculated for factors such as patient gender and age to reflect the model's robustness under different conditions.

[0047] In one possible implementation, the hepatobiliary disease prediction method further includes generating a corresponding prediction heatmap based on the hepatobiliary disease prediction result and the original fundus OCT image, including: obtaining a target category input by the user; determining a corresponding target prediction label in the hepatobiliary disease prediction result based on the target category; obtaining a prediction score corresponding to the target prediction label from the image recognition model based on the target prediction label; using the last residual block in the image recognition model as the target feature layer, obtaining gradient information of the prediction score relative to the target feature layer; averaging the gradient information in the spatial dimension to obtain the importance weights of each channel of the target feature layer; weighted fusing the feature maps corresponding to each channel of the target feature layer according to the importance weights of each channel of the target feature layer to obtain a grayscale heatmap; and overlaying the grayscale heatmap with the original fundus OCT image to generate the prediction heatmap.

[0048] This application provides a method for generating prediction heatmaps, which largely solves the long-standing "black box" problem of deep learning models. By calculating the gradient information of the target category prediction score and mapping it back to the corresponding area of ​​the original image, the key visual cues relied upon by the model in making classification decisions are intuitively revealed. This allows clinicians to clearly observe which eye structures are most related to the pathological changes of hepatobiliary diseases, thereby deepening the understanding of disease mechanisms, verifying the logical rationality of model inferences, improving the credibility of diagnostic results, and enhancing the user experience.

[0049] In a preferred embodiment, GradCAM (Gradient Weighted Class Activation Mapping) technology is used to generate a heatmap of the deep learning model, locating key decision-making regions to improve model reliability. Specifically, the last residual block of ResNet18 is selected as the target feature layer of GradCAM. This layer contains high-level semantic features of the image. For the input fundus OCT image, the gradient information of the class score relative to the target feature layer when the model predicts a certain target class is obtained. Then, the gradient is averaged in the spatial dimension to obtain the importance weight of each channel. These weights are then weighted and fused with the corresponding feature map, and after nonlinear activation processing, the salient regions with the strongest correlation to the class are obtained. The grayscale heatmap is then overlaid with the original fundus OCT image to generate a color fused image, visually displaying the lesion-related regions that the model focuses on. Furthermore, during model training, the fused image can be classified and stored according to the real labels and prediction results to assist in clinical analysis of the model's decision-making basis.

[0050] Example 2: As shown in Figure 3, Example 2 provides a hepatobiliary disease prediction system based on ocular OCT images, including an acquisition module 10, a preprocessing module 20, and an image recognition module 30. The acquisition module 10 acquires raw fundus OCT images. The preprocessing module 20 preprocesses the raw fundus OCT images to obtain standardized images. The image recognition module 30 inputs the standardized images into a preset image recognition model, which sequentially extracts features and performs global average pooling on the standardized images. Then, several binary classifiers in the fully connected layer output corresponding prediction labels, thereby generating hepatobiliary disease prediction results. The image recognition model is based on a pre-trained neural network model and several historical fundus OCT images, trained using a cross-entropy loss function with class weights and transfer learning. The class weights are set inversely proportional to the proportion of each class in the historical fundus OCT images.

[0051] Furthermore, the preprocessing module 20 performs image preprocessing on the original fundus OCT image to obtain a standardized image, including: converting the original fundus OCT image into RGB format to obtain a corresponding RGB image; converting the RGB image into a tensor format image according to the pixel values ​​in the RGB image; and normalizing the tensor format image according to preset normalization parameters to obtain the standardized image, wherein the normalization parameters include a mean parameter and a standard deviation parameter.

[0052] In one possible implementation, the image recognition model sequentially extracts features and performs global average pooling on the standardized image, then generates a hepatobiliary disease prediction result by having several binary classifiers in a fully connected layer each output a corresponding predicted label. This includes: sequentially performing convolution and max pooling operations on the standardized image through convolutional and pooling layers to obtain initial image features; performing several feature transformations and feature fusions on the initial image features through residual blocks with several residual connections to obtain a fused feature map; performing global average pooling on the fused feature map to obtain a feature vector; outputting a corresponding predicted label based on the feature vector through several binary classifiers in a fully connected layer; and combining the predicted labels to obtain the hepatobiliary disease prediction result.

[0053] In one possible implementation, the hepatobiliary disease prediction system further includes a model training module. This module is used to train the image recognition model based on a pre-trained neural network model and several historical fundus OCT images, employing a cross-entropy loss function with class weights and transfer learning. The training module includes: acquiring a pre-trained ResNet18 model; setting the classifier in the fully connected layer structure of the ResNet18 model as a binary classifier according to a preset number of predicted labels, and setting the number of binary classifiers according to the number of predicted labels to obtain an initial image recognition model; adding several disease labels to each of the historical fundus OCT images based on their respective disease examination results, thereby constructing a training dataset; and training the initial image recognition model using transfer learning based on the training dataset and the cross-entropy loss function with class weights to obtain the image recognition model.

[0054] Furthermore, the step of adding several disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images, and then constructing a training dataset, includes: adding several disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images; converting each of the historical fundus OCT images into RGB format to obtain corresponding historical RGB images; performing random rotation and random affine transformation on each of the historical RGB images to generate several transformed images, wherein the disease labels of each transformed image are consistent with the disease labels of the corresponding historical RGB images; converting each of the historical RGB images and each of the transformed images into tensor format to obtain corresponding historical tensor images; normalizing each of the historical tensor images according to preset normalization parameters to obtain corresponding historical standardized images; and constructing a training dataset based on each of the historical standardized images and the corresponding disease labels.

[0055] Furthermore, the hepatobiliary disease prediction system also includes, after obtaining the image recognition model, calculating the model performance index of the image recognition model through a preset validation set and test set, and determining whether the model performance index meets preset conditions. The model performance index includes accuracy, area under the ROC curve, specificity index, sensitivity index, and F1 score.

[0056] In one possible implementation, the hepatobiliary disease prediction system further includes a heatmap generation module. This module generates a corresponding prediction heatmap based on the hepatobiliary disease prediction result and the original fundus OCT image. The module includes: acquiring a target category input by the user; determining a corresponding target prediction label in the hepatobiliary disease prediction result based on the target category; obtaining a prediction score corresponding to the target prediction label from the image recognition model based on the target prediction label; using the last residual block in the image recognition model as the target feature layer, obtaining gradient information of the prediction score relative to the target feature layer; averaging the gradient information in the spatial dimension to obtain the importance weights of each channel of the target feature layer; weighted fusing the feature maps corresponding to each channel of the target feature layer according to the importance weights of each channel to obtain a grayscale heatmap; and overlaying the grayscale heatmap with the original fundus OCT image to generate the prediction heatmap.

[0057] This application provides a hepatobiliary disease prediction system based on ocular OCT images. By introducing a pre-trained neural network model and combining it with a transfer learning method using a cross-entropy loss function with class weights, the accuracy and robustness of hepatobiliary disease prediction are effectively improved. Traditional methods often ignore the imbalance of the number of samples in each class in the dataset when dealing with multi-class classification problems. Especially in medical images, healthy samples usually dominate, while a few diseased samples are easily submerged, causing the model to tend to predict the common class, thereby reducing the detection rate of rare diseases. This invention sets weight coefficients that are inversely proportional to the proportion of samples in each class, so that the model pays more attention to rare classes during the learning process, thereby alleviating the prediction bias problem caused by class imbalance. In addition, by leveraging the advantages of transfer learning, the model can fully utilize visual features pre-learned on large-scale general datasets, avoiding overfitting on small-scale medical image datasets and enhancing the model's generalization ability. Finally, regarding the selection of input data, this application embodiment uses fundus OCT images as input data for the model, combining ocular omics with artificial intelligence technology. It cleverly utilizes the potential correlation between fundus image features and hepatobiliary diseases to predict hepatobiliary diseases, achieving non-invasive prediction and greatly improving the user experience while ensuring the accuracy of hepatobiliary disease prediction.

[0058] For a more detailed explanation of the working principle and procedures of this embodiment, please refer to the relevant description in Embodiment 1.

[0059] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application for those skilled in the art.

Claims

1. A method for predicting hepatobiliary diseases based on ocular OCT images, characterized in that, include: Obtain raw fundus OCT images; perform image preprocessing on the raw fundus OCT images to obtain standardized images; The standardized image is input into a preset image recognition model, which sequentially performs feature extraction and global average pooling on the standardized image. Then, several binary classifiers in the fully connected layer output corresponding predicted labels, thereby generating a prediction result for hepatobiliary diseases. The image recognition model is based on a pre-trained neural network model and several historical fundus OCT images. It is trained using a cross-entropy loss function with class weights and transfer learning. The class weights are set inversely proportional to the proportion of each class in the several historical fundus OCT images.

2. The method for predicting hepatobiliary diseases based on ocular OCT images as described in claim 1, characterized in that, The step of preprocessing the original fundus OCT image to obtain a standardized image includes: converting the original fundus OCT image into RGB format to obtain a corresponding RGB image; converting the RGB image into a tensor format image according to the pixel values ​​in the RGB image; and normalizing the tensor format image according to preset normalization parameters to obtain the standardized image, wherein the normalization parameters include a mean parameter and a standard deviation parameter.

3. The method for predicting hepatobiliary diseases based on ocular OCT images as described in claim 1, characterized in that, The image recognition model sequentially extracts features and performs global average pooling on the standardized image, then generates a hepatobiliary disease prediction result by having several binary classifiers in the fully connected layer output corresponding predicted labels. This includes: sequentially performing convolution and max pooling operations on the standardized image through convolutional and pooling layers to obtain initial image features; performing several feature transformations and feature fusions on the initial image features through residual blocks with several residual connections to obtain a fused feature map; performing global average pooling on the fused feature map to obtain a feature vector; outputting corresponding predicted labels through several binary classifiers in the fully connected layer based on the feature vector; and combining the predicted labels to obtain the hepatobiliary disease prediction result.

4. The method for predicting hepatobiliary diseases based on ocular OCT images as described in claim 1, characterized in that, The image recognition model is based on a pre-trained neural network model and several historical fundus OCT images. It is obtained through training using a cross-entropy loss function with class weights and transfer learning, including: obtaining a pre-trained ResNet18 model; setting the classifier in the fully connected layer structure of the ResNet18 model as a binary classifier according to a preset number of predicted labels, and setting the number of binary classifiers according to the number of predicted labels to obtain an initial image recognition model; adding several disease labels to each of the historical fundus OCT images according to the disease examination results corresponding to each of the historical fundus OCT images, thereby constructing a training dataset; and training the initial image recognition model through transfer learning using the training dataset and the cross-entropy loss function with class weights to obtain the image recognition model.

5. The method for predicting hepatobiliary diseases based on ocular OCT images as described in claim 4, characterized in that, The step of adding disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images, and then constructing a training dataset, includes: adding disease labels to each of the historical fundus OCT images based on the disease examination results corresponding to each of the historical fundus OCT images; converting each of the historical fundus OCT images into RGB format to obtain corresponding historical RGB images; performing random rotation and random affine transformation on each of the historical RGB images to generate several transformed images, wherein the disease labels of each transformed image are consistent with the disease labels of the corresponding historical RGB images; converting each of the historical RGB images and each of the transformed images into tensor format to obtain corresponding historical tensor images; normalizing each of the historical tensor images according to preset normalization parameters to obtain corresponding historical standardized images; and constructing a training dataset based on each of the historical standardized images and the corresponding disease labels.

6. The method for predicting hepatobiliary diseases based on ocular OCT images as described in claim 4, characterized in that, The method for predicting hepatobiliary diseases further includes, after obtaining the image recognition model, calculating the model performance index of the image recognition model through a preset validation set and test set, and determining whether the model performance index meets preset conditions. The model performance index includes accuracy, area under the ROC curve, specificity index, sensitivity index, and F1 score.

7. A method for predicting hepatobiliary diseases based on ocular OCT images as described in any one of claims 1-6, characterized in that, The hepatobiliary disease prediction method further includes generating a corresponding prediction heatmap based on the hepatobiliary disease prediction result and the original fundus OCT image, including: obtaining a target category input by the user; determining a corresponding target prediction label in the hepatobiliary disease prediction result based on the target category; obtaining a prediction score corresponding to the target prediction label from the image recognition model based on the target prediction label; taking the last residual block in the image recognition model as the target feature layer and obtaining the gradient information of the prediction score relative to the target feature layer; averaging the gradient information in the spatial dimension to obtain the importance weight of each channel of the target feature layer; weighted fusing the feature maps corresponding to each channel of the target feature layer according to the importance weight of each channel of the target feature layer to obtain a grayscale heatmap; and overlaying the grayscale heatmap with the original fundus OCT image to generate the prediction heatmap.

8. A liver and gallbladder disease prediction system based on ocular OCT images, characterized in that, The system includes an acquisition module, a preprocessing module, and an image recognition module. The acquisition module acquires raw fundus OCT images. The preprocessing module preprocesses the raw fundus OCT images to obtain standardized images. The image recognition module inputs the standardized images into a preset image recognition model, which sequentially extracts features and performs global average pooling on the standardized images. Then, several binary classifiers in a fully connected layer output corresponding predicted labels, thereby generating prediction results for hepatobiliary diseases. The image recognition model is based on a pre-trained neural network model and several historical fundus OCT images, trained using a cross-entropy loss function with class weights and transfer learning. The class weights are inversely proportional to the proportion of each class in the historical fundus OCT images.

9. A hepatobiliary disease prediction system based on ocular OCT images as described in claim 8, characterized in that, The hepatobiliary disease prediction system further includes a model training module. This module is used to train the image recognition model based on a pre-trained neural network model and several historical fundus OCT images using a cross-entropy loss function with class weights and transfer learning. The training process includes: acquiring a pre-trained ResNet18 model; setting the classifier in the fully connected layer structure of the ResNet18 model as a binary classifier according to a preset number of predicted labels, and setting the number of binary classifiers according to the number of predicted labels to obtain an initial image recognition model; adding several disease labels to each historical fundus OCT image based on the disease examination results corresponding to each historical fundus OCT image, thereby constructing a training dataset; and training the initial image recognition model using transfer learning based on the training dataset and the cross-entropy loss function with class weights to obtain the image recognition model.

10. A hepatobiliary disease prediction system based on ocular OCT images as described in claim 8, characterized in that, The hepatobiliary disease prediction system further includes a heatmap generation module. This module generates a corresponding prediction heatmap based on the hepatobiliary disease prediction results and the original fundus OCT image. The module includes: acquiring a target category input by the user; determining a corresponding target prediction label in the hepatobiliary disease prediction results based on the target category; obtaining a prediction score corresponding to the target prediction label from the image recognition model based on the target prediction label; using the last residual block in the image recognition model as the target feature layer, obtaining gradient information of the prediction score relative to the target feature layer; averaging the gradient information in the spatial dimension to obtain the importance weights of each channel of the target feature layer; weighted fusing the feature maps corresponding to each channel of the target feature layer according to the importance weights of each channel to obtain a grayscale heatmap; and overlaying the grayscale heatmap with the original fundus OCT image to generate the prediction heatmap.

Images