Lesion classification method for magnifying endoscopic images based on cross-domain truncated transfer learning

Abstract

The present invention provides a method for classifying lesions in magnifying endoscopy images based on cross-domain truncated transfer learning, including obtaining real images and lesion categories of magnifying endoscopes, constructing training, verification and test data sets; constructing a magnifying endoscopy image lesion classification model based on cross-domain truncated transfer learning, setting a truncation point and performing training to obtain a trained magnifying endoscopy image lesion classification model; evaluating the model using a verification data set, adjusting the model parameters according to the verification results, and determining the final model; testing using a test data set, and applying the tested magnifying endoscopy image lesion classification model to an actual magnifying endoscopy image lesion classification task. This method is suitable for work on magnifying endoscopy image lesion classification, and uses a hierarchical truncation strategy and a dynamic learning rate adjustment mechanism in the model training process to improve the classification accuracy and training efficiency of the model, and has strong practicality and promotion value.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image classification, and in particular to a magnifying endoscopic image lesion classification method based on cross-domain truncated transfer learning. Background Art

[0002] In the field of computer vision, transfer learning has become a standard approach for many tasks, especially in medical image classification. The core idea of ​​transfer learning is to use a deep convolutional neural network that has been pre-trained on a large-scale dataset (such as ImageNet) and transfer its learned features to a specific task. The advantage of this approach is that the pre-trained model has already captured many common visual features, so it can be quickly adapted to new tasks by fine-tuning the model on a smaller dataset.

[0003] Most traditional transfer learning methods adopt fixed feature extraction or full-layer fine-tuning strategies. The fixed feature extraction strategy usually freezes the underlying features of the pre-trained model and only fine-tunes the last classification layer, while the full-layer fine-tuning strategy updates the parameters of all layers. However, these two methods may have limitations when dealing with complex medical image classification tasks. Because in general, the features of medical images are manifested as low-level to mid-level textures and spots, which can be captured in the bottom to middle layers of deep networks. In contrast, high-level features of pre-trained models, such as complex shapes and patterns, are not always helpful for identifying diseases. Therefore, in medical image classification, fine-tuning all layers may not be the best strategy, and may even increase computational costs and the risk of overfitting.

[0004] In order to solve these problems, a series of differentiated transfer learning strategies have emerged in recent years. These methods use different processing methods when dealing with pre-trained models at different levels. For example, TransFusion (TF) and Layer-wise Finetuning (LWFT) are two typical differentiated transfer learning methods. The TF strategy reduces the computational complexity by streamlining the top layer of the pre-trained model, while the LWFT method differentiates the learning rates of different layers during the layer-by-layer fine-tuning process to gradually optimize the model performance. Although these differentiated transfer learning strategies have improved the effect of medical image classification to a certain extent, they still have problems such as high model complexity and low reasoning efficiency.

[0005] In view of this, this application is filed. Summary of the invention

[0006] The present invention provides a magnifying endoscopic image lesion classification method based on cross-domain truncated transfer learning, which can at least partially improve the above-mentioned problems.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for classifying lesions in magnifying endoscopic images based on cross-domain truncated transfer learning, comprising:

[0009] Acquire real images of the magnifying endoscope and corresponding lesion categories, and construct a training data set, a validation training set, and a test data set according to the real images and lesion categories;

[0010] Constructing a magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning, setting hyperparameters of the model, and using the training data set to train the magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning to obtain a trained magnifying endoscopic image lesion classification model;

[0011] According to the verification training set, the trained magnifying endoscopic image lesion classification model is evaluated in each training, the performance of the model on the verification training set is evaluated, and the model parameters are adjusted according to the verification results to obtain a determined magnifying endoscopic image lesion classification model;

[0012] Using the test data set to test the determined magnified endoscopic image lesion classification model, evaluating the generalization ability of the determined magnified endoscopic image lesion classification model on independent test data, and when the evaluation result is qualified, obtaining the final magnified endoscopic image lesion classification model;

[0013] An image to be classified is acquired, and the image to be classified is input into a final magnifying endoscopic image lesion classification model to obtain a final magnifying endoscopic image lesion classification result.

[0014] In summary, the steps of the magnifying endoscopy image lesion classification method based on cross-domain truncated transfer learning specifically include: obtaining real images of magnifying endoscopes and their corresponding lesion categories, constructing a training data set, a verification training set and a test data set; constructing a magnifying endoscopy image lesion classification model based on cross-domain truncated transfer learning, training the training data set after setting a truncation point, and obtaining a trained magnifying endoscopy image lesion classification model; using the verification data set to evaluate the model in each round of training, verifying the performance of the model on the verification data set, and adjusting the model parameters according to the verification results to determine the final model; using the test data set to test the final magnifying endoscopy image lesion classification model, evaluating its generalization ability on independent test data, and applying the tested magnifying endoscopy image lesion classification model to the actual magnifying endoscopy image lesion classification task.

[0015] In short, the magnified endoscopic image lesion classification method based on cross-domain truncated transfer learning adopts the strategy of truncated transfer learning. For the task of medical image classification, the importance of low-level to mid-level features in medical images is fully considered. The present invention only reuses and fine-tunes the appropriate bottom layer in the pre-trained model to capture key information such as texture and edges, and directly discards unnecessary high-level feature layers to reduce computational complexity. By optimizing the model structure, the present invention ensures the accuracy and generalization ability of lesion classification while improving the model inference speed, and is particularly suitable for the processing and application of large-scale medical data.

[0016] Compared with the prior art, the magnified endoscopic image lesion classification method based on cross-domain truncated transfer learning has the following advantages: 1. By truncating the high-level features of the pre-trained model, only the low-level and mid-level features related to the medical image are retained, which reduces unnecessary computational overhead and makes the model more compact, thereby improving the speed of training and reasoning. This is particularly valuable for processing large-scale medical image data. 2. Aiming at the characteristics of medical image lesions, this method only fine-tunes the underlying features related to the disease, avoiding overfitting of high-level abstract features, reducing the risk of overfitting, and improving the generalization ability of the model on an independent test set, thereby improving the accuracy of lesion classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning provided by the first aspect of the present invention;

[0018] Figure 2 It is a flow chart of a method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning provided by the second aspect of the present invention;

[0019] Figure 3 It is a comparison diagram of some data sets in the source domain and the target domain and three different transfer learning strategies (LWFT, TF, TTL) provided by an embodiment of the present invention;

[0020] Figure 4 It is a structural diagram of a magnified endoscopic image lesion classification model based on cross-domain truncated transfer learning provided by an embodiment of the present invention;

[0021] Figure 5 4 is a structural diagram of a bottleneck residual block provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] refer to Figure 1 , Figure 2 As shown, the first embodiment of the present invention discloses a method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning, which can be performed by a magnified endoscopic image lesion classification device based on cross-domain truncated transfer learning (hereinafter referred to as a classification device), and in particular, is performed by one or more processors in the classification device to implement the following method:

[0024] S1, obtaining a real image of a magnifying endoscope and a corresponding lesion category, and constructing a training data set, a verification training set, and a test data set according to the real image and the lesion category;

[0025] Specifically, step S1 includes: acquiring a magnified endoscopic image of the patient as a real image, and using the acquired lesion category corresponding to the real image as a label;

[0026] A training data set, a validation training set, and a test data set are constructed according to the real images and the labels.

[0027] Preferably, the lesion categories include four categories, namely, hyperplastic polyps, adenomas, sessile serrated lesions and cancers.

[0028] In this embodiment, the real images of the magnifying endoscope and the corresponding lesion categories are obtained to construct a training data set, a verification training set, and a test data set. Specifically, the magnifying endoscope images of the patients are provided by the hospital as real images, and the lesion categories of the magnifying endoscope are provided by the doctors as labels. The obtained magnifying endoscope images and labels are processed, and then the training data set, the verification data set, and the test data set are constructed. Among them, the lesion categories of the magnifying endoscope images are divided into four categories, namely, hyperplastic polyps, adenomas, sessile serrated lesions, and cancer.

[0029] See also Figure 4 , Figure 4 It is a structural diagram of a magnified endoscopic image lesion classification model based on cross-domain truncated transfer learning in an embodiment of the present invention, showing the process of training the model on magnified endoscopic images to predict their lesion categories.

[0030] S2, constructing a magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning, setting hyperparameters of the model, and using the training data set to train the magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning to obtain a trained magnifying endoscopic image lesion classification model;

[0031] Specifically, step S2 includes: constructing a magnifying endoscopic image lesion classification model based on truncated cross-domain transfer learning, and performing data preprocessing on the training data set and the validation training set;

[0032] The hyperparameters for training the magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning are set, and the magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning is trained according to the preprocessed training data set to obtain the trained magnified endoscopic image lesion classification model.

[0033] Preferably, data preprocessing is performed on the training data set and the validation training set, specifically:

[0034] The training data set and the validation training set are initialized, and the data loading path, data processing method, and image size are set;

[0035] According to the preset training or verification mode, dynamically load the corresponding label file, parse the image path and the corresponding lesion category label in the label file, and build the corresponding relationship between the image path and the label;

[0036] Process the image in the label file, wherein when the mode is the training mode, the image is subjected to data augmentation processing, and when the mode is the verification mode, the image is subjected to resizing and center cropping processing;

[0037] Each processed image is converted into a tensor and normalized for use in the training and validation phases of the model.

[0038] Preferably, the hyperparameters for training the magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning are set, and the magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning is trained according to the preprocessed training data set to obtain the trained magnified endoscopic image lesion classification model, specifically:

[0039] Set the category weights and loss function, assign corresponding weights according to the number of samples of each lesion category in the magnified endoscopic image dataset, perform normalization, and define the cross entropy loss function. The formula is: , where C is the total number of categories, p is the predicted probability, is the model's predicted probability for category i, y is the actual label, is the actual label of category i, is the weight of category i;

[0040] Pre-train the model on the large-scale ImageNet dataset in the source domain to obtain common features, load the pre-trained model weights, and adjust the last fully connected layer of the model according to the requirements of the preset target domain, wherein the common features include edges, textures, and colors;

[0041] Set the hyperparameter cutoff index k, cut off the k bottom convolutional layers from the pre-trained model, and build a new predictor on this basis;

[0042] Configure the training hyperparameters, obtain a batch of image data and its labels from the training data loader, input them into the model, pass the data through each bottleneck residual block in turn, and restore the dimension in the 1x1 convolution layer to obtain the final classification result, where the hyperparameters include truncation point, learning rate, batch size, training rounds, optimizer and loss function;

[0043] Perform back propagation for model training, initialize back propagation, set the gradients of all model parameters to zero, forward propagate the data through the model to obtain the prediction results, calculate the loss value through the cross entropy loss function, and calculate the gradient of the loss relative to each model parameter, and use the optimizer to update the model weights according to the preset learning rate and the calculated gradient;

[0044] The model is saved and restored regularly to obtain the trained magnified endoscopic image lesion classification model, wherein the saved content includes the parameters of the current model, the optimizer state, the current epoch number, and the training loss.

[0045] In this embodiment, a magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning is constructed, and the training data set is trained after the truncation point is set to obtain the trained magnifying endoscopic image lesion classification model. Specifically, a magnifying endoscopic image lesion classification model based on truncated cross-domain transfer learning is constructed; the data of the training and validation data sets are loaded and the data is preprocessed; the hyperparameters of model training (such as truncation points) are set, and the constructed magnifying endoscopic image lesion classification model is trained and verified through the preprocessed data to obtain the trained magnifying endoscopic image lesion classification model.

[0046] First, load the data of the training and validation datasets and perform data preprocessing. Initialize the data of the training and validation datasets, set parameters such as data loading path, data processing method, and image size. According to the training or validation mode, dynamically load the corresponding label file, parse the image path and the corresponding lesion category label in the label file, and build the correspondence between the image path and the label. Secondly, process the loaded images. In the training mode, the images are subjected to data enhancement operations such as random cropping and horizontal flipping; in the validation mode, the images are first resized and then center cropped. Whether in training or validation mode, the processed images will be converted into tensors and standardized to meet the input requirements of the model. Next, the lesion category label of each image is converted into a tensor format, and the tensor label corresponding to the image is returned. Finally, the processed images and labels are used in the training and validation stages of the model.

[0047] Secondly, set the category weights and loss function. Before model training, assign corresponding weights to each lesion category based on the number of samples in the magnified endoscopic image dataset and perform normalization. Then define the cross entropy loss function, as follows: , where C is the total number of categories, is the model for the category The predicted probability of is the actual label, Yes Category By introducing the weight term into the loss function, the bias of the model towards the majority category can be reduced, so that the category with fewer samples can obtain a higher weight, thereby effectively alleviating the impact of category imbalance on model training.

[0048] Load the pre-trained model weights. The magnified endoscopic image lesion classification model is based on cross-domain transfer learning. First, the model is pre-trained on the ImageNet large-scale dataset in the source domain to learn common features (such as edges, textures, and colors). Subsequently, the weights of the pre-trained model are loaded. These weights are trained on the source domain dataset and have strong general feature extraction capabilities. According to the requirements of the target domain (magnified endoscopic image lesion classification task), the last fully connected layer of the model is adjusted to adapt to the number of categories of the target task (such as 4 lesion categories). By using the general feature extraction capabilities of the pre-trained model, the adaptation and training of the model in the target domain can be accelerated.

[0049] Set the cutoff point. During model training, by setting an appropriate hyperparameter cutoff index k, cut off the k bottom convolutional layers from the pre-trained model. , and build new predictors based on this For the ResNet50 model, we can first perform a coarse stage search, that is, set the cutoff point between each stage to get the best cutoff point. Then we can perform a fine-grained layer search, that is, search layer by layer near the best cutoff point obtained by the coarse stage search, to further optimize the model performance.

[0050] Step B24: Forward propagation of model training. Before model training, configure the training hyperparameters, including truncation point, learning rate, batch size, training rounds, optimizer, and loss function. After configuration, execute the forward propagation process of the model. Specifically: first obtain a batch of image data and its labels from the training data loader, and input them into the model. The model used for training is ResNet50 loaded with pre-trained weights. Its structure includes 4 stages, each stage consists of several bottleneck residual blocks, and the number of bottleneck blocks is 3, 4, 6, and 3 respectively. During the forward propagation process, the data passes through each bottleneck residual block in turn. As Figure 5As shown in the figure, in each residual block, the data is first reduced in dimension by a 1x1 convolution layer, then features are extracted by a 3x3 convolution layer, and finally the dimension is restored by a 1x1 convolution layer. The output of each residual block is added to the input through a residual connection to form the final output. After the data is processed in all stages, it enters the global average pooling layer and the fully connected layer to generate the final classification result.

[0051] In this embodiment, the back propagation of model training is specifically as follows: first, back propagation is initialized, and the gradients of all model parameters are set to zero during each batch training to avoid gradient accumulation. Next, the data is forward propagated through the model to obtain the prediction result, and the loss value is calculated by the cross entropy loss function. Then, the gradient of the loss relative to each parameter of the model is calculated through the back propagation process. Finally, the optimizer is used to update the weight of the model according to the preset learning rate and the calculated gradient.

[0052] Saving and restoring the model, saving the model status regularly during the training process. The saved content includes the current model parameters, optimizer status, current epoch number, and training loss. The saved model file can be used for subsequent recovery training or model evaluation.

[0053] S3, evaluating the trained magnifying endoscopic image lesion classification model in each training according to the verification training set, evaluating the performance of the model on the verification training set, and adjusting the model parameters according to the verification results to obtain a determined magnifying endoscopic image lesion classification model;

[0054] Specifically, step S3 includes: calculating the loss value and accuracy of the trained magnified endoscopic image lesion classification model for the validation set according to the validation training set to obtain an evaluation result;

[0055] According to the evaluation results, adjusting the learning rate;

[0056] When it is determined that the validation set loss has not significantly decreased within the preset training rounds, the learning rate is halved;

[0057] When it is determined that the current loss value is lower than the historical minimum value, the optimal state data of the model is saved, and the optimal state data includes weight parameters, optimizer state and training information.

[0058] In this embodiment, the model is evaluated in each round of training using a validation data set to verify the performance of the model on the validation data set, and the model parameters are adjusted based on the verification results to determine the final model.

[0059] After each round of training, the model is applied to the validation data set, and the loss value and accuracy of the validation set are calculated to reflect the generalization ability of the model on non-training data. The learning rate is adjusted dynamically based on the evaluation results. When the validation set loss does not decrease significantly within the specified training rounds, the scheduler will halve the learning rate to prevent the model from falling into a local optimum and improve the training effect. If the current loss value is lower than the historical minimum, the optimal state of the model, including weight parameters, optimizer status, and training information, is saved for subsequent recovery and optimization.

[0060] S4, using the test data set to test the determined magnified endoscopic image lesion classification model, evaluating the generalization ability of the determined magnified endoscopic image lesion classification model on independent test data, and when the evaluation result is qualified, obtaining the final magnified endoscopic image lesion classification model;

[0061] Specifically, step S4 includes: using the test data set to comprehensively evaluate the determined magnified endoscopic image lesion classification model, evaluating its generalization ability and prediction performance for unseen data, wherein each image in the test data set is classified to generate a corresponding classification result, the classification result is compared with the true label of the test data set, and multiple performance indicators of the model are calculated to obtain an evaluation result;

[0062] Determining whether the evaluation result meets predetermined performance requirements;

[0063] If so, the final magnified endoscopic image lesion classification model is obtained.

[0064] In this embodiment, the final magnified endoscopic image lesion classification model is tested using a test data set to evaluate its generalization ability on independent test data, and the tested magnified endoscopic image lesion classification model is applied to the actual magnified endoscopic image lesion classification task.

[0065] The optimal magnified endoscopic image lesion classification model is determined based on the evaluation of the validation dataset, and the model is comprehensively evaluated using a pre-divided test dataset. Specifically, the model is applied to an independent test dataset to evaluate its generalization ability and predictive performance on unseen data. During the test, the model will classify each image in the test dataset and generate corresponding classification results. By comparing with the true labels of the test dataset, multiple performance indicators of the model are calculated, including but not limited to accuracy, recall, precision, F1 score, etc. Based on these evaluation results, verify whether the model meets the predetermined performance requirements. If the test performance of the model meets expectations, the model can be applied to the actual magnified endoscopic image lesion classification task, thereby providing effective auxiliary support for clinical diagnosis and treatment decisions.

[0066] S5, obtaining an image to be classified, and inputting the image to be classified into a final magnifying endoscopic image lesion classification model to obtain a final magnifying endoscopic image lesion classification result.

[0067] In this embodiment, the magnified endoscopic image lesion classification method based on cross-domain truncated transfer learning proposes a new strategy, called truncated transfer learning (TTL), which only reuses and fine-tunes the appropriate bottom layer of the pre-trained model and directly discards unnecessary top layers. This method not only provides better performance in medical image classification tasks, but also significantly reduces the complexity and inference time of the model, and improves the inference efficiency of the model while ensuring accuracy. This is particularly important for tasks that require rapid processing and analysis of large amounts of medical data.

[0068] See also Figure 3 , Figure 3 The source domain and target domain partial data sets in the embodiment of the present invention are shown, as well as the comparison of three different transfer learning strategies (LWFT, TF, TTL). The source domain data set is ImageNet, which is a large-scale data set containing more than 14 million images, covering multiple categories of images from animals to vehicles. The pre-trained model trained with this data set can learn common visual features such as edges, textures, and colors. The data set in the target domain is magnified endoscopic images, which contains four lesion categories: hyperplastic polyps, adenomas, sessile serrated lesions, and cancer. In the figure, the LWFT strategy retains all layers of the pre-trained model, but uses differentiated fine-tuning for different layers, such as setting different learning rates for the top and bottom layers for optimization; the TF strategy simplifies the top layer to reduce the complexity of the model; and the TTL strategy directly discards the top layer and only retains the bottom layer for fine-tuning.

[0069] The advantage of the TTL method is that it can more effectively capture low-level and mid-level features in magnified endoscopic images, such as key information of lesions such as texture and spots, while discarding high-level features that are unnecessary for lesion classification. This truncation strategy not only reduces the computational complexity, but also improves the inference speed of the model, ensuring the accuracy and generalization ability of lesion classification. This makes TTL particularly suitable for processing large-scale medical image data and can play a significant role in practical applications.

[0070] In summary, the magnified endoscopic image lesion classification method based on cross-domain truncated transfer learning is suitable for the work on magnified endoscopic image lesion classification. During the model training process, the layered truncation strategy and dynamic learning rate adjustment mechanism are used to improve the classification accuracy and training efficiency of the model, which has strong practicality and promotion value.

[0071] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning, characterized in that: include: Acquire real images of the magnifying endoscope and corresponding lesion categories, and construct a training data set, a validation training set, and a test data set according to the real images and lesion categories; Constructing a magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning, setting hyperparameters of the model, and using the training data set to train the magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning to obtain a trained magnifying endoscopic image lesion classification model; According to the verification training set, the trained magnifying endoscopic image lesion classification model is evaluated in each training, the performance of the model on the verification training set is evaluated, and the model parameters are adjusted according to the verification results to obtain a determined magnifying endoscopic image lesion classification model; Using the test data set to test the determined magnified endoscopic image lesion classification model, evaluating the generalization ability of the determined magnified endoscopic image lesion classification model on independent test data, and when the evaluation result is qualified, obtaining the final magnified endoscopic image lesion classification model; Acquire an image to be classified, input the image to be classified into a final magnified endoscopic image lesion classification model, and obtain a final magnified endoscopic image lesion classification result; A magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning is constructed, the hyperparameters of the model are set, and the training data set is used to train the magnifying endoscopic image lesion classification model based on cross-domain truncated transfer learning to obtain the trained magnifying endoscopic image lesion classification model, specifically: Constructing a magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning, and performing data preprocessing on the training data set and the validation training set; Setting hyperparameters for training a magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning, and training the magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning according to the preprocessed training data set to obtain a trained magnified endoscopic image lesion classification model; The hyperparameters for training the magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning are set, and the magnified endoscopic image lesion classification model based on truncated cross-domain transfer learning is trained according to the preprocessed training data set to obtain the trained magnified endoscopic image lesion classification model, specifically: Set the category weights and loss function, assign corresponding weights according to the number of samples of each lesion category in the magnified endoscopic image dataset, perform normalization, and define the cross entropy loss function. The formula is: Where C is the total number of categories, p is the predicted probability, and p i is the model's predicted probability for category i, y is the actual label, and y i is the actual label of category i, w i is the weight of category i; Pre-train the model on the large-scale ImageNet dataset in the source domain to obtain common features, load the pre-trained model weights, and adjust the last fully connected layer of the model according to the requirements of the preset target domain, wherein the common features include edges, textures, and colors; Set the hyperparameter cutoff index k, cut off the k bottom convolutional layers from the pre-trained model, and build a new predictor on this basis; Configure the training hyperparameters, obtain a batch of image data and its labels from the training data loader, input them into the model, pass the data through each bottleneck residual block in turn, and restore the dimension in the 1x1 convolution layer to obtain the final classification result, where the hyperparameters include truncation point, learning rate, batch size, training rounds, optimizer and loss function; Perform back propagation for model training, initialize back propagation, set the gradients of all model parameters to zero, forward propagate the data through the model to obtain the prediction results, calculate the loss value through the cross entropy loss function, and calculate the gradient of the loss relative to each model parameter, and use the optimizer to update the model weights according to the preset learning rate and the calculated gradient; The model is saved and restored regularly to obtain the trained magnified endoscopic image lesion classification model, wherein the saved content includes the parameters of the current model, the optimizer state, the current epoch number, and the training loss.

2. The method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning according to claim 1, characterized in that: Obtain the real image of the magnifying endoscope and the corresponding lesion category, and construct a training data set, a verification training set, and a test data set based on the real image and the lesion category, specifically: Acquire a magnified endoscopic image of the patient as a real image, and use the acquired lesion category corresponding to the real image as a label; A training data set, a validation training set, and a test data set are constructed according to the real images and the labels.

3. The method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning according to claim 2, characterized in that: The lesion categories include four categories, namely, hyperplastic polyps, adenomas, sessile serrated lesions, and cancers.

4. The method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning according to claim 1, characterized in that: The training data set and the validation training set are subjected to data preprocessing, specifically: The training data set and the validation training set are initialized, and the data loading path, data processing method, and image size are set; According to the preset training or verification mode, dynamically load the corresponding label file, parse the image path and the corresponding lesion category label in the label file, and build the corresponding relationship between the image path and the label; Process the image in the label file, wherein when the mode is the training mode, the image is subjected to data augmentation processing, and when the mode is the verification mode, the image is subjected to resizing and center cropping processing; Each processed image is converted into a tensor and normalized for use in the training and validation phases of the model.

5. The method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning according to claim 1, characterized in that: The trained magnifying endoscopic image lesion classification model is evaluated in each training according to the verification training set, the performance of the model on the verification training set is evaluated, and the model parameters are adjusted according to the verification results to obtain a determined magnifying endoscopic image lesion classification model, specifically: Calculating the loss value and accuracy of the trained magnified endoscopic image lesion classification model for the validation set according to the validation training set to obtain an evaluation result; According to the evaluation results, adjusting the learning rate; When it is determined that the validation set loss has not significantly decreased within the preset training rounds, the learning rate is halved; When it is determined that the current loss value is lower than the historical minimum value, the optimal state data of the model is saved, and the optimal state data includes weight parameters, optimizer state and training information.

6. The method for classifying lesions in magnified endoscopic images based on cross-domain truncated transfer learning according to claim 1, characterized in that: The determined magnifying endoscopic image lesion classification model is tested using the test data set to evaluate the generalization ability of the determined magnifying endoscopic image lesion classification model on the independent test data. When the evaluation result is qualified, the final magnifying endoscopic image lesion classification model is obtained, which is specifically: Using the test data set to comprehensively evaluate the determined magnifying endoscopy image lesion classification model, and evaluate its generalization ability and prediction performance for unseen data, wherein each image in the test data set is classified to generate a corresponding classification result, the classification result is compared with the true label of the test data set, and multiple performance indicators of the model are calculated to obtain an evaluation result; Determining whether the evaluation result meets predetermined performance requirements; If so, the final magnified endoscopic image lesion classification model is obtained.

Images