Cell endoscopic image lesion classification method and system and computer equipment

By constructing a deep-density neural network model, using transfer learning and data augmentation technology, the problems of low diagnostic efficiency and insufficient accuracy of endoscopic image lesions in endoscopy are solved, efficient and intelligent lesion classification is achieved, and the diagnosis accuracy is improved. Sex and efficiency.

CN119068480APending Publication Date: 2024-12-03JILIN UNIVERSITY

Patent Information

Application Number
CN202411554436.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-04
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The diagnostic efficiency and accuracy of endoscopic image lesions in endoscopy are low and insufficient. It depends on the experience of doctors and is difficult to meet the growing demand for medical image data.

Method used

Build an endoscopic image lesion classification model based on deep dense neural networks. By acquiring and preprocessing real image data, the model is trained and adjusted using training, verification and test data sets, dynamically adjusting the learning rate, combining transfer learning and data augmentation technology , achieve efficient lesion classification.

Benefits of technology

It significantly improves the diagnostic efficiency and accuracy of endoscopic image lesions, realizes intelligent lesion classification, reduces dependence on doctor experience, and improves the accuracy and timeliness of diagnosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068480A_ABST
    Figure CN119068480A_ABST
Patent Text Reader

Abstract

The invention discloses a cell endoscopic image lesion classification method and system and computer equipment, and relates to the technical field of medical image processing. The method comprises the following steps: acquiring a real image of a cell endoscope and a lesion category corresponding to the image, and constructing a corresponding training data set, a verification data set and a test data set; preprocessing the real image and the lesion category corresponding to the image; constructing a cell endoscopic image lesion classification model based on the deep dense neural network, loading the training data set to train the model, verifying the model by using the verification data set, and evaluating the model by using the test data set; and applying the finally obtained cell endoscopic image lesion classification model to lesion classification work, and outputting a predicted classification result. According to the invention, the accuracy and timeliness of diagnosis can be improved, and more reliable auxiliary support is provided for clinical diagnosis and treatment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method, system and computer equipment for classifying lesions in cell endoscopy images. Background Art

[0002] With the widespread application of endoscopic technology, endoscopic examination has played a vital role in the diagnosis of gastrointestinal diseases, especially endocytology. Endocytology (EC) is a flexible and extendable optical microendoscope that can be inserted into the digestive tract. It can perform high-definition imaging of the mucosal tissue of the digestive tract, allowing doctors to observe the details of the diseased tissue more clearly and assist in disease diagnosis. The analysis of endoscopic lesion images is crucial for the accurate classification of lesions, but this type of work relies heavily on the doctor's experience. Manual diagnosis is easily limited by personal knowledge and experience. With the increase of medical image data, the efficiency and accuracy of manual diagnosis are difficult to meet the growing demand. Summary of the invention

[0003] The present invention provides a method, system and computer equipment for classifying lesions in cell endoscopy images, so as to solve the technical problems of low efficiency and insufficient precision of endoscopy.

[0004] According to one aspect of the present invention, a method for classifying lesions in cell endoscopic images is provided, comprising: Obtain real endoscopic images and the lesion categories corresponding to the images, and construct corresponding training data sets, validation data sets, and test data sets; Preprocessing the real images of cell endoscopy and the lesion categories corresponding to the images; Construct a cell endoscopy image lesion classification model based on a deep dense neural network, load the training data set to train the model, and after each round of training, use the validation data set to verify the model's performance on the validation data set. Adjust the model parameters based on the validation results to determine the optimal model. The optimal model is evaluated using the test data set to evaluate the generalization ability of the optimal model on the test data. The qualified model is the final cell endoscopy image lesion classification model. The finally obtained cell endoscopic image lesion classification model is applied to the lesion classification work, and the cell endoscopic image lesion classification model is used to output the predicted classification result according to the input cell endoscopic image; After each round of training, the model is verified using the validation data set to verify the performance of the model on the validation data set, and the model parameters are adjusted according to the verification results, including: After each round of training, the model is applied to the validation dataset, and the loss value and accuracy of the validation dataset are calculated to reflect the generalization ability of the model on non-training data; The learning rate is adjusted dynamically according to the calculated loss value and accuracy. When the loss of the validation data set does not decrease within the specified training rounds, the scheduler halves the learning rate to prevent the model from falling into the local optimum and improve the training effect. The scheduler's scheduling process of the learning rate is expressed by the following formula: ;in, It is in The loss value on the validation dataset after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The learning rate used in the training, It is in The learning rate used in training is Next, check whether the current validation set loss is in the past If there is no improvement, then When , the learning rate is halved, otherwise it remains The learning rate used in the training; The model status is saved periodically during the training process. The saved status content includes the parameters of the current model, the optimizer status, the current epoch number, and the training loss. The saved model status file is used for subsequent recovery training or model evaluation.

[0005] Optionally, the obtaining of a real endoscopic image and a lesion category corresponding to the image includes: The endoscopic images of patients provided by the hospital are obtained as real images, and the lesion categories of the endoscopic images provided by doctors are obtained as labels, wherein the lesion categories of the endoscopic images include hyperplastic polyps, sessile serrated lesions, adenomas, and invasive carcinomas.

[0006] Optionally, the preprocessing of the real image of the cell endoscopy and the lesion category corresponding to the image includes: According to the training or verification mode, the corresponding label file is dynamically loaded, the image path and the corresponding lesion category label in the label file are parsed, and the corresponding relationship between the image path and the lesion category label is constructed; Process the loaded image. In training mode, perform data augmentation on the image. The data augmentation operation includes random cropping and horizontal flipping. In verification mode, resize and center crop the image. The processed image is converted into a tensor and normalized to meet the input requirements of the model. The normalization formula is: ;in, is the normalized image, is the original image, and are the mean and standard deviation of the image data, respectively; The lesion category label of each image is converted into tensor format, and the tensor label corresponding to the image is returned. The processed images and lesion category labels will be used for model training and verification.

[0007] Optionally, the cell endoscopy image lesion classification model is based on cross-domain transfer learning, and the model is first pre-trained on the ImageNet large-scale dataset of the source domain to learn common features, and the loading of the training dataset to train the model includes: Load pre-training parameters, including pre-training weights and biases, wherein the pre-training parameters are trained on the source domain data set and have general feature extraction capabilities. The process formula for loading pre-training parameters is expressed as follows: ; ; ; ; and represents the weights and biases of all layers of the pre-trained model trained from the source domain dataset, and represents the weights and biases of the model used in the target domain, where represents the weight of the first layer of the pre-trained model, represents the weight of the second layer of the pre-trained model, Represents the pre-trained model N The weight of the layer, represents the bias of the first layer of the pre-trained model, represents the bias of the second layer of the pre-trained model, Represents the pre-trained model N The bias of the layer, N is the total number of layers, and It means that in the process of transfer learning, the pre-trained weights and pre-trained biases are loaded into the corresponding layers of the endoscopic image lesion classification model; According to the requirements of the endoscopic image lesion classification task, the last fully connected layer of the model is adjusted to adapt to the number of categories of the target task; Configure training hyperparameters, including learning rate, batch size, training rounds, optimizer, and loss function; Execute the forward propagation of the model, including: obtaining a batch of image data and its labels in the training data set from the training data loader, and inputting them into the model used for training. The model used for training is a deep dense neural network model loaded with pre-trained weights. In the forward propagation process, the data first passes through the initial convolution layer to perform preliminary processing on the input image, and then extracts features through 4 dense blocks. A transition layer is set between each dense block to reduce the size of the feature map. Finally, the data enters the classification layer to generate the final predicted classification result. The formula of this process is as follows: ;in, is the input batch of images, is the corresponding true category label, is a data reader, is the batch size; ;in, is the feature map after the initial convolutional layer processing, is the activation function, and are the weights and biases corresponding to the convolutional layer; ;in It is The output feature representation of the layer includes the Dense Block After processing, the feature information Is to enter The input feature representation of each dense block accumulates information by concatenating features layer by layer. ;in, It is The output feature representation of the transition layer includes Transition Layer After processing feature information, the transition layer usually consists of convolutional layers and pooling layers to reduce dimensionality and computational complexity; ;in, is the category probability distribution predicted by the model, indicating the predicted probability of each category, is the output feature map of the last dense block, and are the weights and biases corresponding to the fully connected layer, The function is used to convert the linear output into a category probability distribution; Perform back propagation of model training, including: initializing back propagation, setting the gradients of all model parameters to zero during each batch training to avoid gradient accumulation, forward propagating the data through the model to obtain the prediction results, and calculating the loss value through the cross entropy loss function, calculating the gradient of the loss relative to each model parameter through the back propagation process, and using the optimizer to update the model weights according to the preset learning rate and the calculated gradient. The formula for calculating the loss value through the cross entropy loss function is as follows: ;in, Represents the cross entropy loss value, which is used to measure the model prediction probability p With the actual label y The smaller the loss value, the closer the model's prediction result is to the true label. C is the total number of categories, is the model for the category The predicted probability of is the actual label, Yes Category The weight of .

[0008] Optionally, the optimal model is evaluated using a test data set. The generalization ability of the optimal model on the test data is evaluated including: The optimal model is applied to the test data set to evaluate the generalization ability of the model and the prediction performance of the unseen data. During the test, the model will classify each image in the test data set and generate the corresponding classification results. The performance indicators of the model are calculated by comparing the generated classification results with the real category labels of the test data set. The performance indicators include accuracy, recall, precision, and F1 score. Based on the calculation results of the performance indicators, verify whether the model meets the predetermined performance requirements, and the qualified one is the final cell endoscopy image lesion classification model; Among them, the accuracy , recall rate , Accuracy The calculation formula of F1 score is as follows: ; ; ; ;in, True positive examples are the number of samples that the model correctly predicts as positive examples; True negative examples, that is, the number of samples correctly predicted by the model as negative examples; are false positive examples, that is, the number of negative samples that the model incorrectly predicts as positive examples; is the number of false negative examples, that is, the number of positive samples that the model incorrectly predicts as negative examples.

[0009] Optionally, the finally obtained cell endoscopic image lesion classification model is applied to the lesion classification work, and the classification results predicted by the cell endoscopic image lesion classification model output according to the input cell endoscopic image include: An open source Gradio library is used to create an intelligent classification system for endoscopic images based on a deep dense neural network, wherein the intelligent classification system for endoscopic images includes a human-computer interaction interface, for users to upload endoscopic images through the human-computer interaction interface, run a endoscopic image lesion classification model for classification prediction, and display the results of the classification prediction, wherein the results of the classification prediction include predicted lesion categories and corresponding probabilities.

[0010] Optionally, the cell endoscopic image intelligent classification system is also used for the user to input a result saving instruction through the human-computer interaction interface, and save the classification prediction result into an Excel file in response to the result saving instruction, wherein the content saved in the Excel file includes the input cell endoscopic image, the predicted lesion category and probability.

[0011] According to another aspect of the present invention, a cell endoscopy image lesion classification system is also provided, comprising: The data acquisition module is used to obtain the real images of the cell endoscopy and the lesion categories corresponding to the images, and to construct the corresponding training data set, verification data set and test data set; A data preprocessing module is used to preprocess the real images of the cell endoscope and the lesion categories corresponding to the images; The model training module is used to build a cell endoscopy image lesion classification model based on a deep dense neural network, load the training data set to train the model, and after each round of training, use the verification data set to verify the model, verify the performance of the model on the verification data set, and adjust the model parameters according to the verification results to determine the optimal model; wherein, after each round of training, use the verification data set to verify the model, verify the performance of the model on the verification data set, and adjust the model parameters according to the verification results include: After each round of training, the model is applied to the validation dataset, and the loss value and accuracy of the validation dataset are calculated to reflect the generalization ability of the model on non-training data; The learning rate is adjusted dynamically according to the calculated loss value and accuracy. When the loss of the validation data set does not decrease within the specified training rounds, the scheduler halves the learning rate to prevent the model from falling into the local optimum and improve the training effect. The scheduler's scheduling process of the learning rate is expressed by the following formula: ;in, It is in The loss value on the validation dataset after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The learning rate used in the training, It is in The learning rate used in training is Next, check whether the current validation set loss is in the past If there is no improvement, then When , the learning rate is halved, otherwise it remains The learning rate used in the training; The model status is saved regularly during the training process. The saved status includes the current model parameters, optimizer status, current epoch number, and training loss. The saved model status file is used for subsequent training recovery or model evaluation. The model testing module is used to evaluate the optimal model obtained using the test data set, and evaluate the generalization ability of the optimal model on the test data. The qualified model is the final cell endoscopy image lesion classification model; The model application module is used to apply the final cell endoscopic image lesion classification model to the lesion classification work, and output the predicted classification result based on the input cell endoscopic image using the cell endoscopic image lesion classification model.

[0012] According to another aspect of the present invention, there is also provided a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above-described methods for classifying lesions in cell endoscopic images when executing the computer program.

[0013] The present invention proposes a method for classifying lesions in endoscopic images. By constructing a lesion classification model for endoscopic images based on a deep dense neural network, real endoscopic images and lesion categories corresponding to the images are collected to train and optimize the constructed model. The trained model is then applied to actual endoscopic image lesion classification work. The classification results are output by the classification model, which can help doctors quickly identify and classify different types of lesions, thereby realizing intelligent lesion classification and improving diagnostic efficiency.

[0014] The cell endoscopic image lesion classification method in the present invention is based on deep dense neural network technology and adopts a transfer learning strategy. It loads a deep dense neural network model pre-trained on the ImageNet dataset, realizes feature reuse and efficient gradient flow through its dense connection structure, and accurately extracts and classifies the lesion features of cell endoscopic images, significantly improving the classification accuracy and generalization ability of the model.

[0015] The present invention creates a model system interface through the Gradio library, allowing doctors to input cell endoscopic images through the system interface and automatically display the corresponding predicted classification results, helping doctors to quickly identify and classify different types of lesions, especially common lesion categories such as hyperplastic polyps, sessile serrated lesions, adenomas and invasive carcinomas, thereby improving the accuracy and timeliness of diagnosis and providing more reliable auxiliary support for clinical diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of a method for classifying lesions in cell endoscopy images in an embodiment of the present invention; Figure 2 It is a schematic diagram of the process of training a cell endoscopic image lesion classification model based on a deep dense neural network in an embodiment of the present invention to predict the lesion category; Figure 3 It is a schematic diagram of the structure of the initial convolution layer, dense block, transition layer, and classification layer in the model structure diagram in an embodiment of the present invention; Figure 4 Schematic diagram of the interface of the cell endoscopic image intelligent classification system based on deep dense neural network in an embodiment of the present invention; Figure 5 It is a structural block diagram of a cell endoscopy image lesion classification system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0018] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0019] The terms "comprises" and "having" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusions. For example, a process, method, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0020] Example 1, reference Figure 1 , Figure 1 is a flow chart of a method for classifying lesions in cell endoscopic images according to an embodiment of the present invention. Figure 1 As shown, the method includes: S1, data collection, including: Obtain real endoscopic images and the lesion categories corresponding to the images, and construct corresponding training data sets, validation data sets, and test data sets; The lesion category represents the diagnostic result of the cell endoscopy image, including various types of pathological changes, such as cysts, abscesses, polyps, benign tumors, malignant tumors, etc. Among them, the way to obtain the real cell endoscopy image and the lesion category corresponding to the image can be achieved in cooperation with medical institutions or research institutions. By obtaining the cell endoscopy images and the corresponding diagnostic results stored in the database of the medical institution or research institution, the lesion category of each cell endoscopy image is determined, and a data set of real images and corresponding lesion categories is established. The data set is split into a training data set, a validation data set, and a test data set using a random algorithm. The split ratio is generally 70% to 80% of the total data set for the training data set, 10% to 15% for the validation data set, and 10% to 15% for the test data set. This ratio can be adjusted according to actual conditions.

[0021] Furthermore, in this embodiment, obtaining the real image of the cell endoscopy and the lesion category corresponding to the image includes: The endoscopic images of patients provided by the hospital are obtained as real images, and the lesion categories of the endoscopic images provided by doctors are obtained as labels, wherein the lesion categories of the endoscopic images include hyperplastic polyps, sessile serrated lesions, adenomas, and invasive carcinomas.

[0022] This embodiment divides the lesion categories corresponding to cell endoscopy into four categories, which are common lesion categories under cell endoscopy.

[0023] S2, data preprocessing, including: Preprocessing the real images of cell endoscopy and the lesion categories corresponding to the images; Preprocessing includes image preprocessing and lesion category label preprocessing, wherein image preprocessing includes image cleaning, image enhancement, image standardization, etc., and label preprocessing includes label cleaning, label encoding, label standardization, etc.

[0024] S3, model training, including: Construct a cell endoscopy image lesion classification model based on a deep dense neural network, load the training data set to train the model, and after each round of training, use the validation data set to verify the model's performance on the validation data set. Adjust the model parameters based on the validation results to determine the optimal model. A deep dense neural network (DenseNet) is a convolutional neural network architecture with dense connections. It improves the efficiency and performance of the model by reusing features and reducing the number of parameters, and performs well in image classification tasks. This embodiment builds a cell endoscopic image lesion classification model based on the DenseNet architecture, especially the DenseNet201 architecture.

[0025] The main process of model training includes: setting training parameters, such as loss function, optimizer, and learning rate; performing forward propagation, inputting training data into the model, and calculating the loss between the predicted output and the true label; performing back propagation, and using the optimizer to update the model parameters according to the loss value; iterative training: repeating the forward propagation and back propagation process until the preset number of training rounds is reached or other stopping conditions are met. After each round of training, the model is evaluated using the validation data set, and performance indicators such as accuracy, recall, and F1 score are calculated. According to the validation results, hyperparameters such as learning rate, batch size, number of DenseBlock layers, and Growth Rate are adjusted to optimize the performance of the model.

[0026] S4, model testing, including: The optimal model is evaluated using the test data set to evaluate the generalization ability of the optimal model on the test data. The qualified model is the final cell endoscopy image lesion classification model. S4 is to further evaluate the optimal model obtained in S3, and apply the model obtained in S3 to an independent test dataset to evaluate its generalization ability to unseen data. The test dataset is consistent with the training dataset and the validation dataset in data distribution, but is completely independent. The images and labels in the test dataset have not been used in the previous training or validation process. The model is batch tested using an independent test dataset, and the model is judged to be qualified according to the preset performance index threshold. If the performance index of the model on the test dataset meets the requirements, it will be determined as the final cell endoscopy image lesion classification model.

[0027] S5, model application, including: The final endoscopic image lesion classification model is applied to lesion classification work. According to the input real endoscopic image, the endoscopic image lesion classification model is used to output the predicted classification result.

[0028] The embodiment of the present invention constructs a cell endoscopic image lesion classification model based on a deep dense neural network, collects real cell endoscopic images and the lesion categories corresponding to the images to train and optimize the model, and then applies the trained model to actual cell endoscopic image lesion classification work. The classification model outputs the lesion category prediction results, which can help doctors quickly identify and classify different types of lesions and improve diagnostic efficiency.

[0029] Furthermore, the preprocessing of the real image of the cell endoscopy and the lesion category corresponding to the image in S2 includes: S21, dynamically loading a corresponding label file according to a training or verification mode, parsing an image path and a corresponding lesion category label in the label file, and constructing a corresponding relationship between the image path and the lesion category label; Determine whether the current mode is training mode or verification mode, and select the corresponding label file according to the training or verification mode. The label file usually contains the path (or file name) of the image and the corresponding lesion category label. Use an appropriate parsing method (such as CSV reader, JSON parser, etc.) to read the label file content, extract the image path and lesion category label, and store the extracted image path and label in a data structure, such as a tuple of a dictionary or list, to build a correspondence between the image and the label.

[0030] S22, processing the loaded image, performing data enhancement operation on the image in training mode, wherein the data enhancement operation includes random cropping and horizontal flipping, and performing size adjustment and center cropping on the image in verification mode; Proper preprocessing and enhancement of image data is a key step to improve the generalization ability of the model. In training mode, multiple data enhancement operations are performed on the images to increase the robustness of the model; while in verification mode, more attention is paid to maintaining the consistency of the images in order to accurately evaluate the performance of the model.

[0031] In this example, in training mode, the image is randomly cropped and horizontally flipped. Random cropping and horizontal flipping are common data augmentation strategies. The purpose of these operations is to increase the diversity of data to enhance the generalization ability of the model. In verification mode, data augmentation operations are usually resizing and center cropping to ensure that the images input to the model have consistent sizes and proportions. The purpose of these operations is to ensure the consistency of the evaluation and provide a stable benchmark to evaluate the performance of the model. If random cropping and horizontal flipping are used in verification mode, the same image may be randomly cropped or flipped into different forms in different verification processes, resulting in fluctuations in verification results. The accuracy of the model evaluation results will be reduced and cannot accurately reflect the performance of the model on real data. Therefore, different processing methods are performed in different modes.

[0032] S23, the processed image is converted into a tensor and normalized to meet the input requirements of the model, wherein the formula for normalization is: ;in, is the normalized image, is the original image, and are the mean and standard deviation of the image data, respectively; Tensors are a form of data that can be processed by deep learning models. Whether in training mode or verification mode, the processed images will be converted into tensors.

[0033] S24, converts the lesion category label of each image into a tensor format and returns the tensor label corresponding to the image. The processed images and lesion category labels will be used for model training and verification.

[0034] The lesion category label of each image is also converted into a tensor format so that it can be used together with the image data for model training and verification. The conversion method can be to convert the discrete lesion category label into an integer or one-hot encoding, and then convert the encoded label into a tensor format, usually a one-dimensional tensor or a two-dimensional tensor. The processed image tensor and lesion category label tensor will be used together for model training and verification to train a deep dense neural network model that can accurately identify lesion categories.

[0035] Reference Figure 2 and Figure 3 , Figure 2 It is a schematic diagram of the process of training endoscopic images to predict lesion categories using a endoscopic image lesion classification model based on a deep dense neural network in an embodiment of the present invention. Figure 3 It is a structural schematic diagram of the initial convolutional layer, dense block, transition layer, and classification layer in the model structure diagram in an embodiment of the present invention.

[0036] The cell endoscopic image lesion classification model in the embodiment of the present invention is based on cross-domain transfer learning. The model is first pre-trained on the ImageNet large-scale dataset in the source domain to learn common features such as edges, textures, colors, etc. Among them, ImageNet is a large visual database containing millions of pictures, covering a wide range of object categories, and is one of the commonly used benchmark datasets in the field of computer vision. By pre-training on a dataset such as ImageNet, the model can learn basic and common visual features, laying the foundation for the model's subsequent performance in the classification of cell endoscopic image lesions. The pre-trained model will be migrated to the target domain, that is, the task of lesion classification of cell endoscopic images. Through cross-domain transfer learning, the dependence on a large amount of labeled data is reduced, the generalization ability of the model is improved, and the development and deployment of the model is accelerated.

[0037] Furthermore, loading the training data set in S3 to train the model includes: Load pre-trained parameters, including pre-trained weights and biases. The pre-trained parameters are trained on the source domain dataset (i.e., ImageNet) and have general feature extraction capabilities. The process formula for loading pre-trained parameters is expressed as follows: ; ; ; ; and represents the weights and biases of all layers of the pre-trained model trained from the source domain dataset, and represents the weights and biases of the model used in the target domain, where represents the weight of the first layer of the pre-trained model, represents the weight of the second layer of the pre-trained model, Represents the pre-trained model N The weight of the layer, represents the bias of the first layer of the pre-trained model, represents the bias of the second layer of the pre-trained model, Represents the pre-trained model N The bias of the layer, N is the total number of layers, and It means that in the process of transfer learning, the pre-trained weights and pre-trained biases are loaded into the corresponding layers of the endoscopic image lesion classification model; According to the requirements of the endoscopic image lesion classification task, the last fully connected layer of the model is adjusted to adapt to the number of categories of the target task; In this example, there are four target task categories, namely, hyperplastic polyps, sessile serrated lesions, adenomas, and invasive carcinomas. The weight matrix and bias vector of the last fully connected layer of the model are adjusted to match the size of the output layer, i.e., the number of neurons, each neuron corresponding to one category, to ensure that the output layer of the model can correctly reflect these categories.

[0038] Configure training hyperparameters, including learning rate, batch size, training rounds, optimizer, and loss function; Execute the forward propagation of the model, including: obtaining a batch of image data and its labels in the training data set from the training data loader, and inputting them into the model used for training. The model used for training is a deep dense neural network model loaded with pre-trained weights, namely the DenseNet201 model. DenseNet201 is an efficient convolutional neural network architecture in the field of deep learning. By connecting the output of the previous layer with the input of the current layer, dense connection is achieved. This connection method can promote the flow of information and increase the transmission of gradients, thereby improving the accuracy and stability of the network. In the forward propagation process, the data first passes through the initial convolution layer to perform preliminary processing on the input image, and extracts features through 4 dense blocks. A transition layer is set between each dense block to reduce the size of the feature map. Finally, the data enters the classification layer to generate the final predicted classification result. The formula of this process is expressed as follows: ;in, is the input batch of images, is the corresponding true category label, is a data reader, is the batch size; ;in, is the input batch of images, is the corresponding true category label, is a data reader, is the batch size; ;in, is the feature map after the initial convolutional layer processing, is the activation function, and are the weights and biases corresponding to the convolutional layer; ;in It is The output feature representation of the layer includes the Dense Block After processing, the feature information Is to enter The input feature representation of each dense block accumulates information by concatenating features layer by layer. ;in, It is The output feature representation of the transition layer includes Transition Layer After processing feature information, the transition layer usually consists of convolutional layers and pooling layers to reduce dimensionality and computational complexity; ;in, is the category probability distribution predicted by the model, indicating the predicted probability of each category, is the output feature map of the last dense block, and are the weights and biases corresponding to the fully connected layer, The function is used to convert the linear output into a category probability distribution; Perform back propagation of model training, including: initializing back propagation, setting the gradients of all model parameters to zero during each batch training to avoid gradient accumulation, forward propagating the data through the model to obtain the prediction results, and calculating the loss value through the cross entropy loss function, calculating the gradient of the loss relative to each model parameter through the back propagation process, and using the optimizer to update the model weights according to the preset learning rate and the calculated gradient. The formula for calculating the loss value through the cross entropy loss function is as follows: ;in, Represents the cross entropy loss value, which is used to measure the model prediction probability p With the actual label y The smaller the loss value, the closer the model's prediction result is to the true label. C is the total number of categories, is the model for the category The predicted probability of is the actual label, Yes Category The weight of .

[0039] After each round of training in S3, the model is verified using the validation data set to verify the performance of the model on the validation data set, and the model parameters are adjusted according to the verification results, including: After each round of training, the model is applied to the validation dataset, and the loss value and accuracy of the validation dataset are calculated to reflect the generalization ability of the model on non-training data; The learning rate is adjusted dynamically according to the calculated loss value and accuracy. When the loss of the validation data set does not decrease within the specified training rounds, the scheduler halves the learning rate to prevent the model from falling into the local optimum and improve the training effect. The scheduler's scheduling process of the learning rate can be expressed by the following formula: ;in, It is in The loss value on the validation dataset after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The learning rate used in the training, It is in The learning rate used in training is Next, check whether the current validation set loss is in the past If there is no improvement, then When , the learning rate is halved, otherwise it remains The learning rate used in the training; The model status is saved periodically during the training process. The saved status content includes the parameters of the current model, the optimizer status, the current epoch number, and the training loss. The saved model status file is used for subsequent recovery training or model evaluation.

[0040] Among them, the parameters of the current model include all weights and biases in the model network; the optimizer is responsible for updating the parameters of the model during the training process, and the saved optimizer state includes the current learning rate, gradient accumulation, etc.; epoch refers to the number of times the entire training data set is traversed once, and saving the current epoch number helps to understand the progress of training and continue to start from the right position when resuming training; training loss refers to a numerical indicator calculated based on the difference between the model prediction result and the actual label (or true value) during the training process. By saving the model state regularly during the training process, it is possible to ensure that the training is resumed when the training is interrupted, or to load the best model when the model is evaluated after the training is completed.

[0041] Furthermore, in S4, the obtained optimal model is evaluated using the test data set. The generalization ability of the optimal model on the test data is evaluated including: The optimal model is applied to an independent test dataset to evaluate the generalization ability of the model and its prediction performance on unseen data, i.e., the test dataset. During the test, the model will classify each image in the test dataset and generate corresponding classification results. The performance indicators of the model are calculated by comparing the generated classification results with the true category labels of the test dataset. The performance indicators include accuracy, recall, precision, and F1 score. Among them, accuracy refers to the proportion of samples correctly predicted by the model to the total number of samples; recall refers to the proportion of samples that are correctly predicted as positive samples among all truly positive samples; precision refers to the proportion of truly positive samples among all samples predicted as positive samples; F1 score is the harmonic mean of precision and recall, which is used to comprehensively reflect the accuracy and recall of the model. It is the harmonic mean of precision and recall. The value range of F1 score is between 0 and 1, 1 indicates the best model performance, 0 indicates the worst model performance, and the higher the F1 score, the better the model performance.

[0042] Based on the calculation results of the performance indicators, verify whether the model meets the predetermined performance requirements, and the qualified one is the final cell endoscopy image lesion classification model; The criteria for model evaluation can be: accuracy ( Accuracy ), the classification accuracy of the model on the test data set should reach 90% or above, that is, the model makes correct classification on more than 90% of the test samples; recall rate ( Recall ), for each category, the recall rate should be no less than 85% to ensure that the model has sufficient sensitivity in detecting various types of lesions; the precision rate ( Precision ), the model's precision should reach 85% or above, indicating that the model's accuracy for each type of prediction is high enough to avoid too many false positives; F1 score (F1 Score), taking into account precision and recall, the model's F1 score should reach 0.85 or higher to ensure that the model achieves a balance between the two. In other implementations, the evaluation criteria can be adjusted according to actual conditions.

[0043] Among them, the accuracy , recall rate , Accuracy The calculation formula of F1 score is as follows: ; ; ; ;in, True positive examples are the number of samples that the model correctly predicts as positive examples; True negative examples, that is, the number of samples correctly predicted by the model as negative examples; are false positive examples, that is, the number of negative samples that the model incorrectly predicts as positive examples; is a false negative example, that is, the number of positive samples that the model incorrectly predicts as negative examples, indicating that the model is underreporting.

[0044] Further, refer to Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the interface of the cell endoscopic image intelligent classification system based on a deep dense neural network in an embodiment of the present invention. Figure 4 As shown, in S5, the finally obtained cell endoscopic image lesion classification model is applied to the lesion classification work. According to the input cell endoscopic image, the cell endoscopic image lesion classification model is used to output the predicted classification results including: The open source Gradio library is used to create an intelligent classification system for endoscopic images based on a deep dense neural network. Gradio is an open source Python library for quickly building and sharing interactive demonstration interfaces for machine learning models. The intelligent classification system for endoscopic images includes a human-computer interaction interface for users to upload endoscopic images, run endoscopic image lesion classification models for classification prediction, and display classification prediction results, wherein the classification prediction results include predicted lesion categories and corresponding probabilities. The prediction result output by the model is a probability distribution, which indicates the probability that the image belongs to each preset lesion category, such as Figure 4 The prediction results are presented to users in an intuitive way. Figure 4 Based on the endoscopic images, the system gave a prediction probability of 99.6% for hyperplastic polyps, 0.1% for sessile serrated lesions, 0.2% for adenomas, and 0.1% for invasive carcinomas.

[0045] The human-computer interaction interface of the model system is created through the Gradio library, and doctors are allowed to interact with the model through a web interface. Doctors can upload endoscopic images to the system, which will quickly process and provide classification results to help doctors quickly identify and classify different types of lesions, especially common lesion categories such as hyperplastic polyps, sessile serrated lesions, adenomas and invasive carcinomas, thereby improving the accuracy and timeliness of diagnosis and providing more reliable auxiliary support for clinical diagnosis and treatment.

[0046] Furthermore, the cell endoscopic image intelligent classification system is also used for the user to input a result saving instruction through the human-computer interaction interface, and save the classification prediction result into an Excel file in response to the result saving instruction, wherein the content saved in the Excel file includes the input cell endoscopic image, the predicted lesion category and probability.

[0047] After the system displays the classification prediction results, the user can click the "Save Results" button on the system interface or select the corresponding menu item to enter the result saving command to save the displayed results to an Excel file. After the Excel file is generated and filled with the required content, the system saves it to the location specified by the user. By saving the classification prediction results, users can conduct subsequent data analysis, report generation, or exchange data with other systems.

[0048] Embodiment 2: In this embodiment, a cell endoscopy image lesion classification system is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module" and "unit" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0049] Figure 5 FIG. 4 is a structural block diagram of a cell endoscopy image lesion classification system according to an embodiment of the present invention. Figure 5 As shown, the system includes: a data acquisition module 100, a data preprocessing module 200, a model training module 300, a model testing module 400, and a model application module 500, wherein: The data acquisition module 100 is used to obtain the real image of the cell endoscope and the lesion category corresponding to the image, and to construct the corresponding training data set, verification data set and test data set; A data preprocessing module 200 is used to preprocess the real image of the cell endoscope and the lesion category corresponding to the image; The model training module 300 is used to build a cell endoscopy image lesion classification model based on a deep dense neural network, load a training data set to train the model, and after each round of training, use a validation data set to validate the model to verify the performance of the model on the validation data set, and adjust the model parameters according to the validation results to determine the optimal model; The model testing module 400 is used to evaluate the obtained optimal model using the test data set, and evaluate the generalization ability of the optimal model on the test data. The qualified model is the final cell endoscopy image lesion classification model; The model application module 500 is used to apply the finally obtained cell endoscopic image lesion classification model to the lesion classification work, and output the predicted classification result using the cell endoscopic image lesion classification model according to the input cell endoscopic real image.

[0050] An embodiment of the present invention further provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0051] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0052] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0053] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0054] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0055] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0056] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for classifying lesions in cell endoscopic images, characterized in that: include: Obtain real endoscopic images and the lesion categories corresponding to the images, and construct corresponding training data sets, validation data sets, and test data sets; Preprocessing the real images of cell endoscopy and the lesion categories corresponding to the images; Construct a cell endoscopy image lesion classification model based on a deep dense neural network, load the training data set to train the model, and after each round of training, use the validation data set to verify the model's performance on the validation data set. Adjust the model parameters based on the validation results to determine the optimal model. The optimal model is evaluated using the test data set to evaluate the generalization ability of the optimal model on the test data. The qualified model is the final cell endoscopy image lesion classification model. The finally obtained cell endoscopic image lesion classification model is applied to the lesion classification work, and the cell endoscopic image lesion classification model is used to output the predicted classification result according to the input cell endoscopic image; After each round of training, the model is verified using the validation data set to verify the performance of the model on the validation data set, and the model parameters are adjusted according to the verification results, including: After each round of training, the model is applied to the validation dataset, and the loss value and accuracy of the validation dataset are calculated to reflect the generalization ability of the model on non-training data; The learning rate is adjusted dynamically according to the calculated loss value and accuracy. When the loss of the validation data set does not decrease within the specified training rounds, the scheduler halves the learning rate to prevent the model from falling into the local optimum and improve the training effect. The scheduler's scheduling process of the learning rate is expressed by the following formula: ;in, It is in The loss value on the validation dataset after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The learning rate used in the training, It is in The learning rate used in training is Next, check whether the current validation set loss is in the past If there is no improvement, then When , the learning rate is halved, otherwise it remains The learning rate used in the training; The model status is saved periodically during the training process. The saved status content includes the parameters of the current model, the optimizer status, the current epoch number, and the training loss. The saved model status file is used for subsequent recovery training or model evaluation.

2. The method for classifying lesions in cell endoscopic images according to claim 1, characterized in that: The real image of the cell endoscopy and the lesion category corresponding to the image are obtained as follows: The endoscopic images of patients provided by the hospital are obtained as real images, and the lesion categories of the endoscopic images provided by doctors are obtained as labels, wherein the lesion categories of the endoscopic images include hyperplastic polyps, sessile serrated lesions, adenomas, and invasive carcinomas.

3. The method for classifying lesions in cell endoscopic images according to claim 1, characterized in that: The preprocessing of the real image of the cell endoscope and the lesion category corresponding to the image comprises: According to the training or verification mode, the corresponding label file is dynamically loaded, the image path and the corresponding lesion category label in the label file are parsed, and the corresponding relationship between the image path and the lesion category label is constructed; Process the loaded image. In training mode, perform data augmentation on the image. The data augmentation operation includes random cropping and horizontal flipping. In verification mode, resize and center crop the image. The processed image is converted into a tensor and normalized to meet the input requirements of the model. The normalization formula is: ;in, is the normalized image, is the original image, and are the mean and standard deviation of the image data, respectively; The lesion category label of each image is converted into tensor format, and the tensor label corresponding to the image is returned. The processed images and lesion category labels will be used for model training and verification.

4. The method for classifying lesions in cell endoscopic images according to claim 1, characterized in that: The cell endoscopy image lesion classification model is based on cross-domain transfer learning. The model is first pre-trained on the ImageNet large-scale dataset in the source domain to learn common features. The loading of the training dataset to train the model includes: Load pre-training parameters, including pre-training weights and biases, wherein the pre-training parameters are trained on the source domain data set and have general feature extraction capabilities. The process formula for loading pre-training parameters is expressed as follows: ; ; ; ; and represents the weights and biases of all layers of the pre-trained model trained from the source domain dataset, and represents the weights and biases of the model used in the target domain, where represents the weight of the first layer of the pre-trained model, represents the weight of the second layer of the pre-trained model, Represents the pre-trained model N The weight of the layer, represents the bias of the first layer of the pre-trained model, represents the bias of the second layer of the pre-trained model, Represents the pre-trained model N The bias of the layer, N is the total number of layers, and It means that in the process of transfer learning, the pre-trained weights and pre-trained biases are loaded into the corresponding layers of the endoscopic image lesion classification model; According to the requirements of the endoscopic image lesion classification task, the last fully connected layer of the model is adjusted to adapt to the number of categories of the target task; Configure training hyperparameters, including learning rate, batch size, training rounds, optimizer, and loss function; Execute the forward propagation of the model, including: obtaining a batch of image data and its labels in the training data set from the training data loader, and inputting them into the model used for training. The model used for training is a deep dense neural network model loaded with pre-trained weights. In the forward propagation process, the data first passes through the initial convolution layer to perform preliminary processing on the input image, and then extracts features through 4 dense blocks. A transition layer is set between each dense block to reduce the size of the feature map. Finally, the data enters the classification layer to generate the final predicted classification result. The formula of this process is as follows: ;in, is the input batch of images, is the corresponding true category label, is a data reader, is the batch size; ;in, is the feature map after the initial convolutional layer processing, is the activation function, and are the weights and biases corresponding to the convolutional layer; ;in It is The output feature representation of the layer includes the Dense Block After processing, the feature information Is to enter The input feature representation of each dense block accumulates information by concatenating features layer by layer. ;in, It is The output feature representation of the transition layer includes Transition Layer After processing feature information, the transition layer usually consists of convolutional layers and pooling layers to reduce dimensionality and computational complexity; ;in, is the category probability distribution predicted by the model, indicating the predicted probability of each category, is the output feature map of the last dense block, and are the weights and biases corresponding to the fully connected layer, The function is used to convert the linear output into a category probability distribution; Perform back propagation of model training, including: initializing back propagation, setting the gradients of all model parameters to zero during each batch training to avoid gradient accumulation, forward propagating the data through the model to obtain the prediction results, and calculating the loss value through the cross entropy loss function, calculating the gradient of the loss relative to each model parameter through the back propagation process, and using the optimizer to update the model weights according to the preset learning rate and the calculated gradient. The formula for calculating the loss value through the cross entropy loss function is as follows: ;in, Represents the cross entropy loss value, which is used to measure the model prediction probability p With the actual label y The smaller the loss value, the closer the model's prediction result is to the true label. C is the total number of categories, is the model for the category The predicted probability of is the actual label, Yes Category The weight of .

5. The method for classifying lesions in cell endoscopic images according to claim 1, characterized in that: The optimal model is evaluated using the test data set. The generalization ability of the optimal model on the test data includes: The optimal model is applied to the test data set to evaluate the generalization ability of the model and the prediction performance of the unseen data. During the test, the model will classify each image in the test data set and generate the corresponding classification results. The performance indicators of the model are calculated by comparing the generated classification results with the real category labels of the test data set. The performance indicators include accuracy, recall, precision, and F1 score. Based on the calculation results of the performance indicators, verify whether the model meets the predetermined performance requirements, and the qualified one is the final cell endoscopy image lesion classification model; Among them, the accuracy , recall rate , Accuracy The calculation formula of F1 score is as follows: ; ; ; ;in, True positive examples are the number of samples that the model correctly predicts as positive examples; True negative examples, that is, the number of samples correctly predicted by the model as negative examples; are false positive examples, that is, the number of negative samples that the model incorrectly predicts as positive examples; is the number of false negative examples, that is, the number of positive samples that the model incorrectly predicts as negative examples.

6. The method for classifying lesions in cell endoscopic images according to claim 1, characterized in that: The final endoscopic image lesion classification model is applied to lesion classification. According to the input endoscopic images, the endoscopic image lesion classification model is used to output the predicted classification results, including: An open source Gradio library is used to create an intelligent classification system for endoscopic images based on a deep dense neural network, wherein the intelligent classification system for endoscopic images includes a human-computer interaction interface, for users to upload endoscopic images through the human-computer interaction interface, run a endoscopic image lesion classification model for classification prediction, and display the results of the classification prediction, wherein the results of the classification prediction include predicted lesion categories and corresponding probabilities.

7. The method for classifying lesions in cell endoscopic images according to claim 6, characterized in that: The cell endoscopic image intelligent classification system is also used for the user to input a result saving instruction through the human-computer interaction interface, and save the classification prediction result into an Excel file in response to the result saving instruction, wherein the content saved in the Excel file includes the input cell endoscopic image, the predicted lesion category and probability.

8. A cell endoscopy image lesion classification system, characterized in that: include: The data acquisition module is used to obtain the real images of the cell endoscopy and the lesion categories corresponding to the images, and to construct the corresponding training data set, verification data set and test data set; A data preprocessing module is used to preprocess the real images of the cell endoscope and the lesion categories corresponding to the images; The model training module is used to build a cell endoscopy image lesion classification model based on a deep dense neural network, load the training data set to train the model, and after each round of training, use the validation data set to verify the model, verify the performance of the model on the validation data set, and adjust the model parameters according to the validation results to determine the optimal model; Among them, after each round of training, use the validation data set to verify the model, verify the performance of the model on the validation data set, and adjust the model parameters according to the validation results, including: After each round of training, the model is applied to the validation dataset, and the loss value and accuracy of the validation dataset are calculated to reflect the generalization ability of the model on non-training data; The learning rate is adjusted dynamically according to the calculated loss value and accuracy. When the loss of the validation data set does not decrease within the specified training rounds, the scheduler halves the learning rate to prevent the model from falling into the local optimum and improve the training effect. The scheduler's scheduling process of the learning rate is expressed by the following formula: ;in, It is in The loss value on the validation dataset after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The loss value on the validation set after training, It is in The learning rate used in the training, It is in The learning rate used in training is Next, check whether the current validation set loss is in the past If there is no improvement, then When , the learning rate is halved, otherwise it remains The learning rate used in the training; The model status is saved regularly during the training process. The saved status includes the current model parameters, optimizer status, current epoch number, and training loss. The saved model status file is used for subsequent training recovery or model evaluation. The model testing module is used to evaluate the optimal model obtained using the test data set, and evaluate the generalization ability of the optimal model on the test data. The qualified model is the final cell endoscopy image lesion classification model; The model application module is used to apply the final cell endoscopic image lesion classification model to the lesion classification work, and output the predicted classification result based on the input cell endoscopic image using the cell endoscopic image lesion classification model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for classifying lesions in cell endoscopic images described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Ear endoscope image neural network model construction method based on deep learning and intelligent classification processing method

    CN110796249A

  • Deep learning method for identifying oral squamous cell carcinoma based on visual features

    CN111369501A

  • Dermoscope image generation and identification method for basal cell carcinoma

    CN116630681A

Cited By

  • Multi-modal medical image classification method based on improved DenseNet

    CN119580006A

  • Method and system for constructing auxiliary prediction model of benign and malignant endometrial diseases

    CN120015308A

  • Deep transfer learning-based medical image classification model training method and system

    CN120783138A