Training method and device for image coding large model of pathological section

Abstract

The invention provides a training method and device for an image coding large model of a pathological section, and belongs to the technical field of computer vision and deep learning. The method comprises the following steps: constructing a pathological section image training set and a verification set; constructing an image coding large model for the pathological section, wherein the image coding large model is composed of a pre-training model and an additionally-added task head; freezing the pre-training model, training a task head by using the training set, and checking by using the verification set to obtain a task head after preliminary training; and unfreezing the pre-training model, performing fine tuning training on the large model by using the training set, and performing inspection by using the verification set to obtain a finally trained large model. According to the method, the task head and the pre-training model are trained in stages, so that interference of initial training on the pre-training model can be effectively avoided, finally, in a pathological section image coding task, the performance of an actual task is improved, and then the coding effect and adaptability of the model on the pathological section image are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning technology, and in particular to a training method and device for a large image coding model for pathological sections. Background Art

[0002] With the rapid development of deep learning technology, significant progress has been made in the field of image processing. In particular, large image encoding models, such as DinoV2, can learn rich image feature representations by performing pre-training on large-scale datasets. These pre-trained models perform well in feature extraction and provide a solid foundation for many computer vision tasks (such as image classification, object detection, image segmentation, etc.).

[0003] In the medical field, image analysis of pathological sections is an important part of diagnosis and treatment. Traditional image processing methods usually require training models from scratch, which consumes a lot of computing resources and time, and the performance on specific tasks may be limited by the training data and model structure. The emergence of pre-trained models has changed this situation. They can achieve transfer learning on multiple tasks, greatly improving the performance and training efficiency of the model.

[0004] Although the pre-trained model performs well in feature extraction, it is still necessary to fine-tune the model to meet the specific needs of pathological slice images. This usually involves adding a specific task layer (such as MaskFormer) to the pre-trained model and then performing training on a dataset of pathological slice images.

[0005] However, directly fine-tuning the entire model may lead to the destruction of pre-trained features, thereby affecting the overall performance of the model. The features learned by the pre-trained model on a large-scale dataset are usually very general. These features may capture basic low-level information such as edges and textures, and are applicable to various image tasks. Directly fine-tuning the entire model may change these useful general features, thereby destroying the original advantages of the model and resulting in the failure to fully utilize specific diagnosis-related features in pathological slice images. In addition, the existing pathological slice datasets are usually small, and directly fine-tuning the entire pre-trained model will result in a huge number of model parameters, which may cause overfitting problems. That is, the performance of the model on pathological slice images may become overly dependent on specific training data and lack generalization ability. Therefore, it is of considerable practical significance to develop a training method for a large model of image encoding for pathological slices. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a training method and device for a large model of image coding for pathological sections. The present invention can effectively avoid the interference of the initial training on the pre-trained model by training the task head and the pre-trained model in stages, and ultimately improve the performance of the actual task in the pathological section image coding task, thereby improving the coding effect and adaptability of the model on the pathological section image.

[0007] The first embodiment of the present invention provides a training method for a large image coding model of pathological sections, comprising:

[0008] Construct pathological slide image training set and validation set;

[0009] Constructing a large image encoding model for pathological sections, wherein the large model consists of a pre-trained model and an additional task head;

[0010] Freeze the pre-trained model, train the task head using the training set and test it using the validation set to obtain the task head that has been preliminarily trained;

[0011] The pre-trained model is unfrozen, the large model is fine-tuned using the training set and tested using the validation set to obtain the finally trained large model.

[0012] In a specific embodiment of the present invention, the image coding model adopts a pre-trained DinoV2 model, and a MaskFormer task header is added to the output layer of the DinoV2 model.

[0013] In a specific embodiment of the present invention, the step of constructing a training set and a validation set of pathological slice images includes:

[0014] 1) Acquire pathological slice images to form a pathological slice image dataset;

[0015] The pathological slice image dataset includes pathological slice images corresponding to a plurality of pathological types, wherein each pathological slice image has a consistent size;

[0016] 2) annotating each pathological slice image of the pathological slice image data set of step 1); wherein the annotation content of each pathological slice image includes a polygonal boundary of the lesion area and a corresponding lesion type label;

[0017] 3) Preprocessing the images of the pathological slice image dataset annotated in step 2) to normalize the pixel value of each image to the range of [0, 1];

[0018] 4) The pathological section data set preprocessed in step 3) is randomly divided into a training set and a validation set according to a preset ratio.

[0019] In a specific embodiment of the present invention, it also includes:

[0020] The performance of the trained image coding large model is evaluated using a preset test set, and the image coding large model is adjusted according to the evaluation result to obtain the optimized image coding large model; wherein the pathological slice images in the test set are different from the pathological slice images in the training set and the validation set.

[0021] In a specific embodiment of the present invention, it also includes:

[0022] Data enhancement is performed on the pathological slice images of the training set, including cropping, rotating, and flipping, to increase data diversity.

[0023] The second aspect of the present invention provides a training device for a large model of image coding of pathological sections, comprising:

[0024] Dataset construction module, used to construct pathology slide image training set and validation set;

[0025] A large model building module, used to build a large image encoding model for pathological sections, wherein the large model consists of a pre-trained model and an additional task head;

[0026] A task head training module, used to freeze the pre-trained model, train the task head using the training set and verify it using the verification set to obtain the task head that has been preliminarily trained;

[0027] The large model training module is used to unfreeze the pre-trained model, use the training set to fine-tune the large model and use the verification set to test it, so as to obtain the finally trained large model.

[0028] In a specific embodiment of the present invention, the image coding model adopts a pre-trained DinoV2 model, and a MaskFormer task header is added to the output layer of the DinoV2 model.

[0029] In a specific embodiment of the present invention, the step of constructing a training set and a validation set of pathological slice images includes:

[0030] 1) Acquire pathological slice images to form a pathological slice image dataset;

[0031] The pathological slice image dataset includes pathological slice images corresponding to a plurality of pathological types, wherein each pathological slice image has a consistent size;

[0032] 2) annotating each pathological slice image of the pathological slice image data set of step 1); wherein the annotation content of each pathological slice image includes a polygonal boundary of the lesion area and a corresponding lesion type label;

[0033] 3) Preprocessing the images of the pathological slice image dataset annotated in step 2) to normalize the pixel value of each image to the range of [0, 1];

[0034] 4) The pathological section data set preprocessed in step 3) is randomly divided into a training set and a validation set according to a preset ratio.

[0035] In a specific embodiment of the present invention, it also includes:

[0036] The performance of the trained image coding large model is evaluated using a preset test set, and the image coding large model is adjusted according to the evaluation result to obtain the optimized image coding large model; wherein the pathological slice images in the test set are different from the pathological slice images in the training set and the validation set.

[0037] In a specific embodiment of the present invention, it also includes:

[0038] Data enhancement is performed on the pathological slice images of the training set, including cropping, rotating, and flipping, to increase data diversity.

[0039] A third aspect of the present invention provides an electronic device, including:

[0040] at least one processor; and a memory communicatively coupled to the at least one processor;

[0041] The memory stores instructions executable by the at least one processor, and the instructions are configured to execute the above-mentioned training method for a large image coding model for pathological sections.

[0042] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for enabling the computer to execute the above-mentioned training method for a large image coding model for pathological sections.

[0043] The characteristics and beneficial effects of the present invention are:

[0044] The present invention first freezes the pre-trained model in the image coding model, and only trains the task head, so that the task head can learn preliminary features on the pathological section data; after the task head is preliminarily trained and performs well, the pre-trained model part is unfrozen, and then the entire image coding model is trained. The present invention can effectively avoid the interference of the initial training on the pre-trained model, ensure that the image coding model improves the overall performance while learning the correct pathological section features, thereby improving the coding effect and adaptability of the model on the pathological section image, and has a high application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is an overall flow chart of a method for training a large model of image coding for pathological sections according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention proposes a training method and device for a large model of image coding of pathological sections. The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0047] The first embodiment of the present invention provides a training method for a large image coding model of pathological sections, comprising:

[0048] Construct pathological slide image training set and validation set;

[0049] Constructing a large image encoding model for pathological sections, wherein the large model consists of a pre-trained model and an additional task head;

[0050] Freeze the pre-trained model, train the task head using the training set and test it using the validation set to obtain the task head that has been preliminarily trained;

[0051] The pre-trained model is unfrozen, the large model is fine-tuned using the training set and tested using the validation set to obtain the finally trained large model.

[0052] In a specific implementation of the present invention, the training method for the image encoding large model of pathological sections has the following overall process: Figure 1 As shown, the following steps are included:

[0053] 1) Construct pathological slice image training set and validation set; the specific steps are as follows:

[0054] 1-1) Acquire pathological slice images to form a pathological slice image dataset.

[0055] In this embodiment, the pathological slice image dataset contains pathological slice images corresponding to multiple pathological types, including but not limited to cancer, inflammation, benign lesions, etc., and the number of pathological slice images of each type should be greater than 100. The size of all pathological slice images in the dataset is consistent, and in a specific embodiment of the present invention, they are all 512×512 pixels to ensure consistency and standardization when inputting the model.

[0056] In a specific embodiment of the present invention, a total of 20,000 pathology slide images were collected from multiple pathology laboratories and medical institutions to form a pathology slide image dataset. In order to ensure the diversity and representativeness of the dataset, these images cover different pathological features, patient backgrounds, and sample sources.

[0057] Furthermore, in the process of collecting pathological slice images in this embodiment, special attention is paid to the following points:

[0058] Diversity of pathological types to ensure that the dataset contains a wide range of pathological types, including cancer (such as breast cancer, lung cancer, gastric cancer, etc.), where the segmentation content is mainly the cancerous area and its boundaries; inflammation (such as hepatitis, gastritis, etc.), where the segmentation content is mainly the inflammatory reaction area and its boundaries; benign lesions (such as benign tumors, cysts, etc.), where the segmentation content is mainly the lesion area and its boundaries.

[0059] Diversity of sample sources. This example collects samples from different geographical regions and medical institutions, covering different patient groups and pathological characteristics to ensure the broad representativeness of the dataset.

[0060] In addition, this embodiment selects high-quality pathological slice images to ensure the clarity and details of the images so that the model can learn more useful information.

[0061] 1-2) Annotate each pathological section image in the pathological section image dataset obtained in step 1-1).

[0062] In this embodiment, the annotation content of each pathological slice image includes the polygonal boundary of the lesion area and the corresponding lesion type label, and these annotation data will be used for subsequent model training and verification.

[0063] In a specific embodiment of the present invention, the labeling process includes: preliminary inspection, pathology experts first perform preliminary inspection on each pathology slice image to determine the existence and approximate location of the lesion area; detailed labeling, using professional labeling tools to perform detailed polygonal labeling on each lesion area and accurately outline the lesion boundary; type labeling, assigning a corresponding lesion type label to each labeled lesion area, such as cancer, inflammation or benign lesions; review, all labeling results are reviewed multiple times to ensure the accuracy and consistency of the labeling and avoid mislabeling or missing labels.

[0064] 1-3) Preprocessing the images of the pathological slice image dataset annotated in step 1-2).

[0065] In an embodiment of the present invention, the preprocessing includes: normalizing the image pixel values ​​to a range of [0, 1] to improve the training effect and convergence speed of the model.

[0066] 1-4) Divide the pathological section data set preprocessed in step 1-3) into a training set and a validation set.

[0067] In a specific implementation of the present invention, 15,000 pathological slice image data sets consisting of 20,000 pathological slice images are randomly selected to form a training set, and the remaining 5,000 images form a validation set.

[0068] Optionally, this embodiment can also perform data enhancement on the pathological slice image data set in the training set, such as random cropping, rotation, flipping, etc., to increase the diversity of the training data and improve the robustness of the model. Through these preprocessing steps, it is ensured that the pathological slice image data has a consistent format and quality when input into the model, thereby improving the training effect and performance of the model.

[0069] 2) Build a large image coding model for pathological sections.

[0070] In this embodiment, the existing pre-trained DinoV2 model is used as the main body of the image coding model, and a MaskFormer task head is added to the output layer of the DinoV2 model, and the image coding model is constructed.

[0071] It should be noted that DinoV2 is an advanced self-supervised visual representation learning model that can extract rich image features by training on large-scale image datasets. DinoV2 mainly uses Vision Transformers (ViT), also known as visual attention methods, as the architectural basis, and can efficiently process and understand the global content of images through a mechanism called "self-attention". In the process of self-supervised learning, DinoV2 uses a variety of data enhancement techniques, such as cropping, color perturbation, and Gaussian blur, to generate multiple views of the image and let the model learn the relationship between these views, so as to obtain a more in-depth and robust image representation.

[0072] In this embodiment, in the image coding model, the input of DinoV2 is a preprocessed 512×512 pixel pathological slice image, and the output is a high-dimensional image feature vector, usually a feature map of 16×16×768. This feature map is generated by DinoV2 through its deep visual feature extraction mechanism and contains rich information of the input image.

[0073] MaskFormer is an advanced image segmentation model specially designed to handle complex image segmentation tasks. In this embodiment, the input of MaskFormer is the high-dimensional image feature vector output by DinoV2, and the output is the segmentation mask of each pixel in the pathological slice image. Specifically, the MaskFormer task head consists of three layers of convolutional layers, each with a convolution kernel size of 3×3 and equipped with 8 self-attention heads. Each self-attention head can capture different feature dimensions of the image and enhance the segmentation ability of the model. The classification probability of each pixel is generated through the last softmax layer in the MaskFormer task head (is this softmax layer built into the MaskFormer task head? Or is it added at the end?), ensuring that the output segmentation mask has accurate classification results.

[0074] 3) Using the training set obtained in step 1) to train the image coding model constructed in step 2); the specific steps are as follows:

[0075] 3-1) Freeze the pre-trained model in the image encoding large model and train the task head.

[0076] In this embodiment, the pre-trained parameters of DinoV2 in the large image encoding model are frozen to ensure that these parameters remain unchanged during the training process at this stage. Among them, the parameters of the pre-trained model are frozen in order to maintain the feature extraction capabilities learned by the pre-trained model on a large-scale data set and avoid the destruction of these features during the initial training process; and only training the parameters of the task head allows the task head to focus on learning the specific features of the pathological slice image. In this way, the model can utilize the rich features learned in the pre-training stage to enhance the segmentation ability of the task head. The advantage of this method is that it can fully utilize the powerful feature extraction capabilities of the pre-trained model, while focusing on the optimization of the task head to improve the segmentation performance of the overall model.

[0077] Then, the MaskFormer task head is trained using the pathological slice image training set obtained in step 1). During the training process, the parameters of the task head are optimized by the back propagation algorithm using the same or similar loss function and optimization algorithm as the pre-trained model. Appropriate training parameters, including learning rate, training cycle, batch size, etc., are set to ensure that the task head converges stably during the training process.

[0078] In a specific embodiment of the present invention, the specific steps of training the MaskFormer task head are as follows:

[0079] First, the parameters of the MaskFormer task head are randomly initialized, and the parameters required for training are set, including the learning rate, training cycle, and batch size. In a specific embodiment of the present invention, the learning rate is set to 0.001, the training cycle is set to 50 epochs, and the batch size is set to 32. The cross entropy loss function and the Adam optimizer are selected, and the parameters of the MaskFormer task head are optimized by the back propagation algorithm. The cross entropy loss function is suitable for classification tasks and can effectively measure the difference between the predicted results and the actual labels, while the Adam optimizer ensures stability and rapid convergence during training through adaptive learning rate adjustment.

[0080] Then, the MaskFormer task head is trained using the pathological slice image training set. During the training process, the parameters of the task head are continuously adjusted so that it can accurately segment the lesion area. Through multiple rounds of iterative training, the task head gradually learns how to identify and segment the lesion area on the high-dimensional feature map, thereby improving the segmentation accuracy.

[0081] Finally, monitor the loss value and validation set performance during the training process in real time to ensure the stability and effectiveness of the training. When the loss value on the validation set no longer decreases significantly, you can consider ending the training. Monitoring the training process can help detect overfitting in a timely manner and optimize model performance by adjusting training parameters. For example, the original initial learning rate is 0.001. If the training process is too slow, it can be adjusted to 0.002. If the training process fails to converge, it can be adjusted to 0.0005.

[0082] Through the above steps, the MaskFormer task head can fully learn the specific features of pathological slice images based on the frozen DinoV2 pre-trained model, laying a solid foundation for the subsequent fine-tuning of the full model. This process not only improves the segmentation ability of the model, but also ensures the accuracy and stability of the model when processing complex pathological slice images.

[0083] When the MaskFormer task head is trained, proceed to step 3-2).

[0084] 3-2) Unfreeze the pre-trained model, and then fine-tune the large image encoding model; specifically, it includes:

[0085] Unfreeze the pre-trained parameters of the DinoV2 model so that the parameters of all layers of the DinoV2 model can be updated in the subsequent training process.

[0086] Set the parameters required for fine-tuning training, including learning rate, training cycle, and batch size. The learning rate in the fine-tuning phase should be set low to avoid large perturbations to the learned features. A commonly used learning rate is 0.0001, the training cycle can be set to 20 epochs, and the batch size can be kept at 32. Select the cross entropy loss function and the Adam optimizer to optimize the parameters of the entire model through the back-propagation algorithm.

[0087] It should be noted that the setting of the learning rate is crucial during the fine-tuning training stage. The learning rate is usually reduced in stages. The initial learning rate is high so that the model parameters can be adjusted quickly. As the training progresses, the learning rate is gradually reduced to refine the adjustment of the model parameters. This method can ensure that the model converges stably during the fine-tuning process and avoid training oscillation or overfitting.

[0088] The image coding model is fine-tuned using the pathology slice image training set. During the training process, the parameters of the pre-trained model and the task head are optimized so that they can work better together. Through multiple rounds of iterative training, the image coding model gradually learns how to segment the lesion area more accurately. The goal of fine-tuning training is to improve the performance of the model in the pathology slice image segmentation task by jointly optimizing the parameters of the pre-trained model and the task head.

[0089] The fine-tuning training process requires special attention to the validation set performance of the model. At the end of each training cycle, the large model is evaluated using the validation set, and key indicators such as loss value and segmentation accuracy are recorded. By monitoring these indicators in real time, problems in training, such as overfitting or underfitting, can be discovered in a timely manner, and optimized by adjusting the training parameters. For example, if the performance of the model on the validation set begins to decline, it may be necessary to further reduce the learning rate or increase the regularization strength to prevent the model from overfitting the training data.

[0090] Through the above steps, the pre-trained model and the task head can be jointly optimized in the pathological slice image segmentation task to improve the performance of the overall model. The fine-tuned image encoding large model not only has a powerful feature extraction capability, but also has a high-precision segmentation capability for pathological slice images, thereby meeting the needs of practical applications. The method described in this embodiment can significantly improve the overall performance and adaptability of the large model by making full use of the existing knowledge of the pre-trained model and combining the specific optimization of the task head, providing reliable technical support for actual pathological slice image analysis.

[0091] After fine-tuning is completed, the final trained image encoding model for pathological sections is obtained.

[0092] 4) Further, optionally, this embodiment can also further optimize the model by evaluating the trained large image coding model for pathological sections; the specific steps are as follows:

[0093] 4-1) Evaluate model performance;

[0094] After completing the fine-tuning training of the model, the next step is to fully evaluate the model to ensure that its performance in the pathology image segmentation task meets expectations. The evaluation process includes using a validation set and a test set to measure the segmentation accuracy and robustness of the model. Key evaluation indicators include intersection over union (IoU), pixel accuracy (PA), and Dice coefficient, which can fully reflect the performance of the model in the segmentation task.

[0095] The specific steps are as follows:

[0096] First, prepare an independent test set to ensure that the images in the test set have not been used for training and verification of the large image coding model to ensure the objectivity of the evaluation results. Then, input the pathological slice images of the test set into the trained large image coding model to generate the segmentation results of each pathological slice image in the test set. Based on the segmentation results, calculate the key evaluation indicators of the large image coding model on the test set, including intersection over union (IoU), pixel accuracy (PA), and Dice coefficient.

[0097] The intersection over union (IoU) is used to measure the overlap between the predicted segmentation and the actual segmentation, the pixel accuracy (PA) is used to measure the proportion of correctly classified pixels in the predicted results, and the Dice coefficient is used to evaluate the consistency between the predicted segmentation and the actual segmentation. By combining these evaluation indicators, we can fully understand the segmentation performance of the model.

[0098] 4-2) Optimize and adjust the model;

[0099] According to the evaluation results of step 4-1), the model is further optimized and adjusted to improve its performance in the pathological slice image segmentation task. The optimization process includes adjusting model parameters, improving training strategies, and redesigning data enhancement methods. The goal is to gradually improve the segmentation accuracy and robustness of the model through continuous iteration. The details are as follows:

[0100] First, analyze the evaluation results to identify the strengths and weaknesses of the model. For example, if the model performs poorly on certain types of lesions, consider increasing the amount of training data for these types of lesions or performing specialized parameter tuning for these types of lesions.

[0101] Then, adjust the model's hyperparameters, including learning rate, batch size, regularization coefficient, etc., and conduct multiple experiments to find the best parameter combination. You can use methods such as grid search or random search to systematically explore the effects of different parameter combinations. In addition, you can consider improving the training strategy, such as using more advanced optimization algorithms, introducing new loss functions, or increasing the diversity of training data.

[0102] Next, improve the data augmentation method to increase the diversity of training data and the robustness of the model. For example, more types of image transformation techniques can be introduced, such as color jitter, blur, noise, etc., so that the model can still maintain high performance when facing different image changes.

[0103] Finally, further fine-tuning training is performed to gradually improve the segmentation performance of the model by combining new data and optimization strategies. After each training, the performance of the model is re-evaluated and the next round of optimization is performed based on the evaluation results.

[0104] For example, the image encoding large model finally obtained in step 3) is optimized. First, an independent test set of pathological slice data is input into the model for lesion recognition and segmentation. The result of lesion segmentation is compared with the actual result of actual annotation, and the classification accuracy at the pixel level is calculated. Assuming that the average classification accuracy of different categories calculated in the end is 80%, if it is hoped that the model can better complete the recognition task and achieve a higher accuracy, for example, more than 90%, then the model needs to be adjusted. First, find out which type of lesion recognition accuracy of the large model is low, and increase the weight of the corresponding category in the loss function so that the large model can better learn the characteristics of this type of lesion. For example, if the classification rate of cell edema is low, the weight corresponding to the original edema can be doubled in the loss function. In addition, the number of different types of lesions can be counted. If the number of a certain type of lesions is too small, the large model will not be able to learn the characteristics of this type of lesions, resulting in poor classification results. For example, if there are too few samples of cell proliferation lesions, it is necessary to enhance the data of this type of samples, such as distorting, rotating, adding noise and other operations to increase the number of samples of this type, and then train the large model. After the training is completed, retest the recognition accuracy and continue to look for parts that can be optimized.

[0105] Through the above steps, the model can continuously improve its performance in the pathological slice image segmentation task and finally reach the expected performance level. The evaluation and optimization process not only helps to improve the segmentation accuracy of the model, but also enhances its adaptability and robustness in different types of pathological slice images, so as to better apply it to actual pathological image analysis tasks.

[0106] In this embodiment, the image coding model for pathological sections, after fine-tuning and optimization, has good pathological section feature extraction and coding effects, and can be applied to practical tasks such as pathological image analysis and diagnosis. In actual use, the trained model is first deployed, and then the size of the newly obtained pathological sections to be analyzed is unified to 512×512 pixels, which is input into the image coding model to obtain the model's recognition and segmentation results for different lesions in the pathological sections, thereby obtaining the analysis results of the pathological sections.

[0107] To implement the above embodiment, a second aspect of the present invention provides a training device for a large model of image coding of pathological sections, comprising:

[0108] Dataset construction module, used to construct pathology slide image training set and validation set;

[0109] A large model building module, used to build a large image encoding model for pathological sections, wherein the large model consists of a pre-trained model and an additional task head;

[0110] A task head training module, used to freeze the pre-trained model, train the task head using the training set and verify it using the verification set to obtain the task head that has been preliminarily trained;

[0111] The large model training module is used to unfreeze the pre-trained model, use the training set to fine-tune the large model and use the verification set to test it, so as to obtain the finally trained large model.

[0112] In a specific embodiment of the present invention, the image coding model adopts a pre-trained DinoV2 model, and a MaskFormer task header is added to the output layer of the DinoV2 model.

[0113] In a specific embodiment of the present invention, the step of constructing a training set and a validation set of pathological slice images includes:

[0114] 1) Acquire pathological slice images to form a pathological slice image dataset;

[0115] The pathological slice image dataset includes pathological slice images corresponding to a plurality of pathological types, wherein each pathological slice image has a consistent size;

[0116] 2) annotating each pathological slice image of the pathological slice image data set of step 1); wherein the annotation content of each pathological slice image includes a polygonal boundary of the lesion area and a corresponding lesion type label;

[0117] 3) Preprocessing the images of the pathological slice image dataset annotated in step 2) to normalize the pixel value of each image to the range of [0, 1];

[0118] 4) The pathological section data set preprocessed in step 3) is randomly divided into a training set and a validation set according to a preset ratio.

[0119] In a specific embodiment of the present invention, it also includes:

[0120] The performance of the trained image coding large model is evaluated using a preset test set, and the image coding large model is adjusted according to the evaluation result to obtain the optimized image coding large model; wherein the pathological slice images in the test set are different from the pathological slice images in the training set and the validation set.

[0121] In a specific embodiment of the present invention, it also includes:

[0122] Data enhancement is performed on the pathological slice images of the training set, including cropping, rotating, and flipping, to increase data diversity.

[0123] In this way, the task head and pre-trained model can be trained in stages, effectively avoiding the interference of the initial training on the pre-trained model, and finally improving the performance of the actual task in the pathological slice image encoding task, thereby improving the encoding effect and adaptability of the model on the pathological slice image.

[0124] To implement the above embodiment, a third aspect of the present invention provides an electronic device, including:

[0125] at least one processor; and a memory communicatively coupled to the at least one processor;

[0126] The memory stores instructions executable by the at least one processor, and the instructions are configured to execute the above-mentioned training method for a large image coding model for pathological sections.

[0127] To implement the above-mentioned embodiment, the fourth aspect of the present invention proposes a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned training method for the large image coding model for pathological sections.

[0128] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0129] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device executes a training method for a large image coding model for pathological sections in the above embodiment.

[0130] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0131] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0132] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0133] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0134] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0135] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0136] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0137] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0138] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A training method for a large model of image coding of pathological sections, characterized in that: include: Construct pathological slide image training set and validation set; Constructing a large image encoding model for pathological sections, wherein the large model consists of a pre-trained model and an additional task head; Freeze the pre-trained model, train the task head using the training set and test it using the validation set to obtain the task head that has been preliminarily trained; The pre-trained model is unfrozen, the large model is fine-tuned using the training set and tested using the validation set to obtain the finally trained large model.

2. The method according to claim 1, characterized in that The image coding model adopts the pre-trained DinoV2 model, and adds a MaskFormer task head to the output layer of the DinoV2 model.

3. The method according to claim 1, characterized in that: The step of constructing a training set and a validation set of pathological slice images includes: 1) Acquire pathological slice images to form a pathological slice image dataset; The pathological slice image dataset includes pathological slice images corresponding to a plurality of pathological types, wherein each pathological slice image has a consistent size; 2) annotating each pathological slice image of the pathological slice image data set of step 1); wherein the annotation content of each pathological slice image includes a polygonal boundary of the lesion area and a corresponding lesion type label; 3) Preprocessing the images of the pathological slice image dataset annotated in step 2) to normalize the pixel value of each image to the range of [0, 1]; 4) The pathological section data set preprocessed in step 3) is randomly divided into a training set and a validation set according to a preset ratio.

4. The method according to claim 1, characterized in that Also includes: The performance of the trained image coding large model is evaluated using a preset test set, and the image coding large model is adjusted according to the evaluation result to obtain the optimized image coding large model; wherein the pathological slice images in the test set are different from the pathological slice images in the training set and the validation set.

5. The method according to claim 3, characterized in that: Also includes: Data enhancement is performed on the pathological slice images of the training set, including cropping, rotating, and flipping, to increase data diversity.

6. A training device for a large model of image coding of pathological sections, characterized in that: include: Dataset construction module, used to construct pathology slice image training set and validation set; A large model building module, used to build an image encoding large model for pathological sections, wherein the large model consists of a pre-trained model and an additional task head; A task head training module, used to freeze the pre-trained model, train the task head using the training set and verify it using the verification set to obtain the task head that has been preliminarily trained; The large model training module is used to unfreeze the pre-trained model, use the training set to fine-tune the large model and use the verification set to test it, so as to obtain the finally trained large model.

7. The device according to claim 6, characterized in that The image coding model adopts the pre-trained DinoV2 model, and adds a MaskFormer task head to the output layer of the DinoV2 model.

8. The device according to claim 6, characterized in that The step of constructing a training set and a validation set of pathological slice images includes: 1) Acquire pathological slice images to form a pathological slice image dataset; The pathological slice image dataset includes pathological slice images corresponding to a plurality of pathological types, wherein each pathological slice image has a consistent size; 2) annotating each pathological slice image of the pathological slice image data set of step 1); wherein the annotation content of each pathological slice image includes a polygonal boundary of the lesion area and a corresponding lesion type label; 3) Preprocessing the images of the pathological slice image dataset annotated in step 2) to normalize the pixel value of each image to the range of [0, 1]; 4) The pathological section data set preprocessed in step 3) is randomly divided into a training set and a validation set according to a preset ratio.

9. The device according to claim 6, characterized in that Also includes: The performance of the trained image coding large model is evaluated using a preset test set, and the image coding large model is adjusted according to the evaluation result to obtain the optimized image coding large model; wherein the pathological slice images in the test set are different from the pathological slice images in the training set and the validation set.

10. The device according to claim 8, characterized in that Also includes: Data enhancement is performed on the pathological slice images of the training set, including cropping, rotating, and flipping, to increase data diversity.

Images