A pathological section image analysis method and device based on online learning

Abstract

This invention proposes a method and apparatus for pathological slide image analysis based on online learning, belonging to the fields of computer vision and medical image analysis. The method includes: inputting a pathological slide image into a current pathological slide analysis model to obtain and identify lesion segmentation results; selecting regions with better identification results from those with poor overall recognition performance for correction; combining the corrected portion with the corresponding local image in the pathological slide image to form an online learning sample; performing online learning on the current pathological slide analysis model to obtain an updated model; and weighted fusing the models before and after the update to obtain a new current pathological slide analysis model, thereby generating the final lesion segmentation result of the pathological slide image. This invention utilizes feedback and actual data during use to retrain and optimize the offline-trained pathological slide analysis model in real time, improving its recognition accuracy and stability.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, deep learning, and medical image analysis technology, specifically to a method and apparatus for pathological slide image analysis based on online learning. Background Technology

[0002] Traditional pathological slide analysis relies primarily on experienced pathologists' visual judgment under a microscope. While intuitive, this method has significant limitations. Subjective differences in judgment among pathologists can lead to inconsistent results. Furthermore, this manual method is inefficient; each slide requires meticulous examination, and with a large number of samples, pathologists often struggle to complete all analyses quickly. Prolonged microscopic observation can also cause eye fatigue and loss of concentration, increasing the likelihood of misdiagnosis.

[0003] With the development of computer vision and deep learning technologies, automated pathological image analysis techniques have been increasingly adopted. These techniques, through image preprocessing, feature extraction and representation, image segmentation and classification, have significantly improved the efficiency and accuracy of pathological diagnosis. However, most existing automated pathological analysis systems employ offline training, meaning that the model is trained on a fixed dataset and then no longer updated. This makes it unable to adapt to new sample data in clinical practice, limiting the model's generalization ability and adaptability. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a method and device for pathological slide image analysis based on online learning. This invention utilizes feedback from doctors and actual data during use to retrain and optimize the offline-trained pathological slide analysis model in real time, improving its recognition accuracy and stability, promoting the development of intelligent and automated pathological diagnosis, and ultimately improving the accuracy and efficiency of clinical analysis.

[0005] A first aspect of this invention proposes a method for analyzing pathological slide images based on online learning, comprising:

[0006] The newly acquired pathological slide image is input into the current pathological slide analysis model, and the model outputs the pathological slide lesion segmentation result of the image;

[0007] The pathological slide lesion segmentation results are identified, wherein regions with better identification results are selected from the pathological slide lesion segmentation results with poor overall identification results and corrected. Then, the local image of the corrected part in the corresponding pathological slide image and the corrected segmentation results are combined to form an online learning sample.

[0008] The current pathological slide analysis model is learned online using the online learning samples to obtain an updated pathological slide analysis model.

[0009] The current pathological slide analysis model and the updated pathological slide analysis model are weighted and fused to obtain a new current pathological slide analysis model.

[0010] Using the new current pathological slide analysis model, the pathological slide lesion segmentation results of the pathological slide image to be analyzed are generated;

[0011] The step of using the online learning samples to perform online learning on the current pathological slide analysis model to obtain an updated pathological slide analysis model includes:

[0012] Let D be a batch of online learning samples. new The confidence score of each labeled region in each sample is calculated. The expression for calculating the confidence score of each labeled region in a local image of one of the online learning samples is as follows:

[0013]

[0014] Where i is the index of the labeled region in the current local image, and y i,c For the i-th labeled region, Let be the predicted label for the i-th labeled region, C be the total number of lesion categories, and c be the lesion category index;

[0015] After calculating the confidence level, D new The local image in the image and the corresponding lesion label image output by the current pathological slide analysis model are denoted as follows: Where x i For the i-th pathological slide, y i For the corresponding lesion label image, n is the number of samples; D new The local image is then input again into the current pathological slide analysis model for forward propagation to obtain the output result. The cross-entropy loss function is used to calculate the error between the predicted result and the true label. The expression for the loss function is as follows:

[0016]

[0017] Backpropagation and model weight updates are performed based on the loss function calculation results. The mini-batch stochastic gradient descent method is used to update the model weights, and the weight update calculation expression is as follows:

[0018]

[0019] Where, θ tThe model parameters θ were updated before the weights were adjusted. t+1 Here are the model parameters after one weight update, and η is the learning rate;

[0020] The model obtained from each training round is saved separately. When the online learning reaches the preset number of training rounds, the model saved after each training round is used to re-identify the pathological slide images with poor recognition results. If the recognition results meet the preset requirements, the model obtained from each training round is retained as the updated pathological slide analysis model.

[0021] In one specific embodiment of the present invention, it further includes:

[0022] The initial current pathological slide analysis model was obtained through offline training, specifically including:

[0023] Construct a training set and a validation set for pathological slide images;

[0024] A pathological slide analysis model is constructed, which consists of a pre-trained model and additional task heads.

[0025] Freeze the pre-trained model, train the task head using the training set and verify it using the validation set to obtain the task head that has been initially trained;

[0026] Unfreeze the pre-trained model, fine-tune the pathological slide analysis model using the training set, and verify it using the validation set to obtain the final offline-trained pathological slide analysis model.

[0027] In one specific embodiment of the present invention, the pathological slide analysis model uses a pre-trained DinoV2 model as the main body, and a MaskFormer task header is added to the output layer of the DinoV2 model.

[0028] In a specific embodiment of the present invention, the construction of the training set and validation set of pathological slide images includes:

[0029] 1) Obtain pathological slide images to form a pathological slide image dataset;

[0030] The pathological slide image dataset contains pathological slide images corresponding to various pathological types, wherein each pathological slide image is of the same size.

[0031] 2) Label each pathological slide image in the pathological slide image dataset of step 1); wherein, the labeling content of each pathological slide image includes the polygonal boundary of the lesion area and the corresponding lesion type label.

[0032] 3) Preprocess the images in the pathological slide image dataset after step 2) annotation, and normalize the pixel values ​​of each image to the range of [0,1].

[0033] 4) The pathological slide dataset that has been preprocessed in step 3) is randomly divided into a training set and a validation set according to a preset ratio.

[0034] In one specific embodiment of the present invention, it further includes:

[0035] The performance of the pathological slide analysis model trained offline is evaluated using a preset test set. Based on the evaluation results, the pathological slide analysis model is adjusted to obtain an optimized pathological slide analysis model. The pathological slide images in the test set are different from those in the training set and the validation set.

[0036] In one specific embodiment of the present invention, it further includes:

[0037] Using a multi-model voting mechanism, the same pathological slide image is identified using the pathological slide analysis model before adjustment and the pathological slide analysis model saved from each round of training, generating multiple identification results.

[0038] The recognition results of multiple models are voted on, and the multiple models are weighted and fused based on the confidence level in the voting results;

[0039] The expression for obtaining the voting results is as follows:

[0040]

[0041] Among them, v m,c Let I be the voting result of the m-th model for pathological category c, I be the total number of labeled regions in the current slice, i be the index of the labeled region, and l be the index of the labeled region. m,i,c Let l be the determination result made by the m-th model in the i-th labeled region for pathological category c. That is, if the labeled region belongs to pathological category c, then l m,i,c =1, otherwise, l m,i,c =0;

[0042] The fusion weights for each model are determined based on the voting results. The weight calculation expression is as follows:

[0043]

[0044] Among them, w m Let v be the weight of the m-th model. m,c For the voting results, y m,c Let be the confidence level of the m-th model for pathological category c;

[0045] Then, based on the calculated weights, the models before and after adjustment are fused together. The fusion expression is as follows:

[0046]

[0047] Where, θ fusion Here, M represents the parameters of the fused model, M represents the total number of training epochs when using online learning for a single adjustment, and θ represents the parameters of the fused model. m For the parameters of the m-th model, w m Let M1 be the weight of the m-th model; M0 is the pathological slide analysis model before a single adjustment using online learning, and the parameter of model M0 is θ0; M1 to M... m The pathological slide analysis models saved during each round of online learning training, models M1 to M... m The parameters are θ1 to θ m .

[0048] A second aspect of the present invention provides a pathological slide image analysis device based on online learning, comprising:

[0049] The preliminary pathological section analysis module is used to input newly acquired pathological section images into the current pathological section analysis model, and the model outputs the pathological section lesion segmentation results of the image;

[0050] The online learning sample construction module is used to identify the lesion segmentation results of the pathological slides. It selects regions with better identification results from the lesion segmentation results of the pathological slides with poor overall identification results and corrects them. Then, the local image of the corrected part in the corresponding pathological slide image and the corrected segmentation results are combined to form an online learning sample.

[0051] The online learning module is used to learn the current pathological slide analysis model online using the online learning samples to obtain an updated pathological slide analysis model.

[0052] The fusion module is used to perform weighted fusion of the current pathological slide analysis model and the updated pathological slide analysis model to obtain a new current pathological slide analysis model.

[0053] The pathological slide final analysis module is used to generate pathological slide lesion segmentation results of the pathological slide image to be analyzed using the new current pathological slide analysis model.

[0054] The step of using the online learning samples to perform online learning on the current pathological slide analysis model to obtain an updated pathological slide analysis model includes:

[0055] Let D be a batch of online learning samples. newThe confidence score of each labeled region in each sample is calculated. The expression for calculating the confidence score of each labeled region in a local image of one of the online learning samples is as follows:

[0056]

[0057] Where i is the index of the labeled region in the current local image, and y i,c For the i-th labeled region, Let be the predicted label for the i-th labeled region, C be the total number of lesion categories, and c be the lesion category index;

[0058] After calculating the confidence level, D new The local image in the image and the corresponding lesion label image output by the current pathological slide analysis model are denoted as follows: where x i For the i-th pathological slide, y i For the corresponding lesion label image, n is the number of samples; D new The local image is then input again into the current pathological slide analysis model for forward propagation to obtain the output result. The cross-entropy loss function is used to calculate the error between the predicted result and the true label. The expression for the loss function is as follows:

[0059]

[0060] Backpropagation and model weight updates are performed based on the loss function calculation results. The mini-batch stochastic gradient descent method is used to update the model weights, and the weight update calculation expression is as follows:

[0061]

[0062] Where, θ t The model parameters θ were updated before the weights were adjusted. t+1 Here are the model parameters after one weight update, and η is the learning rate;

[0063] The model obtained from each training round is saved separately. When the online learning reaches the preset number of training rounds, the model saved after each training round is used to re-identify the pathological slide images with poor recognition results. If the recognition results meet the preset requirements, the model obtained from each training round is retained as the updated pathological slide analysis model.

[0064] A third aspect of the present invention provides an electronic device comprising:

[0065] At least one processor; and a memory communicatively connected to said at least one processor;

[0066] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to perform the above-described online learning-based pathological slide image analysis method.

[0067] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the above-described online learning-based pathological slide image analysis method.

[0068] The features and beneficial effects of this invention are as follows:

[0069] This invention combines online learning technology with intelligent pathological slide analysis technology. It uses new pathological slide images acquired in actual use to train the pathological slide analysis model online, realizing real-time updates and dynamic optimization of the model. This results in a model with stronger generalization performance, significantly improving the accuracy and efficiency of pathological image analysis.

[0070] This invention not only retains the analytical capabilities of existing models while learning new pathological slide analysis knowledge, but also accelerates the analysis process, reduces the workload of pathologists, and improves work efficiency. When doctors find slides with poor recognition performance, they can select a small number of well-annotated areas from the overall large slide, use these areas to train the model online, and then use the trained model to re-identify the previously poorly performing slides.

[0071] Overall, this invention has significant advantages and application prospects in improving the accuracy and stability of pathological slide image analysis, reducing misdiagnosis rates, improving the efficiency of pathological diagnosis, enhancing the adaptability and stability of the model, and accelerating the clinical analysis process. Attached Figure Description

[0072] Figure 1 This is an overall flowchart of a pathological slide image analysis method based on online learning, according to an embodiment of the present invention. Detailed Implementation

[0073] This invention proposes a method and system for pathological slide image analysis based on online learning, which will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] A first aspect of this invention proposes a method for analyzing pathological slide images based on online learning, comprising:

[0075] The newly acquired pathological slide image is input into the current pathological slide analysis model, and the model outputs the pathological slide lesion segmentation result of the image;

[0076] The pathological slide lesion segmentation results are identified, wherein regions with better identification results are selected from the pathological slide lesion segmentation results with poor overall identification results and corrected. Then, the local image of the corrected part in the corresponding pathological slide image and the corrected segmentation results are combined to form an online learning sample.

[0077] The current pathological slide analysis model is learned online using the online learning samples to obtain an updated pathological slide analysis model.

[0078] The current pathological slide analysis model and the updated pathological slide analysis model are weighted and fused to obtain a new current pathological slide analysis model.

[0079] Using the new current pathological slide analysis model, the pathological slide lesion segmentation results of the pathological slide image to be analyzed are generated.

[0080] In a specific embodiment of the present invention, the overall process of the pathological slide image analysis method based on online learning is as follows: Figure 1 As shown, it includes the following steps:

[0081] 1) Construct a training set and a validation set of pathological slide images; the specific steps are as follows:

[0082] 1-1) Obtain pathological slide images to form a pathological slide image dataset.

[0083] In this embodiment, the pathological slide image dataset contains pathological slide images corresponding to various pathological types, including but not limited to cancer, inflammation, and benign lesions. The number of pathological slide images for each type should be greater than 100. All pathological slide images in the dataset have the same size; in one specific embodiment of this invention, they are all 512×512 pixels to ensure consistency and standardization when inputting into the model.

[0084] In one specific embodiment of the present invention, a pathological slide image dataset was composed of a total of 20,000 pathological slide images collected from multiple pathology laboratories and medical institutions. To ensure the diversity and representativeness of the dataset, these images cover different pathological features, patient backgrounds, and sample sources.

[0085] Furthermore, in this embodiment, the following points are particularly emphasized during the process of collecting slice images:

[0086] The dataset includes a wide variety of pathological types, including cancers (such as breast cancer, lung cancer, and gastric cancer), with segmentation primarily focusing on the cancerous region and its boundaries; inflammations (such as hepatitis and gastritis), with segmentation primarily focusing on the inflammatory response region and its boundaries; and benign lesions (such as benign tumors and cysts), with segmentation primarily focusing on the lesion region and its boundaries.

[0087] The sample sources are diverse. This embodiment collects samples from different geographical regions and medical institutions, covering different patient groups and pathological characteristics, to ensure the broad representativeness of the dataset.

[0088] In addition, this embodiment selects high-quality pathological slide images to ensure image clarity and detail, so that the model can learn more useful information.

[0089] 1-2) Label each pathological slide image in the pathological slide image dataset obtained in step 1-1).

[0090] In this embodiment, the annotation content of each pathological slide image includes the polygonal boundary of the lesion area and the corresponding lesion type label. This annotation data will be used for subsequent model training and validation.

[0091] In one specific embodiment of the present invention, the annotation process includes: preliminary examination, where pathology experts first conduct a preliminary examination of each pathological slide image to determine the presence and approximate location of the lesion area; detailed annotation, where professional annotation tools are used to perform detailed polygon annotation on each lesion area to accurately delineate the lesion boundary; type annotation, where each annotated lesion area is assigned a corresponding lesion type label, such as cancer, inflammation, or benign lesion; and verification, where all annotation results are verified multiple times to ensure the accuracy and consistency of the annotation and avoid mis-annotation or omission.

[0092] 1-3) Preprocess the images in the pathological slide image dataset after annotation in step 1-2).

[0093] In this embodiment of the invention, the preprocessing includes: normalizing the image pixel values ​​to the range of [0,1] to improve the training effect and convergence speed of the model.

[0094] 1-4) Divide the dataset that has been preprocessed in steps 1-3) into a training set and a validation set.

[0095] If the present invention must retain the validation set, what is the ratio between the training set and the validation set?

[0096] In one specific embodiment of the present invention, 15,000 images are randomly selected from a dataset of 20,000 pathological slide images to form a training set, and the remaining 5,000 images form a validation set. Generally, the ratio of the training set to the validation set can be 3:1.

[0097] Optionally, this embodiment can also perform data augmentation on the pathological slide image dataset in the training set, such as random cropping, rotation, and flipping, to increase the diversity of the training data and improve the robustness of the model. These preprocessing steps ensure that the pathological slide image data has a consistent format and quality when input into the model, thereby improving the training effect and performance of the model.

[0098] 2) Construct a pathological slide analysis model.

[0099] In this embodiment, the existing pre-trained DinoV2 model is used as the main body of the pathological slide analysis model. A MaskFormer task header is added to the output layer of the DinoV2 model, and the pathological slide analysis model is completed.

[0100] It's worth noting that DinoV2 is an advanced self-supervised visual representation learning model that, through training on large-scale image datasets, can extract rich image features. DinoV2 primarily utilizes Vision Transformers (ViT), also known as visual attention methods, as its architectural foundation. Through a mechanism called "self-attention," it efficiently processes and understands the global content of an image. During the self-supervised learning process, DinoV2 employs various data augmentation techniques, such as cropping, color perturbation, and Gaussian blur, to generate multiple views of the image and allow the model to learn the relationships between these views, thereby obtaining a deeper and more robust image representation.

[0101] In this embodiment, in the pathological slide analysis model, the input of DinoV2 is a preprocessed 512×512 pixel pathological slide image, and the output is a high-dimensional image feature vector, typically a 16×16×768 feature map. This feature map is generated by DinoV2 through its deep visual feature extraction mechanism and contains rich information from the input image.

[0102] MaskFormer is an advanced image segmentation model specifically designed to handle complex image segmentation tasks. In this embodiment, the input to MaskFormer is the high-dimensional image feature vector output by DinoV2, and the output is a segmentation mask for each pixel in the pathological slide image. Specifically, the MaskFormer task head consists of three convolutional layers, each with a 3×3 kernel and eight self-attention heads. Each self-attention head can capture different feature dimensions of the image, enhancing the model's segmentation capability. The final softmax operation in the MaskFormer task head generates the classification probability for each pixel, ensuring that the output segmentation mask has accurate classification results to complete the lesion identification task in the pathological slide.

[0103] 3) Train the pathological slide analysis model constructed in step 2) using the training set obtained in step 1); the specific steps are as follows:

[0104] 3-1) Pre-trained model in the frozen pathological section analysis model, training task head.

[0105] In this embodiment, the pre-training parameters of the DinoV2 pathological slide analysis model are frozen to ensure that these parameters remain unchanged during this training phase. Freezing the pre-trained model's parameters preserves its feature extraction capabilities learned on large-scale datasets, preventing the degradation of these features during initial training. Training only the task head's parameters allows the task head to focus on learning specific features of pathological slide images. This allows the model to leverage the rich features learned during pre-training to enhance the task head's segmentation capabilities. The advantage of this method is that it fully utilizes the powerful feature extraction capabilities of the pre-trained model while focusing on optimizing the task head, thus improving the overall model's segmentation performance.

[0106] Then, the MaskFormer task head is trained using the pathological slide image training set obtained in step 1). During training, the same or similar loss function and optimization algorithm as the pre-trained model are used to optimize the parameters of the task head through backpropagation. Appropriate training parameters are set, including learning rate, training period, batch size, etc., to ensure that the task head converges stably during training.

[0107] In a specific embodiment of the present invention, the specific steps for training the MaskFormer task header are as follows:

[0108] First, the parameters of the MaskFormer task head are randomly initialized, setting the parameters required for training, including the learning rate, training period, and batch size. In one specific embodiment of this invention, the learning rate is set to 0.001, the training period is set to 50 epochs, and the batch size is set to 32. The cross-entropy loss function and the Adam optimizer are selected, and the parameters of the MaskFormer task head are optimized using the backpropagation algorithm. The cross-entropy loss function is suitable for classification tasks and can effectively measure the difference between the predicted result and the actual label, while the Adam optimizer ensures stability and fast convergence during training through adaptive learning rate adjustment.

[0109] Then, the MaskFormer task head is trained using a training set of pathological slide images. During training, the parameters of the task head are continuously adjusted to enable it to accurately segment lesion regions. Through multiple rounds of iterative training, the task head gradually learns how to identify and segment lesion regions on high-dimensional feature maps, thereby improving segmentation accuracy.

[0110] Finally, monitor the loss value and validation set performance in real time during training to ensure the stability and effectiveness of the training. When the loss value on the validation set no longer decreases significantly, consider ending the training. Monitoring the training process helps to identify overfitting in a timely manner and optimize model performance by adjusting training parameters. For example, if the initial learning rate is 0.001, it can be adjusted to 0.002 if the training process is too slow, and to 0.0005 if the training fails to converge.

[0111] Through the above steps, the MaskFormer task head can fully learn the specific features of pathological slide images based on the frozen DinoV2 pre-trained model, laying a solid foundation for subsequent full model fine-tuning. This process not only improves the model's segmentation ability but also ensures the model's accuracy and stability when processing complex pathological slide images.

[0112] Once the MaskFormer task header has been trained, proceed to step 3-2.

[0113] 3-2) Thaw the pre-trained model, and then fine-tune the pathological slide analysis model; specifically including:

[0114] Unfreeze the pre-trained parameters of the DinoV2 model so that the parameters of all layers of the DinoV2 model can be updated during subsequent training.

[0115] Set the parameters needed for fine-tuning the training, including the learning rate, training epochs, and batch size. The learning rate should be set low during the fine-tuning phase to avoid large perturbations to the learned features. A commonly used learning rate is 0.0001, the training epochs can be set to 20, and the batch size can be kept at 32. Choose the cross-entropy loss function and the Adam optimizer, and optimize the parameters of the entire model using the backpropagation algorithm.

[0116] It is important to note that the learning rate setting is crucial during the fine-tuning training phase. A common approach is to gradually decrease the learning rate, starting with a higher rate initially to quickly adjust model parameters; as training progresses, the learning rate is gradually reduced to refine adjustments to larger model parameters. This method ensures stable convergence of the pathological slide analysis model during fine-tuning, avoiding training oscillations or overfitting.

[0117] Through the above steps, the pre-trained model and the task head can be jointly optimized in pathological slide image segmentation tasks, improving the overall model performance. The fine-tuned pathological slide analysis model not only has powerful feature extraction capabilities but also high-precision segmentation capabilities for pathological slide images, thus meeting practical application needs. The method described in this embodiment, by fully utilizing the existing knowledge of the pre-trained model and combining it with specific optimizations of the task head, can significantly improve the overall performance and adaptability of large models, providing reliable technical support for practical pathological slide image analysis.

[0118] After fine-tuning, the final offline-trained pathological slide analysis model for pathological slides is obtained.

[0119] 3-3) Further, optionally, this embodiment can also optimize the model by evaluating the trained pathological slide analysis model; the specific steps are as follows:

[0120] 3-3-1) Evaluate model performance;

[0121] After fine-tuning the large model, the next step is to comprehensively evaluate it to ensure its performance in the pathological slide image segmentation task meets expectations. The evaluation process includes measuring the model's segmentation accuracy and robustness using validation and test sets. Key evaluation metrics include Intersection over Union (IoU), Pixel Accuracy (PA), and Dice coefficient, which comprehensively reflect the model's performance in the segmentation task. Specifically, these include:

[0122] First, a separate test set is prepared to ensure that the images in the test set are not used for training and validation of the pathological slide analysis model, thus guaranteeing the objectivity of the evaluation results. Then, the pathological slide images from the test set are input into the trained pathological slide analysis model to generate segmentation results for each pathological slide image in the test set. Based on these segmentation results, key evaluation metrics of the pathological slide analysis model on the test set are calculated, including Intersection over Union (IoU), Pixel Accuracy (PA), and Dice coefficient.

[0123] Intersection over Union (IoU) measures the overlap between the predicted and actual segments, Pixel Accuracy (PA) measures the proportion of correctly classified pixels in the prediction, and the Dice coefficient evaluates the consistency between the predicted and actual segments. By combining these evaluation metrics, a comprehensive understanding of the model's segmentation performance can be obtained.

[0124] 3-3-2) Optimize and adjust the model;

[0125] Based on the evaluation results of step 3-3-1), the model is further optimized and adjusted to improve its performance in pathological slide image segmentation tasks. The optimization process includes adjusting model parameters, improving training strategies, and redesigning data augmentation methods. The goal is to gradually improve the model's segmentation accuracy and robustness through continuous iteration. Specifically, this includes:

[0126] First, analyze the evaluation results to identify the model's strengths and weaknesses. For example, if the model performs poorly on certain types of lesions, consider increasing the amount of training data for those types of lesions or performing specific parameter tuning for those types of lesions.

[0127] Then, adjust the model's hyperparameters, including learning rate, batch size, and regularization coefficient, and conduct multiple experiments to find the optimal parameter combination. Methods such as grid search or random search can be used to systematically explore the effects of different parameter combinations. Furthermore, consider improving training strategies, such as using more advanced optimization algorithms, introducing new loss functions, or increasing the diversity of training data.

[0128] Next, data augmentation methods can be improved to increase the diversity of training data and the robustness of the model. For example, more types of image transformation techniques can be introduced, such as color jitter, blurring, and noise, so that the model can maintain high performance when faced with different image changes.

[0129] Finally, further fine-tuning training is performed, gradually improving the model's segmentation performance by incorporating new data and optimization strategies. After each training iteration, the model's performance is re-evaluated, and the next round of optimization is conducted based on the evaluation results.

[0130] For example, in this embodiment, the pathological slide analysis model obtained in step 3-2) is optimized. First, independent test set pathological slide data is input into the model for lesion identification and segmentation. The lesion segmentation results are compared with the actual labeled results to calculate the pixel-level classification accuracy. Assuming that the average classification accuracy of different categories is 80%, if we want the model to better complete the identification task and achieve a higher accuracy, such as above 90%, then the model needs to be adjusted. First, we need to find out which type of lesion has a low recognition accuracy for the large model and increase the weight of the corresponding category in the loss function so that the large model can better learn the characteristics of that type of lesion. For example, if the classification rate of cell edema is low, the weight corresponding to edema in the loss function can be doubled. In addition, the number of different types of lesions can be counted. If the number of a certain type of lesion is too small, the large model will not be able to learn the characteristics of that type of lesion, resulting in poor classification performance. For example, if there are too few samples of cell proliferation lesions, data augmentation is needed for that type of sample. For example, the images of that type of sample can be distorted, rotated, or noise added to increase the number of samples of that type. Then, the large model can be trained. After the training is completed, the recognition accuracy is retested to continue to find parts that can be optimized.

[0131] In this embodiment, the pathological slide analysis model, after fine-tuning and optimization, has good results in pathological slide feature extraction and encoding, and can be applied to practical tasks such as pathological image analysis and diagnosis.

[0132] 4) Use the pathological slide analysis model trained in step 3) to process the newly acquired pathological slide images.

[0133] In this embodiment, a new pathological slide image is acquired and adjusted to 512×512 pixels. The adjusted pathological slide image is then input into the pathological slide analysis model trained in step 3). The model outputs a 512×512 pixel pathological slide lesion segmentation result as the prediction label for the corresponding input image.

[0134] Furthermore, if the pathologist determines that the overall recognition effect is poor after identifying the lesion segmentation result of the pathological slide (in this embodiment, the judgment criteria are more than 10 unmarked lesions or more than 10 normal tissues that are incorrectly marked as lesions), the pathologist can manually select the area with better effect in the lesion segmentation result of the pathological slide (in this embodiment, the judgment criteria are no more than 2 unmarked lesions or no more than 2 normal tissues that are incorrectly marked as lesions) and correct it (i.e., manually mark the missed lesions and remove the incorrectly marked normal tissues). Then, the local image corresponding to the corrected part in the pathological slide image and the corrected segmentation result (i.e., the true label) are combined to form a new online learning sample.

[0135] In this embodiment, when the number of images with poor recognition performance reaches a set number, all generated online learning samples are used to enter the subsequent online learning stage.

[0136] 5) Utilize online learning to adjust the pathological slide analysis model.

[0137] Online learning is a technique that learns and optimizes predictive models from sequentially arriving data stream instances. It dynamically learns from and adjusts the model based on samples in the data stream. This dynamic learning approach makes online learning more scalable and more memory efficient, while retaining a certain degree of transferability for the model. For pathological slides, the styles of pathological slide images obtained from different staining reagents and scanning instruments can vary significantly, which can pose a great challenge to the lesion identification task of the pathological slide analysis model. Therefore, it is necessary to adjust and optimize the model using sample data from actual use to make the lesion identification and slide analysis results more accurate.

[0138] In this embodiment, the batch of online learning samples generated after step 4) is denoted as D. new Online learning is achieved using a confidence-weighted average method. The confidence score of each labeled region in each sample is calculated. The confidence score of each labeled region in a local image of one of the online learning samples can be calculated by the following expression:

[0139]

[0140] Where i is the index of the labeled region in the current local image, and y i,c For the i-th labeled region, Let C be the predicted label for the i-th labeled region, C be the total number of lesion categories (3 in this embodiment), and c be the lesion category index (1 for lesion villi, 2 for edema, and 3 for hyperplasia in this embodiment).

[0141] After calculating the confidence level, D new The local image in the image and the corresponding lesion label image output by the current pathological slide analysis model are denoted as follows: where x i For the i-th pathological slide, y i For the corresponding lesion label image, n is the number of samples, and D new The local image is then input again into the current pathological slide analysis model for forward propagation to obtain the output result. The cross-entropy loss function is used to calculate the error between the predicted result and the true label. The expression for the loss function is as follows:

[0142]

[0143] Then, backpropagation and model weight updates are performed. This process uses a confidence-weighted average method to update the model weights. The weight update calculation expression is as follows:

[0144]

[0145] Where, θ t The model parameters θ were updated before the weights were adjusted. t+1 Here, η represents the model parameters after one weight update, η is the learning rate (which should be a small value, such as 0.0001), and n is the labeled region.

[0146] In this embodiment, the training cycle is set to repeat training 5 times, the batch size is set to 1, and this batch of data D is used. new The model is trained and the model obtained from each training session is saved. The model after 5 training sessions is used to re-identify the pathological slide images with poor recognition results. The recognition result with the highest confidence is selected and submitted to the pathologist for review and judgment. If the recognition result is good (the judgment criterion in this embodiment is that there are no more than 2 unmarked lesions or no more than 2 normal tissues that are incorrectly marked as lesions in the local image), the next step is performed. Otherwise, this step is repeated until the ideal recognition effect is achieved.

[0147] 6) A multi-model voting mechanism is used. The model before adjustment and the model saved from each round of training during adjustment are used to identify the same pathological slide image, generating multiple recognition results. The recognition results of multiple models are voted on, and the multiple models are weighted and fused according to the confidence of the voting results. This allows the method described in this embodiment to retain the analytical capabilities of the existing model while learning new pathological slide analysis knowledge, ensuring a smooth transition of model updates and avoiding functional fluctuations.

[0148] Specifically, let the model before the single adjustment using online learning be the initial model, denoted as M0, and the parameters in this model be denoted as θ0; let the model after the adjustment using online learning (step 5) be the fine-tuned model, denoted as M1 to M5 respectively in a specific embodiment of the present invention, and the parameters in this model be denoted as θ1 to θ5.

[0149] The expression for retrieving the voting results is as follows:

[0150]

[0151] Among them, v m,c Let I be the voting result of the m-th model for pathological category c, I be the total number of labeled regions in the current slice, i be the index of the labeled region, and l be the index of the labeled region. m,i,c Let l be the determination result made by the m-th model in the i-th labeled region for pathological category c. That is, if the labeled region belongs to pathological category c, then lm,i,c =1, otherwise, l m,i,c =0.

[0152] The fusion weights for each model are determined based on the voting results. Models with higher confidence levels in the voting results have larger weights, and vice versa. The weight calculation expression is as follows:

[0153]

[0154] Among them, w m Let v be the weight of the m-th model. m,c For the voting results, y m,c Let be the confidence level of the m-th model for pathological category c.

[0155] Then, based on the calculated weights, the models before and after adjustment are fused together. The fusion expression is as follows:

[0156]

[0157] Where, θ fusion Here, M represents the parameters of the fused model, M represents the total number of training epochs when using online learning for a single adjustment, and θ represents the parameters of the fused model. m For the parameters of the m-th model, w m Let M0 be the weight of the m-th model. M0 is the pathological slide analysis model before a single adjustment using online learning, and the parameters of model M0 are θ0; M1 to M... m The pathological slide analysis models saved during each round of online learning training, models M1 to M... m The parameters are θ1 to θ m This method can balance the learning of new knowledge and the retention of existing knowledge during the online learning process of the pathological slide analysis model, achieving smooth model updates and stable performance improvements.

[0158] 7) Save the fused pathological slide analysis model obtained in step 6) as the new current pathological slide analysis model, and then return to step 4).

[0159] The method described in this embodiment enables the pathological slide analysis model to be continuously iterated and optimized, thereby achieving better performance and greater stability.

[0160] To implement the above embodiments, a second aspect of the present invention proposes a pathological slide image analysis device based on online learning, comprising:

[0161] The preliminary pathological section analysis module is used to input newly acquired pathological section images into the current pathological section analysis model, and the model outputs the pathological section lesion segmentation results of the image;

[0162] The online learning sample construction module is used to identify the lesion segmentation results of the pathological slides. It selects regions with better identification results from the lesion segmentation results of the pathological slides with poor overall identification results and corrects them. Then, the local image of the corrected part in the corresponding pathological slide image and the corrected segmentation results are combined to form an online learning sample.

[0163] The online learning module is used to learn the current pathological slide analysis model online using the online learning samples to obtain an updated pathological slide analysis model.

[0164] The fusion module is used to perform weighted fusion of the current pathological slide analysis model and the updated pathological slide analysis model to obtain a new current pathological slide analysis model.

[0165] The pathological slide final analysis module is used to generate pathological slide lesion segmentation results of the pathological slide image to be analyzed using the new current pathological slide analysis model.

[0166] It should be noted that the foregoing explanation of an embodiment of a pathological slide image analysis method based on online learning also applies to a pathological slide image analysis device based on online learning in this embodiment, and will not be repeated here. According to an embodiment of the present invention, a pathological slide image analysis device based on online learning involves inputting a newly acquired pathological slide image into a current pathological slide analysis model, which outputs the pathological slide lesion segmentation result of the image; identifying the pathological slide lesion segmentation result, wherein regions with better identification results are selected from the pathological slide lesion segmentation results with poor overall identification performance and corrected, and then the local image of the corrected portion in the corresponding pathological slide image and the corrected segmentation result are combined to form an online learning sample; the current pathological slide analysis model is trained online using the online learning sample to obtain an updated pathological slide analysis model; the current pathological slide analysis model and the updated pathological slide analysis model are weighted and fused to obtain a new current pathological slide analysis model; and the pathological slide lesion segmentation result of the pathological slide image to be analyzed is generated using the new current pathological slide analysis model. This allows for real-time retraining and optimization of offline-trained pathological slide analysis models by utilizing feedback from doctors and actual data during use, thereby improving their recognition accuracy and stability, promoting the development of pathological diagnosis towards intelligence and automation, and ultimately improving the accuracy and efficiency of clinical analysis.

[0167] To implement the above embodiments, a third aspect of the present invention provides an electronic device, comprising:

[0168] At least one processor; and a memory communicatively connected to said at least one processor;

[0169] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to perform the above-described online learning-based pathological slide image analysis method.

[0170] To implement the above embodiments, a fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the above-described online learning-based pathological slide image analysis method.

[0171] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0172] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform a pathological slide image analysis method based on online learning according to the above embodiments.

[0173] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0174] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0175] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0176] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0177] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0178] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0179] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0180] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0181] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for analyzing pathological slide images based on online learning, characterized in that, include: The newly acquired pathological slide image is input into the current pathological slide analysis model, and the model outputs the pathological slide lesion segmentation result of the image; The pathological slide lesion segmentation results are identified, wherein regions with better identification results are selected from the pathological slide lesion segmentation results with poor overall identification results and corrected. Then, the local image of the corrected part in the corresponding pathological slide image and the corrected segmentation results are combined to form an online learning sample. The current pathological slide analysis model is learned online using the online learning samples to obtain an updated pathological slide analysis model. The current pathological slide analysis model and the updated pathological slide analysis model are weighted and fused to obtain a new current pathological slide analysis model. Using the new current pathological slide analysis model, the pathological slide lesion segmentation results of the pathological slide image to be analyzed are generated; The step of using the online learning samples to perform online learning on the current pathological slide analysis model to obtain an updated pathological slide analysis model includes: Let D be a batch of online learning samples. new The confidence score of each labeled region in each sample is calculated. The expression for calculating the confidence score of each labeled region in a local image of one of the online learning samples is as follows: Where i is the index of the labeled region in the current local image, and y i,c For the i-th labeled region, Let be the predicted label for the i-th labeled region, C be the total number of lesion categories, and c be the lesion category index; After calculating the confidence level, D new The local image in the image and the corresponding lesion label image output by the current pathological slide analysis model are denoted as follows: Where x i For a partial image of the i-th pathological slide, y i For the corresponding lesion label image, n is the number of samples; D new The local image is then input again into the current pathological slide analysis model for forward propagation to obtain the output result. The cross-entropy loss function is used to calculate the error between the predicted result and the true label. The expression for the loss function is as follows: Backpropagation and model weight updates are performed based on the loss function calculation results. The mini-batch stochastic gradient descent method is used to update the model weights, and the weight update calculation expression is as follows: Where, θ t The model parameters θ were updated before the weights were adjusted. t+1 Here are the model parameters after one weight update, and η is the learning rate; The model obtained from each training round is saved separately. When the online learning reaches the preset number of training rounds, the model saved after each training round is used to re-identify the pathological slide images with poor recognition results. If the recognition results meet the preset requirements, the model obtained from each training round is retained as the updated pathological slide analysis model.

2. The method according to claim 1, characterized in that, Also includes: The initial current pathological slide analysis model was obtained through offline training, specifically including: Construct a training set and a validation set for pathological slide images; A pathological slide analysis model is constructed, which consists of a pre-trained model and additional task heads. Freeze the pre-trained model, train the task head using the training set and verify it using the validation set to obtain the task head that has been initially trained; Unfreeze the pre-trained model, fine-tune the pathological slide analysis model using the training set, and verify it using the validation set to obtain the final offline-trained pathological slide analysis model.

3. The method according to claim 2, characterized in that, The pathological slide analysis model uses a pre-trained DinoV2 model as the main body, and adds a MaskFormer task header to the output layer of the DinoV2 model.

4. The method according to claim 2, characterized in that, The construction of the training and validation sets for pathological slide images includes: 1) Obtain pathological slide images to form a pathological slide image dataset; The pathological slide image dataset contains pathological slide images corresponding to various pathological types, wherein each pathological slide image is of the same size. 2) Label each pathological slide image in the pathological slide image dataset of step 1); wherein, the labeling content of each pathological slide image includes the polygonal boundary of the lesion area and the corresponding lesion type label. 3) Preprocess the images in the pathological slide image dataset after step 2) annotation, and normalize the pixel values ​​of each image to the range of [0,1]. 4) The pathological slide dataset that has been preprocessed in step 3) is randomly divided into a training set and a validation set according to a preset ratio.

5. The method according to claim 2, characterized in that, Also includes: The performance of the pathological slide analysis model trained offline is evaluated using a preset test set. Based on the evaluation results, the pathological slide analysis model is adjusted to obtain an optimized pathological slide analysis model. The pathological slide images in the test set are different from those in the training set and the validation set.

6. The method according to claim 2, characterized in that, Also includes: Using a multi-model voting mechanism, the same pathological slide image is identified using the pathological slide analysis model before adjustment and the pathological slide analysis model saved from each round of training, generating multiple identification results. The recognition results of multiple models are voted on, and the multiple models are weighted and fused based on the confidence level in the voting results; The expression for obtaining the voting results is as follows: Among them, v m,c Let I be the voting result of the m-th model for pathological category c, I be the total number of labeled regions in the current slice, i be the index of the labeled region, and l be the index of the labeled region. m,i,c Let l be the determination result made by the m-th model in the i-th labeled region for pathological category c. That is, if the labeled region belongs to pathological category c, then l m,i,c =1, otherwise, l m,i,c =0; The fusion weights for each model are determined based on the voting results. The weight calculation expression is as follows: Among them, w m Let v be the weight of the m-th model. m,c For the voting results, y m,c Let be the confidence level of the m-th model for pathological category c; Then, based on the calculated weights, the models before and after adjustment are fused together. The fusion expression is as follows: Where, θ fusion Here, M represents the parameters of the fused model, M represents the total number of training epochs when using online learning for a single adjustment, and θ represents the parameters of the fused model. m For the parameters of the m-th model, w m Let M1 be the weight of the m-th model; M0 is the pathological slide analysis model before a single adjustment using online learning, and the parameter of model M0 is θ0; M1 to M... m The pathological slide analysis models saved during each round of online learning training, models M1 to M... m The parameters are θ1 to θ m .

7. A pathological slide image analysis device based on online learning, characterized in that, include: The preliminary pathological section analysis module is used to input newly acquired pathological section images into the current pathological section analysis model, and the model outputs the pathological section lesion segmentation results of the image; The online learning sample construction module is used to identify the lesion segmentation results of the pathological slides. It selects regions with better identification results from the lesion segmentation results of the pathological slides with poor overall identification results and corrects them. Then, the local image of the corrected part in the corresponding pathological slide image and the corrected segmentation results are combined to form an online learning sample. The online learning module is used to learn the current pathological slide analysis model online using the online learning samples to obtain an updated pathological slide analysis model. The fusion module is used to perform weighted fusion of the current pathological slide analysis model and the updated pathological slide analysis model to obtain a new current pathological slide analysis model. The pathological slide final analysis module is used to generate pathological slide lesion segmentation results of the pathological slide image to be analyzed using the new current pathological slide analysis model. The step of using the online learning samples to perform online learning on the current pathological slide analysis model to obtain an updated pathological slide analysis model includes: Let D be a batch of online learning samples. new The confidence score of each labeled region in each sample is calculated. The expression for calculating the confidence score of each labeled region in a local image of one of the online learning samples is as follows: Where i is the index of the labeled region in the current local image, and y i,c For the i-th labeled region, Let be the predicted label for the i-th labeled region, C be the total number of lesion categories, and c be the lesion category index; After calculating the confidence level, D new The local image in the image and the corresponding lesion label image output by the current pathological slide analysis model are denoted as follows: Where x i For a partial image of the i-th pathological slide, y i For the corresponding lesion label image, n is the number of samples; D new The local image is then input again into the current pathological slide analysis model for forward propagation to obtain the output result. The cross-entropy loss function is used to calculate the error between the predicted result and the true label. The expression for the loss function is as follows: Backpropagation and model weight updates are performed based on the loss function calculation results. The mini-batch stochastic gradient descent method is used to update the model weights, and the weight update calculation expression is as follows: Where, θ t The model parameters θ were updated before the weights were adjusted. t+1 Here are the model parameters after one weight update, and η is the learning rate; The model obtained from each training round is saved separately. When the online learning reaches the preset number of training rounds, the model saved after each training round is used to re-identify the pathological slide images with poor recognition results. If the recognition results meet the preset requirements, the model obtained from each training round is retained as the updated pathological slide analysis model.

8. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method according to any one of claims 1-6.

Images