A method for training a medical image processing model and a medical image processing method

Abstract

The present application belongs to the technical field of medical image processing, and particularly relates to a medical image processing model training method and a medical image processing method. In the self-supervised pre-training of the MIM encoder, an image blurring operation is introduced, thereby constructing a pre-training proxy task of deblurring mask image modeling, so that the hidden representation learned in the pre-training stage contains additional attention to the details of the ultrasound image. After weight migration using the MIM encoder, the downstream task trained can also pay additional attention to the details in the medical image, so that better prediction performance can be obtained. Therefore, the present application has good application prospects in the processing of medical images.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to a training method for a medical image processing model and a medical image processing method. Background Technology

[0002] In recent years, deep learning-based ultrasound image analysis has made significant progress. Ultrasound imaging, as a non-invasive, radiation-free, cost-effective, and real-time imaging method, is widely used in clinical diagnosis, such as in the diagnosis of thyroid nodules and breast nodules. Deep learning is an artificial intelligence method that uses artificial neural networks as its architecture to learn representations from large amounts of labeled data to perform specific tasks. In the field of ultrasound imaging, this typically requires building a sufficiently large labeled dataset for large-scale supervised learning, which can then be used for tasks such as classification and segmentation. One major drawback of supervised learning is the difficulty in obtaining large amounts of labeled data and the high cost of data labeling, a drawback particularly pronounced in the field of medical imaging. For example, in the task of classifying benign and malignant tumors, the gold standard is the result of a biopsy, a process that is time-consuming, labor-intensive, and can have uncertain effects on patients. The lack of high-quality labeled data has constrained the development of deep learning-based ultrasound image analysis.

[0003] In recent years, generative self-supervised learning, represented by Masked Image Modelling (MIM), has become a research hotspot in computer vision within the field of deep learning. Self-supervised learning belongs to unsupervised machine learning and aims to learn useful data representations from large amounts of unlabeled data. Compared to supervised learning, self-supervised learning does not require manually labeled data, making it more efficient and flexible. The basic idea of ​​self-supervised learning is to extract certain implicit information or features from unlabeled data and then apply these features to downstream specific tasks, such as classification, detection, and segmentation. In self-supervised learning, tasks with labels that can be automatically obtained from the data are typically designed to guide the model's learning; these tasks are called proxy tasks, and the learning process for proxy tasks is called pre-training. The labels for proxy tasks do not require manual annotation and can be automatically obtained from the data. After the pre-training stage, the learned representations can be used for transfer learning to specific tasks. These specific tasks are called downstream tasks.

[0004] In self-supervised learning, techniques such as Masked Image Modelling (MIM) have achieved excellent performance in the field of Computer Vision (CV). The proxy task of MIM is masked image prediction; that is, during the pre-training phase, some image patches are randomly masked, and the remaining visible image patches are used to predict the masked patches. Representative methods based on MIM include Masked Autoencoders (MAE) and SimMIM.

[0005] Most current masked image modeling methods are based on natural images, while research on masked image modeling in medical images is relatively lacking. The imaging process of most medical images differs fundamentally from that of natural images. For example, ultrasound images are based on ultrasound imaging, while natural images are based on visible light imaging. Therefore, existing methods for processing natural images lack consideration for the imaging characteristics of medical images. Furthermore, for natural images, detailed features are not very important for downstream tasks such as image recognition and segmentation, while for medical images such as ultrasound images, detailed features are particularly important for these tasks. For example, microcalcifications are a crucial indicator for clinically differentiating the benign or malignant nature of thyroid nodules, and in ultrasound images, microcalcifications manifest as tiny bright spots. Current masked image modeling methods inherently lack focus on these details during the pre-training phase. Therefore, there is an urgent need in this field for a masked image modeling method suitable for the imaging characteristics of medical images. Summary of the Invention

[0006] To address the problems of existing technologies, this invention provides a training method for a medical image processing model and a medical image processing method. The purpose is to make the model more suitable for the imaging characteristics of medical images in masked image modeling, and to pay more attention to the detailed features of the image, thereby enabling the model to have better predictive performance.

[0007] A pre-training method for MIM encoders suitable for medical image processing includes the following steps:

[0008] Step 1, input the original medical image;

[0009] Step 2: Blur the image to obtain a blurred image;

[0010] Step 3: Perform a random mask to obtain a blurred masked image;

[0011] Step 4: Input the masked blurred image into the MIM model to complete the training of the MIM model; the MIM model includes a MIM encoder and a MIM decoder.

[0012] Preferably, the medical image is an ultrasound image.

[0013] Preferably, in step 2, the image blurring method is selected from at least one of the following methods: Gaussian blur, mean blur, or median blur.

[0014] This invention also provides a method for training a medical image processing model, comprising the following steps:

[0015] Step a, input the original medical image;

[0016] Step b: Blur the image to obtain a blurred image;

[0017] Step c, constructing a medical image processing model, the medical image processing model including a task head for implementing the prediction task and a MIM encoder trained according to any one of claims 1-3;

[0018] Step d: Input the blurred image into the medical image processing model to complete the training of the medical image processing model.

[0019] Preferably, the medical image is an ultrasound image.

[0020] Preferably, in step b, the image blurring method is selected from at least one of the following methods: Gaussian blur, mean blur, or median blur.

[0021] Preferably, the prediction task is selected from nodule benign / malignant classification, nodule region segmentation, nodule detection, or Hashimoto's thyroiditis classification.

[0022] The present invention also provides a medical image processing method for non-diagnostic purposes, comprising the following steps:

[0023] Step 1: Input the original medical image;

[0024] Step II: Blur the image to obtain a blurred image;

[0025] Step III: Input the blurred image into the medical image processing model to obtain the prediction result; the medical image processing model is trained according to the above training method.

[0026] The present invention also provides a medical image processing system, comprising:

[0027] The input module is used to input raw medical images;

[0028] The preprocessing module is used to blur the image and obtain a blurred image;

[0029] The prediction module is used to input the blurred image into the medical image processing model to obtain the prediction result; the medical image processing model is trained according to the above training method.

[0030] The output module is used to output the prediction results.

[0031] The present invention also provides a computer-readable storage medium storing: a computer program for implementing the above-described MIM encoder pre-training method suitable for medical image processing, or for implementing the above-described medical image processing model training method, or for implementing the non-diagnostic medical image processing method of claim 8.

[0032] This invention constructs a deblurred masked image modeling pre-training proxy task by blurring the original medical image (e.g., ultrasound image) during the self-supervised pre-training stage within a masked image modeling framework. This allows the latent representations learned during pre-training to incorporate additional attention to medical image details, thereby improving the accuracy of downstream tasks in medical image processing. Specifically, ultrasound imaging inherently possesses a high signal-to-noise ratio. Many image features relevant to specific diagnostic tasks are minute details; for example, in ultrasound-based classification of benign and malignant thyroid nodules, microcalcifications are a crucial indicator, appearing as tiny white dots in ultrasound images. Therefore, the deblurring proposed in this scheme encourages the pre-trained model to pay more attention to image detail features during pre-training, enhancing its representation extraction capability and enabling better generalization to downstream tasks, thus improving their performance.

[0033] Obviously, based on the above description of the present invention, and according to common technical knowledge and conventional methods in the field, various other modifications, substitutions or alterations can be made without departing from the basic technical concept of the present invention.

[0034] The following detailed embodiments further illustrate the above-described content of the present invention. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the above-described content of the present invention fall within the scope of the present invention. Attached Figure Description

[0035] Figure 1 The above are flowcharts of Examples 1 and 2. Detailed Implementation

[0036] It should be noted that the algorithms for data acquisition, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structures and circuit connections not specifically described, can all be implemented using content already disclosed in the prior art.

[0037] Example 1: MIM encoder pre-training method applicable to medical image processing

[0038] The purpose of this embodiment is to pre-train a MIM encoder that can be used for downstream tasks in medical image (ultrasound image processing in this embodiment) processing. The process is as follows: Figure 1 As shown, the specific steps are as follows:

[0039] The input raw ultrasound image x is first processed by an image blurring algorithm to obtain a blurred image x. b The image blurring algorithm in this embodiment selects Gaussian blur. Then, the blurred image x... b Some blocks in the image are randomly masked at a given mask rate, resulting in a masked blurred image. Next, the masked blurred image Input the MIM model. The MIM model receives the input masked blur image. Output reconstructed image The loss function in the pre-training phase is the reconstructed image. The reconstruction loss between the original ultrasound image and the original ultrasound image, i.e., the mean square error (MSE) loss: The MIM model consists of an encoder and a decoder. The encoder receives a masked and blurred image as input and extracts latent representations from it. These latent representations are then input into the decoder, which outputs a reconstructed image based on them. Once the pre-training phase is complete, the MIM decoder is no longer needed, and the learned MIM encoder becomes the latent representation extractor, which can be used in the downstream task transfer learning phase.

[0040] Example 2: Training Method of Medical Image Processing Model

[0041] The purpose of this embodiment is to train a medical image processing model to perform a prediction task for medical images. This medical image processing model includes the MIM encoder obtained in Embodiment 1 and a task head for a specific prediction task. The specific process is as follows: Figure 1 As shown, it includes the following steps:

[0042] The original ultrasound image x is first processed by an image blurring algorithm to obtain a blurred image x. b The image blurring algorithm in this embodiment selects Gaussian blur. The blurred image x... b The medical image processing model is trained by inputting the data. The transfer learning stage in this embodiment is supervised learning, and the types of tasks include, but are not limited to, image classification and image segmentation. Figure 1 The text lists two typical tasks for ultrasound imaging: nodule classification and nodule segmentation.

[0043] The image blurring algorithm used in this invention can encompass various image blurring, image denoising, and image smoothing algorithms, such as Gaussian blur, mean blur, and median blur. The Gaussian blur algorithm is illustrated below.

[0044] Gaussian blur is a common and widely used blurring algorithm in image processing to reduce noise or smooth images. It uses a Gaussian function to generate a kernel, which is then used to perform convolution calculations on the image to obtain the blurred image. It can be represented as:

[0045]

[0046] Where x represents the original input image, x b G(σ) represents the blurred image, where σ represents the standard deviation, which determines the degree of blurring of the blurred image. * represents the convolution operation, and (u,v) represents the coordinates of the Gaussian convolution kernel.

[0047] Example 3: Medical Image Processing Method and System

[0048] The system in this embodiment includes:

[0049] The input module is used to input raw medical images;

[0050] The preprocessing module is used to blur the image and obtain a blurred image;

[0051] The prediction module is used to input the blurred image into the medical image processing model obtained in Example 2 to obtain the prediction result;

[0052] The output module is used to output the prediction results.

[0053] The method for medical image processing using the above system includes the following steps:

[0054] Step 1: Input the original medical image;

[0055] Step II: Blur the image to obtain a blurred image;

[0056] Step III: Input the blurred image into the medical image processing model to obtain the prediction result.

[0057] The technical solution of the present invention will be further explained below through experimental data.

[0058] Performance comparison of different models in Experiment Example 1

[0059] I. Experimental Methods

[0060] This experiment uses a dataset collected by West China Hospital to perform nodule benign / malignant classification and nodule segmentation tasks. The model used is as follows:

[0061] MAE: Masked Autoencoders (MAE) model in the prior art;

[0062] Deblurring MAE: Train the MAE model according to the methods in Examples 1 and 2.

[0063] II. Experimental Results

[0064] The following experimental results are based on a nodule benign / malignant classification task:

[0065]

[0066] The following experimental results are based on the nodule segmentation task (using UpperNet as the segmentation model):

[0067]

[0068] The results above show that adding fuzzy operations to MAE can effectively improve the accuracy of nodule benign / malignant classification and nodule segmentation.

[0069] Experiment Example 2: The Impact of Fuzzy Methods on Model Performance

[0070] I. Experimental Methods

[0071] The experimental dataset was collected from West China Hospital and contains 4493 thyroid ultrasound images, of which 2576 images contain benign nodules and 1917 images contain malignant nodules. Results for malignant nodules were obtained from pathological biopsies, while results for benign nodules were obtained from clinical and pathological diagnoses. The dataset was randomly divided into training, validation, and test sets in a 3:1:1 ratio.

[0072] This experiment uses a dataset collected by West China Hospital to perform nodule benign / malignant classification tasks. The models used in each experimental group are as follows:

[0073] Unambiguous: Masked Autoencoders (MAE) models in the prior art;

[0074] Gaussian blur: The MAE model was trained according to the methods in Examples 1 and 2, and the blurring method was Gaussian blur;

[0075] Mean blurring: The MAE model was trained according to the methods in Examples 1 and 2, and the blurring method was Gaussian blurring;

[0076] Median fuzzing: The MAE model was trained according to the methods in Examples 1 and 2, and the fuzzing method was Gaussian fuzzing.

[0077] II. Experimental Results

[0078]

[0079] Based on the table above, we can see that Gaussian blur has the best effect, followed by mean blur, while median blur has almost no effect.

[0080] As can be seen from the above embodiments, this invention addresses the imaging characteristics of medical images and the need for additional attention to detailed features. By introducing an image blurring operation within the masked image modeling framework, it enables the pre-training stage to learn image deblurring. The medical image processing model constructed according to the method of this invention is more suitable for medical images and can better focus on the detailed features in medical images, resulting in better predictive performance. Therefore, this invention has excellent application prospects.

Claims

1. A training method for a medical image processing model, characterized in that, Includes the following steps: Step a, input the original medical image; Step b: Blur the image to obtain a blurred image; Step c, construct a medical image processing model, which includes a task head and a MIM encoder for implementing the prediction task; Step d: Input the blurred image into the medical image processing model to complete the training of the medical image processing model; The MIM encoder was trained using the following method: Step 1, input the original medical image; Step 2: Blur the image to obtain a blurred image; Step 3: Perform a random mask to obtain a blurred masked image; Step 4: Input the masked blurred image into the MIM model to complete the training of the MIM model; the MIM model includes a MIM encoder and a MIM decoder; The medical image is an ultrasound image, and the image blurring method is selected from Gaussian blurring.

2. The training method according to claim 1, characterized in that: The prediction task is selected from nodule benign / malignant classification, nodule region segmentation, nodule detection, or Hashimoto's thyroiditis classification.

3. A medical image processing method for non-diagnostic purposes, characterized in that, Includes the following steps: Step 1: Input the original medical image; Step II: Blur the image to obtain a blurred image; Step III: Input the blurred image into the medical image processing model to obtain the prediction result; the medical image processing model is trained according to the training method described in claim 1 or 2.

4. A medical image processing system, characterized in that, include: The input module is used to input raw medical images; The preprocessing module is used to blur the image and obtain a blurred image; The prediction module is used to input the blurred image into the medical image processing model to obtain the prediction result; The medical image processing model is trained according to the training method described in claim 1 or 2; The output module is used to output the prediction results.

5. A computer-readable storage medium, characterized in that, It stores: a training method for implementing the medical image processing model of claim 1 or 2, or a computer program for implementing the medical image processing method for non-diagnostic purposes of claim 3.

Images